5 - Sri Doddapaneni of the TPTP Project

[music]

Riyad Kalla: Welcome to episode five of ten in the EclipseZone Callisto podcast series. I'm your host, Riyad Kalla. In this series we're interviewing key members from each of the Eclipse projects that will be included in Eclipse Callisto release. Today we'll be interviewing Sri Doddapaneni, who is part of the TPTP project. Thank you for joining us today, Sri. Why don't we start with you giving us a description of what the TPTP project is all about, and what role it plays in Callisto.

Sri Doddapaneni: Very good. The TPTP stands for Test and Performance Tools Platform, and TPTP is an open development platform for testing performance tools that are used throughout the software life cycle. It covers a wide range of tools that start from tracing or test execution, logging, statistical behaviour, [xx] execution environment. And TPTP is one of the ten major projects that are part of the Callisto simultaneous release. And TPTP - the frameworks are built on top of the Eclipse platform, JDT, the Eclipse modelling framework. But technically we are about two hops away from the Eclipse platform in the dependency chain. We are about, in terms of synchronising the milestones, we are two weeks after the Eclipse platform, and one week after EMF, but in reality, we also have built optional features that leverage the integration with web tools, but the business in collisions and supporting.

Riyad Kalla: Right. So it would be misleading to call TPTP "just a profiler, " correct?

Sri Doddapaneni: Correct. It's more than that, it covers actually a wide range of profiling tools, as well as testing and monitoring tools.

Riyad Kalla: Well now, tell me how TPTP got started. What was the motivation behind it, conversations that took place, things like that?

Sri Doddapaneni: OK, the background is, originally there was a project called Hyades in the Eclipse community, and the motivation for that was to really create a platform of common code for tests, trace and monitoring. We observed that several other vendors, that there were a lot of [xx] and infrastructure that were costing more than the value. So it's a full resource and infrastructure based on open standards is the main driving factor. Basically we try to have a high-quality infrastructure that's built on standards, and it will enable a rich class a wide variety of tools in these areas to be developed as free extensions and as value-add extensions.

Riyad Kalla: Now how did you get involved in the project? Were you part of Hyades back then and then just rolled forward when it became TPTP?

Sri Doddapaneni: Yes, then my involvement came when Intel got involved in the Eclipse community, and that was roughly about more than two years ago, then Intel is one of the founding members and then it moved from the old structure to the new Eclipse foundation structure around that time. We were one of the initial strategy developer members, and then as a part of that obligation or interest, we were asked to lead a top level project. We looked at this, the test and performance, as the one that is interesting for us, and also one that needed some additional focus, that could use additional help. And we looked at the Hyades, which at that point was under Eclipse tools top level projects. We helped expand the scope and then elevate it to a top level project and try to enlist more adoption and also contribution. Part of that effort, Intel, was applying an engineering team towards helping create this Test and Performance Tools Platform, and I was in engineering, or the technical lead for that effort at Intel.

Riyad Kalla: I see. Now as a lot of people get involved with Eclipse, a lot of users will come to TPTP looking for a code profiler, so while it's more than that, a lot of people will run across that project because that's what they're looking for. So for a moment here, let's look at TPTP from just a code profiling point of view. One of the things that some folks will notice is the agent controller design and approach of TPTP as opposed to just a button in the toolbar in Eclipse that you click and profile your project, there's a little more to it. Can you go over the advantages of using this agent controller design that TPTP uses, and the flexibility it allows?

Sri Doddapaneni: Yeah, sure. The original Hyades, when it got started, one of the teams that was trying to solve, the problem domain that it originated from, was really the problem determination tools for distributed application. So with that in mind really, it's a distributed infrastructure, to really collect trace and logs and other kinds of odd performance and execution information, runtime execution information and be able to provide tools that serve into this distributed environment. And that's the [xx] audience [xx] You needed an agent controller than would let you start these remote executions and then collect data there, so that's a strength. And you do realise that when you are running more a standard option on your local system, then you wouldn't need an agent controller, per se. But agent controller also could be used because it's a much more powerful tool, but it's not something that is required for local.


What we went, actually in this direction, is we went and looked at this to improve this, make it more usable for the simple use case, where someone has a simple job, an Eclipse plugin they want to profile, they wouldn't have to go an install an agent controller and then manage the service. So we made it more of an integral feature so it can run inside the workbench, the Eclipse workbench, and it would be more transparent to the user, so they wouldn't have to go get another package, part of the Eclipse plugin that manages it. We did that in the TPTP 4.0 time frame which was about a year ago, and then since then we have included the performance integrated agent controller. Still the agent controller from the first of the data collection, but one wouldn't have to worry about it because it's more integrated from the Eclipse workbench.

Riyad Kalla: I see. Now as part of the 4.2 plan, and the release that's non-Callisto, we noticed that a JVMTI agent implementation was introduced to keep up with the new Java six debugging changes. Were there any major hurdles moving to the new API while still maintaining the JVMPI support in the older versions?

Sri Doddapaneni: In fact, not. Probably what expanding a little, because in the context of this upgrade to the JVMTI standard, we are actually doing a lot more than simply upgrading to the new standards. So what we are really doing is we left the JVMPI based support intact, it's not impacted in any way by the migration to the JVMTI. Really the JVMTI support comes as another agent, so either wouldn't have to know it, but when they have the JVMTI enabled Java runtime for JVM, then you could choose to use the JVMTI agent. And it is fully, I mean, a production feature. Right now we are [xx] on the JVMPI feature. But [xx] would be we use both the PI agent and the TI agent to compliment for the older runtimes that don't support JVMTI.


And continue to use what we had. And for the newer environments, the Java five and Java six environments, we will, users would be able to select JVMTI-based capabilities. And so that's not really causing any problem, going to the new API. In the context, we have expanded this this code, we said this is an opportunity where we could retool, really make it into a very best-in-class, low overhead profile data collection.

Riyad Kalla: Right, so it that what the advantages are to the users now, in the new 4.2 release?

Sri Doddapaneni: Yes, 4.2 release, there's a tech preview in the JVMTI, there is this feature, and we added some additional capabilities in the PI, JVMPI area, the profiling interface-based collector. Also we added some aggregation at the collection time, which really again will help reduce the overhead, when running profiling, profile data collection, it has potential that it won't be as slowed down.


But the main distinction really, with the JVS1 you can really, you would really need an instrumentation-based approach, which means you have a much finer control over the overhead, with regular collecting, whereas with profiling interface most is VM which is monitoring for the events, that is across the board.

Riyad Kalla: I see. Now what about, you and I had talked previously, and you had brought up the X-ray and Memory Manager monitors. Tell me a little bit about those, and why they were moved from platform up into TPTP.

Sri Doddapaneni: Interesting, when Hyades got started and then eventually been moved to the TPTP, our main focus was supporting with distributed environments and then trying to do a much more polished feature for the local user as well. Still our solution was not so focused on helping the platform improve. And in that area, the X-ray tool was really being used or created in the platform project to help diagnose the performance issues with the plugin loading. So the X-ray in some sense is really a plugin profiler.


And it is very well used in the platform projects to optimise various runtime issues. And Chris Laffra, the programmer who originally developed this in the platform project, and then, but the right home for that is really where most of the profiling and performance related projects are, so we talked to the platform project and mutually felt that it was best to host it under TPTP. So for now it is still available as a standalone tool, as a preview feature, but we are looking towards its capabilities and features, in bringing into the JVMTI base. So you should expect to see in about six months to nine months time frame that we would identify which of those features are good to have in the TPTP profiler, and then start adding it to the main capability.

Riyad Kalla: I see. Now some of the other enhancements in 4.2 that people are really excited about, some of these static analysis features that are making their way into TPTP are looking pretty nice. Recently in EclipseZone, some of the users had asked if the static analysis was capable of metrics, like McCabe index lines of code, things like that. Is is currently capable of doing things like that in the 4.2 release, and if not, are those types of things planned?

Sri Doddapaneni: I am not the best qualified guy to talk about that specific issue, but my initial feel for that is that we don't have such capabilities today, and really where we excel in that space is the extensible framework for hooking up your static analysis in general. We do provide one for Java, and we are working on another one for C++ analysis. But also one could hook their own analysis engine into our framework. In the TPTP framework. And the rest of our analysis and UI features would be enabled to the user of the framework. And today we have Java, best quoting, best practice, rule set out of the box so you could use TPTP, if you're a Java developer, or you could use static analysis too. Kind of look at what maybe some of the issues with your code, and that'll help them, and then we are working, there is a preview feature with the C and C++ that's got a smaller rule set, maybe about 20 or so right now, but we hope to grow it and eventually have a bigger set. So this is something that we are also working on, collaborating with the CVT to see how to make that happen.

Riyad Kalla: Is the CVT a big consumer of TPTP? I notice you've mentioned the C and C++ development a couple of times already.

Sri Doddapaneni: This is not the strong point of the TPTP today, we don't have out-of-the-box tooling for end users if you want to pick up TPTP other than the C++ preview feature. But the frameworks in terms of how the performance data is modelled or how the rest of the tools or even test tools if you can extend them, you can leverage a big part of the frameworks to support such tooling. Right now more of the strength in that area really is in the framework rather than the exemplary tools. We are missing in that area, but we hope to do more.

Riyad Kalla: Now talking about code coverage, I noticed in 4.2, there's a screencast on the site, about the JUnit code coverage that TPTP offers now. It's very similar to how Clover Plugin for IntelliJ works, it'll actually run your test cases, and it'll show you the code coverage percentages down your package view, package explorer view for each of your classes, which is very slick. But it was marked as "preview." What more is coming? Should we expect those types of features that are really user-centric in TPTP?

Sri Doddapaneni: This is certainly one area where we wish to do more, but the reason it really is taking effect here is this is the very first time we have really brought this to the users. Our tendency is generally to go into more of a tech review or integration, if you will, approach, try these features and get the feedback, and see if we really want to make them more of a permanent feature. So that's still motivation rather than validity, is it really good or not. Even JVMPTI work is a preview, and we have code coverage as tech preview, the C++ code review is tech preview. We've got a number of tech preview features. And some of these will graduate into official features in the official release pack. What is more coming in that area, we wish to do more. Right now, none of the contributing members' priorities match with those. There is a need both in the Eclipse community and also by the Eclipse project for such features or tools so we can really have the best of the Eclipse projects in terms of code coverage and tooling.

Riyad Kalla: Now some of the other--

Sri Doddapaneni: But there should be more in terms of maybe have a more polished presentation of the results, code coverage, metrics, maybe more charting and parting of the data. I could think of a number of ways that we could add more sugar to it.

Riyad Kalla: Right. Well, some of the new features, we're talking about some of these new tech previews, also noticed in 4.2 there's the Automated GUI Recorder, that is going on for recording, for people who don't know, it's for recording GUI actions in an Eclipse-based project and then playing them back for testing purposes. Is this also a tech preview item, or is this something that is done and mature, and also is there the possibility or the plan to allow support for applications outside of the Eclipse platform, using this technology?

Sri Doddapaneni: Excellent question. So it is a tech preview feature today in 4.0 of the Callisto release. And so this is something we actually use internally in the project for our own testing, test automation, that we test automation. And what is more needed, one, is we need to do some restructuring in our framework to allow it to support standalone apps, not Eclipse plugins, but RCP apps or other Java apps. So you can't use this now to test your RCP or other standalone apps [xx], it's really more of lack of resources to do the restructuring so we can be able to free up the unnecessary dependants.


But otherwise, it's in good shape, a lot of people use it, in fact this is the most sought-after feature from TPTP from the community. As a part of our TPTP 4.3 planning, we promised to get feedback from the community through Bugzilla [xx], and this is the one that everyone wants and they want it extended to the RCP apps. And we are of course, we don't have resources today to meet that demand, but we are open to anyone who wishes to concentrate in that area. We kind of indicate that our highlight such a solicitation of contribution indicating that such a feature is needing help, by designating them as "Help Wanted" in Bugzilla or also in our plan description.

Riyad Kalla: I see. Now, something that I just heard about recently, and I don't know what it is, is the Build-To-Manage Toolkit feature in TPTP. Can you introduce us to that, to what that is, and more importantly, who should be concerned about that, who should know about that, who can benefit from that?

Sri Doddapaneni: This is really to help in the domain of managed agent, or web services-based or standards-based agent, really. So when you have a web site without data and you want to be able to either browse it or control it, this is what we call a managed resource explorer. Built-To-Manage is really coming out of this system of web services based on more web services-based standards. So anybody who is working in those areas, I think they should take a look at it. And also this tooling in TPTP allows for any standards-based or web services like managed agents out there, or resources. And also we can support such managed agents based on Apache's Muse Project. In fact, the bigger team at IBM that's really contributing to this, they have been making contributions into the Apache Muse Project and also making contributions.

Riyad Kalla: I see-- yeah, go ahead.

Sri Doddapaneni: So, who should really care about it, really, anytime you want to browse any resources that are standards-based. Hopefully we can add more of these in the future, but these are all a standards-based agent. Web services, protocols, and things.

Riyad Kalla: I see. Now switching gears here for a second, and focusing a little closer to home. TPTP has been working very closely with two big, high-profile projects, which are BIRT and WTP, and for BIRT, you guys have been helping with the report generation, I believe, and then WTP has been doing the server-side profiling. There's a lot of cross-project integration going on. Where does the motivation come from to integrate big-scale projects like this, did TPTP want to work with BIRT and WTP, or did those guys come to you and say "Hey, let's think of something?"

Sri Doddapaneni: That's an interesting question, I think I'm not sure, maybe a little of both. Really where the motivation is coming from is really, we share common users. If you look at Java developers, they need Java development tools like the JD provide, then they need testing and performance tools. We're really doing this in the context of web tools, of web services or web applications, and you want web tools. So really to provide that seamlessness, or try to be a part, a contributor to the whole solution, it makes sense for us to enable this so that the user can really leverage their life cycle. And then TPTP has been very good at this, we recognise this, and we are more of a platform - our center is more in the platform, rather an extensible platform, than as an exemplary tool as say two years ago. And now we're getting better in the exemplary tooling. And as we are adding more and more exemplary tooling, then we realise that these are the areas where we need to add them and then we see an opportunity to work with WTP for providing support for profiling on servers.


And then for BIRT, interestingly, there is a common interest. It happens that TPTP actually had charting and reporting capabilities prior to BIRT's formation, in Hyades. And then BIRT had got a proposal that got floated around, we looked at it, we said we shared a common interest, but we do have some SVG charting, we don't particularly care to host them in TPTP. You guys, you seem like the project which should be hosting them. So we actually made a donation of that part of the code base to BIRT. And also one of our [xx] was also made a [xx] into BIRT to help do that. And then subsequently we adopted the extended version of the framework that BIRT created for our reporting team. So in order to actually get reports from various views in TPTP then you would need BIRT Report Generation Engine, and we have BIRT Reports Template that we provide, and then by getting the BIRT and our TPTP provided report templates for the BIRT, you can actually generate various reports.

Riyad Kalla: It sounds like there is quite a bit of, I mean as far as that old phrase goes, "Eat your own dog food." There's quite a bit of using of TPTP within the Eclipse foundation throughout the products. Not only just integration, but for example, a lot of the work that went in to Eclipse 2.2 was performance and profiling and finetuning the platform. Is most of your feedback coming from other Eclipse projects, or is the majority coming from users?

Sri Doddapaneni: Yeah, it is coming from users and also from adopters. The distinction is really, adopters are people who are extending or taking advantage of the API and the extensibility we provide, to create further tooling. So the latter is really the one who drives [xx] more, but in the last three quarters, and especially with Callisto in the last two quarters. We've gotten a lot more user feedback, and a lot more visibility.


So, going forward I can see it being much more of a balanced thing. Yet, both products built on top of TPTP providing has feedback and this needs to be extensible. This, and we need to make our functionality more general or expanded.

Riyad Kalla: What is the community involvement like for TPTP? Do you have a lot of users submitting fixes or feature requests or wanting to help out on the project, or are they pretty hands off at this point?

Sri Doddapaneni: No, there's a lot of community involvement. We get a lot of bug reports. In fact that's very valuable to us. In fact, the amount of feedback to various different effects and enhancements requests has been growing constantly. It's amazing actually when I look back at the recent release of TPTP, 4.2, we ended up actually fixing about twice the amount of defects we normally fix in such a duration. And we still ended up actually growing our backlog by about two megs. On initial parts we looked at it and said, "This is not good we are doing, our backlogs ", or, "What's wrong?"


And when you look at it it's really because we are growing more popular among the users and we get more bug reports, and that's really what is driving. And that actually connects with the activity we see on the discussion forums, the development [xx], the mailing lists.

Riyad Kalla: Now, as far as the bugs, I noticed on the 4.3 timeline, the targets to get done for that: Mac support is listed as a priority, and there's, of other things, mentions the porting of the ageing controller. What were the primary reasons that Mac wasn't supported? What were the challenges on the Mac platform for something like TPTP.

Sri Doddapaneni: In fact, technical [xx]. It's really about resources available. The real current contributions are, this is really falling below the resource capability. And we have actually [xx] to the community to provide as a port, even contribute bills. So, one part of the thing, I mean, the actual port should be fairly smaller, faster, compared to the maintenance and the test effort and all going forward and [xx].


So that's really what drives this we have a large number of target platforms with the port. And even the new platform we had with our worker, in terms of being able to test and validate and release under the platform. So, we were hopeful that, we had actually received a few offers to contribute a Mac port, and hopefully we'll be able to help the community that wanted to contribute this.


This is really very [xx]. Out of the plan. In fact it is in the plan, we kind of call it, "Help wanted, " because obviously we haven't resources. Answers that is being there, I wouldn't give it a high probability, but it is possible. It's too lot of time.

Riyad Kalla: Right. Moving on to the more exciting things. Now that Callisto is out the door and TPTP 4.2 is out there, what are your favourite features in 4.2 that people should really keep an eye out for?

Sri Doddapaneni: In 4.2 we have, actually, a number of features, but the most notable one is the static analysis in [xx], which was actually include static analysis in [xx], [xx] much expanded for java. [xx].


Another area that we had actually announced significantly is what is called Probe Kit. Probe Kit is really a way for users to define what probes they want to insert in their core, and where they want to insert it is very easy to, to user interface and then the tools of [xx] the probe byte corks[sp] for the probes, and then insert that byte cork into the [xx], and all the hard stuff, it does it. But in the past it was only able to do it statically, meaning you would have to, the instrument you're class of before you deploy on the file system. And you did that. So the new one we said, this makes sense to do it at the run time that none of the users class files are impacted, and the environment is not impacted. And we were able to, that was, that makes it significantly more useable, because it doesn't leave any after-effects in terms of end user probe, and also makes it flexible to add more probes at run time.

Riyad Kalla: Right. Now, what functionality should people be excited about that are coming in 4.3.

Sri Doddapaneni: 4.3? It is going to be a much harder variation, so don't expect, it will have the same amount as 4.2, but we have some additional things planned. The main focus is, one of the main focuses is going to be on improving our infrastructure actually, so we will be able to respond with next year's simultaneously with [xx] much better than [xx].


So we're going to make some improvements to our infrastructure to automate further, automate the testing, so we can shorten our life/death cycles for each milestone, and also we'll be able to add more test cases, and we improve the [xx] and fix, make some significant dent in the backlog of the defects also.


Quality is one focus, and other areas where we are actually trying to invest in our tech review features so we are able to mature them all the time. So you should see, in fact in all of the tech review features I think there is going to be very sharp focus, and more capabilities. I mean, even if they don't graduate out of the tech preview at this stage, to see gradual enhancements and maturation.

Riyad Kalla: Right. Well, we're getting close to the end of the podcast, is there anything you want to add before we wrap up?

Sri Doddapaneni: What do I want to add more? I think, really we are excited about the direction we are going, and I think it's been validated with the kind of feedback and the increased participation from the community that we are seeing in various, obvious to the forums, but we want to thank the community and hope that they continue to support us and give us the feedback in the ways that we can make Eclipse even better.


[music]

Riyad Kalla: Right. Well, we want to thank our guest Sri Doddapaneni from the TPTP project for taking the time to sit down and talk with us about his role his project played in the Eclipse Callisto release. We appreciate everyone tuning in, and hope you enjoyed yourself.


This concludes episode five of ten in the EclipseZone Callisto podcast series.


[music]

Transcription by CastingWords