6 - Doug Schaefer of the CDT Project

Riyad Kalla: All right. Welcome to episode six of ten in the Eclipse Callisto podcast series. I'm your host Riyad Kalla. In this series, we're interviewing key members from each of the Eclipse projects that will be included in the Eclipse Callisto release. Today we'll be interviewing Doug Schaefer whose part of the CDT project. Thanks for joining us today Doug. Why don't we start with you giving us a description of what the CDT project is all about and what role it plays in Callisto?

Doug Schaefer: Sure. The CDT project essentially provides a C and C++ development environment in Eclipse very similar to what people are used to when they're running Java programs using Eclipse. It's really trying to bring as much of the advantages of Eclipse frameworks and the community to C and C++ developers. Callisto has really allowed us to bring that technology to the masses of Eclipse. Previously we had our own release cycles and release management and kind of took a different street than the rest of Eclipse and Callisto really allowed us to become an equal member in the Eclipse group of projects and open up the audience a lot more for C and C++ developers in Eclipse.

Riyad Kalla: Right. Now introduce us to how CDT got started. What was the motivation behind it and where did the original contributions come from and so on?

Doug Schaefer: In the beginning, there were actually two C and C++ development environments that were being built on top of Eclipse. Both of them were pretty similar in need, really. People were used to using Eclipse at that time. They knew about how it could help Java developers and how you can plug in additional tools to create a nice integrated environment, something us in the tools industry have really dreamed of for a long time, and Eclipse is really bringing that together. And really just bringing that environment to C and C++ developers, opening the door for value added tooling. The two projects were kind of started in different directions. There's one from IBM which is focused on C development for mainframes and that environment looked a lot more like what the mainframe developers were used to seeing.

Riyad Kalla: I see.

Doug Schaefer: And the second one was started by QNX which really took the JDT as a template and extended that to C++ and actually integrated, being able to integrate third party compilers and debuggers.

Riyad Kalla: Were both of these based on Eclipse when they started out?

Doug Schaefer: Yup, they're based on Eclipse. They're very similar in workflow which is different architectures. Really, everybody started rallying around the QNX one and that's the one that you see today. You look at a lot of the code and you still see the odd Java comment because a lot of it was very similar to what the JDT had done. So it's really helped grow the architecture.

Riyad Kalla: Ok. So why don't you tell us how you got involved in the CDT project.

Doug Schaefer: The CDT project started about four years ago. I was working at Rational Software at the time and had been working on their modeling tools and their co-generation tools. I was there for a few years and then I kind of heard through the grapevine about QNX contributing their C++ development environment to Eclipse. And Rational was interested in getting involved so I kind of nosed my way into that because I wanted to get more involved with direct user tooling, people actually writing code, and just being part of the open source community that's out there. I've always had this dream of writing a C++ parser and this really opened the opportunity for me to get involved at that level of the tooling.

Riyad Kalla: Did you enjoy it?

Doug Schaefer: Yeah! It's not the glamour area in the computer science industry, but it was pretty successful and it's still one of the features today that people appreciate with the CDT.

Riyad Kalla: Absolutely right. So these two projects, QNX donated their implementation, and at that time you then moved to QNX and contributed to it, or how did that happen?

Doug Schaefer: Well, I stayed with Rational. We were very interested in getting C++ tooling for Rational's products at the time. Since then, it was bought by IBM and became part of the bigger IBM. There are still people there working on the CDT and integrating tooling with it. I moved to QNX last summer. There was an opening here in a project lead role and that was something I wanted to get more involved with. I'm now the project lead for CDT and I'm helping grow that community.

Riyad Kalla: Now this is interesting. When I saw CDT come out, the first thing that popped into my head is I wonder what the challenges are in implementing a C and C++ development environment on top of a Java platform. So what kind of difficulties do you guys have of, say, debugging or some of the lower level native work implementing CDT on top of the Eclipse platform?

Doug Schaefer: Certainly. Since most of us were C and C++ developers, we had this urge to use some of the libraries we've used in the past, which is not as easy when you're building a Java application, in order to mix those two worlds. So certainly we have a number of different protocols we've set up to communicate with the external tools that do a lot of the work for the CDT, so keeping those two worlds together. And really what's lacking today in the Eclipse environment is that mixed Java, C programming environment. That's something we're pretty interested in pursuing with the CDT.

Riyad Kalla: Do you guys have a lot of native dependencies now between the CDT and platform tools like GCC or things like that?

Doug Schaefer: Not particularly. We try to keep that to a minimum because of those challenges because every time you make a change in a native library, you're faced with that. We support nine different platforms with the CDT. So we kind of kept that to a minimum but we used things like standard input and output to communicate with the compiler and we have some internet protocols that we use to communicate with remote debuggers.

Riyad Kalla: Oh, I see. Now, what compilers do you support? You've got Intel, Microsoft, GCC. I'm sure I'm missing some others in there too.

Doug Schaefer: We essentially set it up to ensure that anybody who has a compiler out there can integrate with the CDT. Out of the box, we support the GNU tool chain, mainly because it runs in every environment that we support. And it's open source; it kind of fits in with that paradigm so we can get many people in the open source community using the CDT. We do have integrations with Intel, working hard and integrating with their compilers especially on Linux. We have a number of embedded operating system companies including QNX and integrate with their own compiler suites and debuggers as well. We have a lot of different companies that have integrated their specialized debugging as well as our standard GDB support that we provide.

Riyad Kalla: I see. So it sounds like, or correct me if I'm wrong, is the target audience for CDT not so much your Microsoft Visual Studio developers, but more your Linux and open source and straight C and C++ developers?

Doug Schaefer: I think it's mainly that way because of the market needs of the commercial vendors that are contributing to the CDT. Certainly it's on my wish list to better support GNU's development, especially as Microsoft starts giving away more of their tool chain, the.Net framework.

Riyad Kalla: You brought up a good point there. Do you see because of the movement to.Net for Microsoft and also the Mono project, you're talking about open source, is there talk of CDT including C# at some point?

Doug Schaefer: Yes, there has been some discussion and I'm working on that on the side, being able to integrate C# in particular and being able to support the Mono tool chain as well as the.Net tool chain. That's kind of just started and it's going to be slow coming together. As more people contribute to it, it will grow.

Riyad Kalla: Is that so similar to Java, the C# support that you guys are building? Is it closer to JDT than to CDT, or which one are you focusing on?

Doug Schaefer: It's probably more focused on extending the CDT frameworks. Today we support C and C++ and we actually treat them as two different language environments because they are a bit different. So we're really looking at C# as a third language, per se, as well. We also support, there's a Fortran project at Eclipse that's building on top of the CDT. So we're starting to grow into a multi-language framework and C# is the natural next one, as long as we can get people who want to contribute to it.

Riyad Kalla: Now how do you guys handle this? I think of the JDT and all the great tooling that's there and it's obviously Java, and you've got your type hierarchies, and your outline deal, and your auto-complete, and your syntax highlighting, and validation, all this stuff, and I think for CDT, every time you support a new language, it's totally different. You take C and there's no polymorphism and there's no inheritance. Then you take C++ and then you have multiple inheritance. Then you take C# and I believe that's single inheritance like Java, I'm not quite sure. Every time you support a new language which is quote, in the C family, they're totally different. So how do you guys keep these tools separate from each other? Like your outline views are all going to be different, your auto-complete, your syntax stuff, all this stuff is different every time, isn't it?

Doug Schaefer: I guess that's what the advantage is because they're not totally different. There's a fair amount, just being called a C like language, there are a number of concepts that are in common. So what we've done is try to build our infrastructure so that we can capture the concepts at a general level and be able to extend those for specific language features. So we do treat C and C++ separately, they have different parsers, but a lot of the internal data structures, they share a common class hierarchy or interface hierarchy. So we do get some reuse that way. Really the challenge will be to drawing the right line and that's why we want to get more languages involved, to see how extensible, how much reuse we can actually get out of the CDT.

Riyad Kalla: All right, so now I'm wondering is there any features that have been requested for the C and C++ or any future languages that the CDT team just had to say no to because they couldn't support it on top of the Eclipse platform?

Doug Schaefer: I guess one of the features people have been asking for is to be able to do what the JDT does with their built-in compiler and do syntax checking on the fly. The CDT parser that we have just isn't robust enough and full-featured enough to do that. A lot of people are proposing to be able to link in different parser engines and I think that's going to prove too difficult to integrate into a Java environment. And we're kind of faced with issues since C and C++ are so complex, much more complex than Java, we've really run into a lot of performance issues when trying to do too much of this parsing and syntax checking.

Riyad Kalla: Right. Is there thoughts that maybe rewriting some of this stuff down the road may help or, like you said, is the problem just too large that some other company may need to step in and take charge of, say, the C editor and then some other company handle the C++ editor?

Doug Schaefer: Yeah, to some degree that's already happening with the CDT. We're getting a lot more participation. But a lot of the issues are just that it's expensive to do and whether it's going to add enough value to the CDT as it is today to really make it worth pursuing.

Riyad Kalla: I see. Now, is CDT getting inspiration or requests for features from other tool sets like Visual Studio or even K Develop or the Junta I believe the other one is?

Doug Schaefer: Junta?

Riyad Kalla: I think that's the name.

Doug Schaefer: We certainly have a lot of Windows developers and this is where Intel, who's participating in the CDT, come from. They want to provide a similar environment for their Linux compilers as what they have with Visual Studio. And a lot us, including myself, come from the environment where I was a big Visual Studio user and really grew fond of the features that it has. So that's certainly one of the major inspirations for the CDT as well as the JDT itself. Every release they introduce so many new features that prove so useful to doing general development that we'd like to bring more of those to the CDT as well: things like quick fix and refactoring. Factoring's a big issue with C and C++. We have the beginnings of support for that but, since the languages are so complex, we really need to take a lot more time and find the best way to do that.

Riyad Kalla: Right, right. The CDT team, as some people go to the site and take a look, the companies that contribute to the CDT run the entire gamut from embedded real time development to large scale enterprise applications, things along those lines. How do you guys, as the CDT team, how do you decide what features get added because those companies, I'm guessing, have very different requirements?

Doug Schaefer: Yeah, it's actually one of the interesting challenges that we have because companies focusing on developing embedded are more focused on integrating their tool chains and debugging, remote debugging. The enterprise companies are more interested in supporting rich libraries. We also have high performance computing, the scientific community starting to use the CDT and they're dealing with these big, monster supercomputers with 10,000 processors and such. We try to focus on providing the core, what the core C and C++ developer needs in their daily work and we're starting to look at ways of growing more specific needs, actually adding more separately downloadable features that people can plug into the CDT, to support things like Linux development and such.

Riyad Kalla: Now what changes over the years has CDT had to request of the platform to get where it needs to get?

Doug Schaefer: I think one of our biggest challenges we've had with the platform is just how the whole build infrastructure goes. The platform build is really aimed at the Java developer where compiles are almost instantaneous and so they can do it, every time you save a file you can fire up a build and it will finish quickly. With the way we integrate external tools to actually do the builds, whether its make or the different compilers. It takes a long time to actually build a C and C++ application. So, we had to work with the platform guys to ensure that flexibility, where the user has a bit more control when builds are done and how they were done.

Riyad Kalla: I see.

Doug Schaefer: And they've been very responsive and very helpful.

Riyad Kalla: I see. What are the builds based on in CDT? Are these makefiles or automake? What are they?

Doug Schaefer: Yeah. We have two different build systems that we support in the CDT today. One essentially calls to any application that you have externally. Most users there write their own makefiles and use make. But we also have instances of people using different build tools like SCons and Jam. There's a whole mess of them. And we also have -- the second build system is what we call a "managed build system." And it does make file generation for you.

Riyad Kalla: I see.

Doug Schaefer: But it still calls to external tools to do that.

Riyad Kalla: I see. Now, looking at some of the movers and shakers in the open source development environment, we see -- or, at least, I think people have seen this online. In the last few years, it seem like Red Hat has really put its weight behind getting the native builds of Eclipse going and just Eclipse in general. And it seems like the motivation there at some point is going to be Red Hat wanting to normalize their tools on the Eclipse platform in their Java work as well as their C++ work. Where does that leave you guys? Because Red Hat does a lot of kernel development, GTK development, things like that. And also, other open source developers are doing very similar things that want to use the CDT.

Doug Schaefer: Exactly.

Riyad Kalla: What kind of requests are coming from that? Or help from these companies?

Doug Schaefer: That's a very timely question. There is a Linux developers conference happening up here, in Ottawa. And there is a team from Red Hat that made the point to come and visit me yesterday.

Riyad Kalla: Oh, ok.

Doug Schaefer: You are right. They are very keen on Eclipse support and CDT support. And we talked about how do we get the GNU development, office developers -- basically, getting them an environment based on the CDT, so that they can use it in those environments. So, definitely that is one of the areas where we have support going forward. Red Hat guys promised bodies to come and help the CDT achieve those goals.

Riyad Kalla: Oh, wow.

Doug Schaefer: So, yeah, we are working well with them. We are also working with Novell, who is doing the Susie front, so there's a lot of interest on using Eclipse and CDT for Linux development and basically, we are going to work hard to make sure those guys are successful.

Riyad Kalla: And do you think this collaboration is going to be a part of version four of whatever is coming in Europa or is that still too far off to tell?

Doug Schaefer: They are still just starting out. Basically, I am encouraging them to participate in the CDT community, to get some feedback, and basically, to focus on better integration with the auto tools and how to support those developers better. So, I think that will probably be one of the separately downloadable pieces, but that may come before Europa, but definitely want to see what we can get done for Europa for that.

Riyad Kalla: Now, given the things they walked in with, saying they needed, what kind of changes and challenges to you see need to happen to CDT to make that happen?

Doug Schaefer: Basically, it's the integration. It's the configure scripts and the whole auto tools set of tools that they use. We really need to look and define what the build system is going to look like. It's not a matter anymore of simply calling it a make, like we've done in the past. We need to do more coordination with those tools. That will be the challenge that they will have to address.

Riyad Kalla: I see. So, what right now is the most requested feature in CDT? And has it been added and if not, when do you think it will be added?

Doug Schaefer: Certainly, supporting large projects is one of our most requested features. From a performance standpoint, because right now with Eclipse it is difficult to integrate a source base that contains 50 million lines of code, which we have. There are a number of companies out there that have large applications like that and want to use the CDT. It requires coming up with a different strategy on how those developers will work, whether we present views on that file system. There is an Eclipse file system, which is a new component in the Eclipse platform which was added, and whether we can take advantage of that. So, that's definitely something we are looking for in the 4.0 time frame.
Probably the other bid requests we are getting is the refactoring. I've seen people using the JDTs have become to rely on refactoring.

Riyad Kalla: Yeah, absolutely.

Doug Schaefer: We are looking for that to be a feature in the CDT. I think that's probably more long-term. We do have some research students working in Europe and the United States looking at it, but it's still a research topic at the moment.

Riyad Kalla: Because refactoring goes to the bones of any good IDE, based on the strength of the grammar and modeling of the language. Is there -- well, I forgot my question.

Riyad Kalla: I just completely drew a blank right there. Ok, we'll skip that one. Why don't you tell us what your favorite new feature in 3.1 is.

Doug Schaefer: Sure, that's probably the feature I worked on. In the past, we've had a lot of issues with the scalability of CDT parser and being able to generate an index in a timely fashion. So, we took a new approach at it, where we are basically removing duplication. We used to parse header files over and over again, just like a compiler does. So we took a new tact and a new strategy on indexing to make sure we did as fast as we could. One of the projects I use as a scalability test basis, the Mozilla project. I presented some of that at Eclipse Con last year. In previous releases, it used to take almost an hour to build an index for that source base, about four million lines of code there. Within the new framework, we got that down to under 15 minutes.

Riyad Kalla: Oh, wow!

Doug Schaefer: It was kind of a major celebration when we got our first timing like that. It looks like we were successful. It certainly was one of the major stumbling blocks that new users had complained about, especially on large projects. So hopefully this will alleviate that and get more people more successful.

Riyad Kalla: Right. Well, I've got good news -- I remembered my question. What I was saying is, because refactoring is such an in-depth function and really brings out all the features in IDE to perform correctly, can you guys look at other IDEs, like say KDevelope, open source IDEs, see how they do it and get inspiration or pull solutions into the CDT or are these apples and oranges?

Doug Schaefer: I don't know. I haven't seen refactoring in other tools. I know there's some out there.

Riyad Kalla: I see.

Doug Schaefer: But we are certainly getting a lot of inspiration from what the JDT team has done. They have a really cool framework that makes refactoring a lot less work than it could have been. So, we are modeling our parser information to follow the same one that the JDT team has done. And we'll be able to reuse some of their frameworks, in fact, and be able to implement it. I guess the real issue with refactoring is being able to have enough information to do it correctly.

Riyad Kalla: Right, right.

Doug Schaefer: Because a lot of the C and C++ build demands on the build environment... Since there is conditional compilation in C and C++. You got to take that into account, make sure that your factoring operation don't cause more problems than solve.

Riyad Kalla: Yeah, that's true.

Doug Schaefer: But there are certainly a lot of people interested in it. We have some name refactoring right now. There are some simple things that we can do over the next couple of years.

Riyad Kalla: Well, we saw that planning for version four is already up on the site. You got some line items listed for it. What are the features that the team is most excited about having for four that you are committed to? You've mentioned that some other contributions may come from developer Red Hat, but as far as things you know that are coming in four. What are the more exciting ones?

Doug Schaefer: Some of the more exciting ones: WinRiver is stepping up their contributions to the CDT and they have some pretty neat views on source code view things like type hierarchy view, call browser, who's calling who. There are going to be a lot more views that are built under CDT's index, so it'll give people an easier snapshot of what their software looks like. As I mentioned, there's going to be multi-language support, hopefully getting C# getting built on top of the CDT, and other languages. Making sure we have the right framework for integration new languages.

Riyad Kalla: Right.

Doug Schaefer: I think remote development is going to be a big topic that we are working on, not only embedded, but we have IBM stepping up again to ensure that CDT works with their remote development for their mainframe environments.

Riyad Kalla: Wow. So, we got a lot of stuff coming for.

Doug Schaefer: Yeah. During the last month's CDT conference call, I did a head count and there were 20 people on that call, all of whom, I think, will be contributing code. So, the CDT has definitely grown a huge amount over the last year, should be a lot of cool activity finishing CDT 4.0 for Europa.

Riyad Kalla: Very cool. Well, we are getting close to the end of this podcast. Is there anything else you wanted to add?

Doug Schaefer: I think it's just elaborating on that a bit. The CDT community, it started out as a small band of companies, based in Ottawa and we've grown so much over the last four years and CDT is being used more and more for things I've didn't even think we could use it for. There are companies doing system C, which is a hardware modeling that looks like C++ and they are actually using CDT to build those hardware models. Just the way the CDT is grown and made an impact on the industry, that's something we are very proud of and it keeps us all working hard on it, making it better.

Riyad Kalla: Yeah, given the growth rate, it was obviously filling a need, and people area jumping on board as fast as they can because it's doing what they need it to do.

Doug Schaefer: Yeah, other than Visual Studio for Windows development, there really hasn't been that standard IDE for C and C++ development in the industry and people are really latching on to the CDT as hopefully that new de facto standard they can use and rely on and be there for the future.

Riyad Kalla: I agree with you, there hasn't been the editor that everyone needs there. Ok, we wanted to thank our guest Doug Schafer from the CDT project for taking the time to sit down and talk with us about the role his project is going to play in the Eclipse Callisto release. We appreciate everyone tuning in and hope you enjoyed yourselves. This concludes episode six of ten in the Eclipse Zone Callisto podcast series.