Final DTP Survey Results October 2010

We had eight people total complete the survey (a big thank you to all of you), which was more than I was expecting honestly. With as quiet as things have been in DTP lately, I was expecting the sound of crickets to be prevalent. :)

However... As promised, here are the final survey results...

Question 1: Where do you use DTP?

a) Standalone in Eclipse for the database functionality (green)

b) Within WTP or Dali (JPA) (red)

c) Within a commercial product from IBM, Oracle, Sybase, or other company (yellow)

d) Other (please specify) (blue)

As I said in my last post, this was a bit surprising to me. I had no idea that this many folks actually used DTP for its standalone database functionality rather than as part of a larger suite.

The responses to "Other" included:

• As part of our tool. For reading database meta data and for parsing SQL (planned).
• BIRT

I'm not sure how I forgot BIRT - it wasn't deliberate. But it's nice to confirm that DTP is in fact being used in a few different contexts.

2. How do you use DTP?

a. As a user

b. As a developer

Definitely more use of the DTP toolset for developers than users, but it's nice to see users represented in there as well.

Question 3: Where would you like to see DTP used?

This was an open-ended question and we ended up with three replies:

• As a replacement for SQuirreL SQL inside Eclipse.
• As more integrated with the IDE
• Or as a standalone RCP SQL Tool like PL/SQL Developer

Question 4: What functionality would you like to see in DTP? Or what changes would you like to see?

This was another open-ended question with more responses:

• Provide documentation about what is supported for each database... e.g. "how do I see an SQL query explain for Sybase ASE or DB2 UDB"?. After hours of experimenting I *think* its not possible but can't tell for sure.
• Offer the ability to specify an order when executing a batch of sql files
• ETL functionality - data extract and load
• Easier handling - see PL/SQL Developer! More DB-Status functions
• Improve the SQL parser as a standalone library

Each of these will probably find their way into Bugzilla as feature requests. And though not all will likely get done, it's good fodder for discussions going forward.

Question 5. Would you like to help with DTP?

There were only three responses - yes, no, and other. The other described itself as "willing to report bugs and test cases", which is great. "No" isn't helpful, but is understood due to a lack of time or resources. And "Yes" is what we always like to hear!

All in all, I'm pleased with the results. We received some constructive feedback in addition to simply discovering that our community still has a heartbeat. :)

As I said last time... If *YOU* have more suggestions for us, feel free to join the mailing list (dtp-dev@eclipse.org) or put a message on the forum/newsgroup and let us know what you're thinking. As always, we're looking for help - whether it's testing and reporting bugs, contributing patches, or becoming a committer!

Thanks again to everyone who filled out the survey. It helps to know what you are thinking!

Have a great Halloween and a wonderful weekend!

Where is DTP being used these days?

As we start looking at what will go into DTP (Data Tools Platform) 1.9 for the Eclipse Indigo (June 2011) release, it struck me that we don't know that much about how DTP is being used currently. It's been a while since we "polled the studio audience" to see what was up...

And though we have a skeleton crew now helping us with bug fixes and new features, we'd love to have more folks pitch in to help - whether it's by writing articles or blog posts, reporting bugs, or offering to lend a hand with patches for existing issues or new functionality. However the help is offered, we'll probably consider it!

But we'd love it if you use DTP if you could take some time and please answer a few survey questions:

Click here to take survey

Your answers will help us figure out where our priorities lie for this next release and what we may or may not have help with from the community!

Thanks in advance and I'll write up another blog post with the results a month from now to share those details.

Hope you're having a great week!

--Fitz

(Image from Discovery Education Resources)

DTP at JBoss

Dang there's some serious dust on this blog...

But I saw a demo this morning for some new functionality coming to JBoss Tools by the end of the year that uses DTP connection profiles and several views!

Teiid Designer provides tooling for the Teiid server, which has all kinds of cool functionality. As they say in the Teiid site - "Teiid is comprised of tools, components and services for creating and executing bi-directional data services. Through abstraction and federation, data is accessed and integrated in real-time across distributed data sources without copying or otherwise moving data from its system of record."

In the demo I saw this morning, I saw two disparate data sources - one Oracle, one SQL Server I think - that were then abstracted via a virtual database that could then access both. So this is sort of like ODA in BIRT, which combines data from different sources to create a particular report. But unlike BIRT which is largely a one way funnel of information to the report engine from what I understand, Teiid provides bi-directional support so you can use this abstraction to go back and forth between data sources.

Long story short, I am encouraged by the fact that - without much help from those of us left in DTP - the Teiid tooling guys managed to adopt the connection profile frameworks and reuse a good amount of tooling for their own stuff which had been on their own proprietary technology.

So a big congratulations to the Teiid guys for their work integrating DTP into their tools. :) As soon as our new milestone release of JBT goes live, I'll provide a link to their "New and Noteworthy" page so you can see some screen shots and more about what this cool tool does.

Hopefully we can see more of this kind of integration with DTP and Eclipse and possibly gain some more folks to help keep DTP alive for the future!

I'll try not to let my next post be another 5 months from now... Geez.

More EclipseCon 2010 Thoughts

Hi all...

I try to write up a summary each year I go to EclipseCon documenting what talks I went to, what I thought was interesting, etc.

I've done that here this year on a JBoss Wiki page: https://community.jboss.org/docs/DOC-15091

Ignore it if you don't find it useful, but I thought it was a great conference this year. :)

--Fitz

EclipseCon 2010 - Some thoughts

Hi all...

Yes, it's been a long while (again) since I had an opportunity to write here. But after such a great EclipseCon 2010 experience, I feel it would be a disservice to not talk about it. :)

After joining Red Hat in the middle of 2009, I entered the maelstrom of tooling development in the exciting, chaotic world of Red Hat and JBoss. Leaving Sybase was tough, but it was time to do something different after nearly 13 years at New Era and Sybase. But it was a shock to the system to go from the slow and steady of Sybase to the full speed ahead of Red Hat.

That said, I've had little time to spend on DTP at Eclipse. I feel bad about this. But I'm trying to help as much as I can, just like Brian and Hemant from IBM and Linda from Actuate. None of us can spend as much time as we might like on this great project.

So I went to EclipseCon 2010 with a bit of trepidation. How would DTP be received? We really don't have much to show in our Helios release beyond a few fixes here and there.

We had one talk for DTP this year - DTP in the Real World. I spoke about our improved RCP support in DTP and provided a plea for more community involvement. And Ruth Soliani, from the Eldorado Research Institute in Brazil, who spoke about how they used DTP in their MotoDev Development Studio product (with Motorola) to connect to a SQLite database on an Android device. Pretty darn cool stuff there and they were very kind to say nice things about the DTP team. :)

On the first night of the conference, we had a DTP Birds of a Feather (BoF) that was very encouraging. We met Ray and Chris from Ingres who will be helping us out from that side of the DBMS world where they can. We met Shenxue Zhou from Oracle, who we've been trying to help out as we can as well.

So the show of community support was awesome. At EclipseCon 2009, we had a very low attendance to DTP talks and nobody attended the BoF. This year we had 30-40 people attend the DTP talk and 3 people (beyond the DTP PMC) show up, plus one guy who was hoping DTP meant "Desktop Publishing" (Sorry about that!). We had great conversations with Ruth's team, the guys from Ingres, and Shengxoe from Oracle.

Now I'm actually hopeful DTP will survive. And that's a good thing!

A big thanks go to Oisin Hurley and Don Smith who put on a terrific conference this year. Every talk I went to had a good number of people attending and a few were standing room only (like Mik Kiersten's talk about Mylyn!). Not to mention the awesome keynotes and the giant moon robots that looked like something out of The Matrix. :)

Thanks to everyone who attended and especially the DTP community who's coming back to life!

--Fitz

Looking for DTP Hitchhikers...

Hi everyone...

Wow... It's been a while since I last wrote on this blog. I've been keeping busy at Red Hat since I hopped the fence back in June. :)

Unfortunately, with Sybase's reduction in DTP resources and a few other folks we've lost by attrition, we're suffering from a serious lack of help in the Connectivity and Enablement projects at the moment. Though Linda from Actuate helps out in Connectivity every now and then, it's pretty much just yours truly with occasional help from our friends at Codehoop. And my time these days is pretty minimal for DTP.

Talking with Brian Payton, our fearless leader these days in DTP, it came up that with the small core of folks involved it was like we were in a book "Hitchhiking Through Europe on Pennies a Day" - not getting too far, but when you do get picked up and make some progress on your journey it's often for a brief burst of speed and then you're out of the car on foot again.

So we're looking for a few hitchhikers to help us out.

I come to you, our friends in the DTP community and the broader Eclipse community, and ask for help. We're not asking for full time, devoted folks. We're asking that if you get some time and want to dig into DTP, we have lots of Bugzilla entries that could use some tender loving care.

If everyone who uses DTP would dig into one DTP bug every few months and provide a patch, I'm pretty sure our list of issues would decrease dramatically.

We already have a few mediums out there to communicate... We have the Eclipse DTP mailing lists and newsgroup (check out http://wiki.eclipse.org/DTP_New_Committer_Info to get the list of how to find those resources). And I just created a new IRC chat (#eclipse-dtp) on freenode that I'll be on any time I happen to be logged in. Maybe we can get the conversation going between other DTP community members and bring things back to life a bit!

Personally, I've been involved with DTP since 2006 and have really enjoyed working with everyone I've met. Who knows who you might meet? We can't offer fame or fortune for your contributions, but we can definitely offer our gratitude and most likely the gratitude of a grateful DTP community.

Who's interested in helping us out?

--Fitz

Greetings from the land of Red Hats!

Hi all...

It's been a while since I last posted anything on this blog, but I want to assure you it's not dead. Well, not quite anyway. My new position at Red Hat has been and will continue to be a challenge - a good one! But that means I have less time to spend on DTP topics...

I know I still owe (oh ye few who read these words) the remainder of the YouTube/DTP plug-in I promised way back in March with my EclipseCon 2009 talk. But I hope to get back to that sooner than later.

In the meantime, I wanted to congratulate Brian Payton from IBM for becoming the new DTP lead. He's a great guy who's kept a handle on the SQL Editor for quite a while now and I'm sure he'll do great for the rest of DTP too! I'll continue to help out where I can and be involved, but to a much smaller extent.

As such, if you see him around please welcome Brian P to his new position if you get a chance. I'm sure he'd be happy to hear from our community!

We're in the process of finishing up 1.7.1 for the end of September with the Galileo SR1 release train and will then see what we can do for Helios.

If you, our great community, have some extra cycles to help out, we'd love to have you join the party. Give us a holler on the mailing lists or newsgroups and let us know what you're thinking. The more the merrier!

Thanks a ton for your continued support of DTP and hopefully I'll get back on track with some new posts here before long. :)

--Fitz

DTPtv - Part 1 - Using YouTube APIs in Eclipse

Hey there...

So yes, I'm back to talking about DTPtv... (You can see my introduction to the series here.)

Image via Wikipedia

Now that we know generally what we want to do, we'll start by focusing on making the YouTube APIs usable in Eclipse and perform a simple test. Easy enough, right?

So the first thing I had to do was grab the YouTube jars from Google. I found those here. I grabbed the latest version of gdata-samples.java. The YouTube/Google APIs also have some dependencies, so I had to go out and grab imap.jar, mailapi.jar, pop3,jar, and smtp.jar. I was able to re-use a plug-in wrapper for javax.activation.jar from Orbit, which I'll talk about in a sec.

With all of the jars downloaded, I just had to create an Eclipse plug-in wrapper so they were available to other plug-ins. To do this is cake... Right-click in the Package Explorer, select New->Project, and in the list of Wizards select "Plug-in from existing JAR archives". Select your external jars (it will copy them into the plug-in wrapper). I named mine "com.google.gdata.youtube" and unchecked the "Unzip the JAR archives into the project" so it kept the jars as jars, not source code. Click Finish and watch the magic happen.

When it's done, I had to do one last thing to add a dependency. The Google APIs still depend on the javax.activation.jar...

Do you remember I mentioned the Orbit project? Well, you can read more about it here, but the idea is that it provides a repository for a number of third-party projects/resources that are shared across Eclipse. These have already gone through the IP process and are approved for our use. (Note however that even if a library is approved by the Foundation for use by all projects, project teams must still fill out a Contribution Questionnaire and notify the Foundation of their intentions to use a library). And there are quite a few of them. If you look at the latest build (from back in August 2008), you'll see 83 different packages available.

In this case, I just want javax.activation, so I locate it in the list, download it, and drop it in my Eclipse environment. Once the workbench picks it up, I can add it as a dependency in the MANIFEST.MF file for my wrapper plug-in.

Cool. So that pretty much wraps up my jar wrapper plug-in. Not too tough there.

So now what? Now that we have this plug-in, we can write a quick little application to do something with it.

I've gone ahead and created a simple class that does a very basic YouTube search...

package org.eclipse.datatools.sample.utube.sandbox;

import java.io.IOException;
import java.net.URL;

import com.google.gdata.client.youtube.YouTubeQuery;
import com.google.gdata.client.youtube.YouTubeService;
import com.google.gdata.data.youtube.VideoFeed;
import com.google.gdata.util.ServiceException;

public class UTubeUtils {

public static String TR_FEED_URL = "http://gdata.youtube.com/feeds/api/standardfeeds/top_rated";//$NON-NLS-1$

/**
* Return a list of video entries back to the calling method
*/
public static VideoFeed getResults(String author, String title) throws IOException, ServiceException
{
  YouTubeService myService = new YouTubeService(
          "", //$NON-NLS-1$
          "");  //$NON-NLS-1$

  String VIDEO_FEED = TR_FEED_URL;
  YouTubeQuery query = new YouTubeQuery(new URL(VIDEO_FEED));

  //set the author
  if( (author != null) && author.length() > 0)
  {
      query.setAuthor(author);
  }
  //set the actual query string
  if((title != null) && title.length() > 0)
  {
      query.setFullTextQuery(title);
  }

  //choose most viewed as the ordering
  query.setOrderBy(YouTubeQuery.OrderBy.VIEW_COUNT);

  //get the video feed
  VideoFeed feed = myService.query(query,VideoFeed.class);
  return feed;
}
}

So all that this really does is get a list of the top rated videos currently at YouTube. Pretty straightforward.

You'll notice a couple of things about the code. You have to have a developer key (your own ID as a developer or one for a particular company) and a client ID (seems you can have many of these). This basically lets the Google & YouTube APIs know that you're legitimately asking for data and aren't some rogue hacker trying to cause trouble. You can get these two items here. Once you have them, you can just create new constants for them and just use the constants.

So if you swap your developer key and client ID into the above code, it should do a quick search. But how should we test it?

I like creating new plug-in projects with example menu actions. To do this is pretty easy... Right-click in the Package Explorer, select New->Project, and in the list of Wizards select "Plug-in Project". Name it and set the plug-in ID and other info, and on the "Templates" page in the wizard, select "Plug-in with a popup menu." By default it keys off an IFile, so you can get to it from the Navigator or Project Navigator in the workbench.

It goes off and creates the basic code and you can then use your new utility class pretty easily by changing the run() method in the action class to look something like this:

    public void run(IAction action) {
   try {
       VideoFeed feed = UTubeUtils.getResults(null, null);
       if (feed != null) {
           if (feed.getEntries() != null && feed.getEntries().size() > 0) {
               System.out.println("Found some videos...");
               Iterator<VideoEntry> iter = feed.getEntries().listIterator();
               while (iter.hasNext()) {
                   VideoEntry entry = iter.next();
                   System.out.println("Video Entry: " + entry.getTitle().getPlainText());
               }
           }
           // clean up
           feed = null;
       }
   } catch (ServiceException se) {
       se.printStackTrace();
   } catch (IOException ie) {
       ie.printStackTrace();
   }
}

When you run the workbench and right-click on your action in the Navigator, you should see something like this in your development workbench console view...

Success! We have a plug-in wrapper for our YouTube jars and their dependencies. And we've verified that we can use those APIs in an Eclipse environment.

Next we have to figure out how to hook up YouTube to DTP and show a video in the workbench. And after that we can fine tune our look and feel to make it easier for our users.

Maybe this was a bit longer to write up than I'd thought originally, but this part of the process only took me about half a day when I was creating this code the first time.

Questions? Comments? Drop me a note here and I'll be happy to get back to you.

Next time I'll write about hooking up YouTube and DTP and how I used the built-in Eclipse web browser UI component to show videos.

--Fitz

Related articles by Zemanta

• I Want My DTPtv... (fitzdtp.blogspot.com)

DTP APis and How to Use a Transient Connection Profile

Hi there...

On the DTP newsgroup we had a question about using DTP APIs to create a new transient connection profile and then use that to execute some DDL...

It's pretty easy actually... The trick for the transient profile is knowing all the bits and pieces you have to have ahead of time, like the:

• provider ID, which is the connection profile type ID
  
• vendor and version, which relate to the vendor/version of the database you're connecting to
• and then the driver path. Note that you can also use a pre-defined driver and get the DriverInstance from the DriverManager, then retrieve various properties like the vendor, version, class name, and driver path from there
  

So you end up with something like this:

    private static String providerID = "org.eclipse.datatools.connectivity.db.derby.embedded.connectionProfile"; //$NON-NLS-1$
   private static String vendor = "Derby"; //$NON-NLS-1$
   private static String version = "10.1"; //$NON-NLS-1$

   private static String jarList = "C:\\Derby10.1.3.1\\db-derby-10.1.3.1-bin\\lib\\derby.jar"; //$NON-NLS-1$
   private static String dbPath = "c:\\DerbyDatabases\\MyDB"; //$NON-NLS-1$
   private static String userName = ""; //$NON-NLS-1$
   private static String password = ""; //$NON-NLS-1$

   private static String driverClass = "org.apache.derby.jdbc.EmbeddedDriver"; //$NON-NLS-1$
   private static String driverURL = "jdbc:derby:" + dbPath + ";create=true"; //$NON-NLS-1$ //$NON-NLS-2$

   public static Properties generateTransientDerbyProperties() {
       Properties baseProperties = new Properties();
       baseProperties.setProperty( IDriverMgmtConstants.PROP_DEFN_JARLIST, jarList );
       baseProperties.setProperty(IJDBCConnectionProfileConstants.DRIVER_CLASS_PROP_ID, driverClass);
       baseProperties.setProperty(IJDBCConnectionProfileConstants.URL_PROP_ID, driverURL);
       baseProperties.setProperty(IJDBCConnectionProfileConstants.USERNAME_PROP_ID, userName);
       baseProperties.setProperty(IJDBCConnectionProfileConstants.PASSWORD_PROP_ID, password);
       baseProperties.setProperty(IJDBCConnectionProfileConstants.DATABASE_VENDOR_PROP_ID, vendor);
       baseProperties.setProperty(IJDBCConnectionProfileConstants.DATABASE_VERSION_PROP_ID, version);
       baseProperties.setProperty( IJDBCConnectionProfileConstants.SAVE_PASSWORD_PROP_ID, String.valueOf( true ) );
       return baseProperties;
   }

   public void createTransientDerbyProfile() throws Exception {
       ProfileManager pm = ProfileManager.getInstance();
      
       IConnectionProfile transientDerby = pm.createTransientProfile(providerID, generateTransientDerbyProperties());
       // do something with the profile
      
   }

And then once you have your transient profile, connect, get the Java connection object, and execute your DDL...

        IStatus status = transientDerby.connect();
       if (status.equals(IStatus.OK)) {
           // success
           java.sql.Connection conn = getJavaConnectionForProfile(transientDerby);
           if (conn != null) {
               try {
                   java.sql.Statement stmt = conn.createStatement();
                   java.sql.ResultSet results = stmt.executeQuery("<INSERT QUERY/DDL HERE>");
               } catch (java.sql.SQLException sqle) {
                   sqle.printStackTrace();
               }

           }
          
       } else {
           // failure :(
           if (status.getException() != null) {
               status.getException().printStackTrace();
           }
       }

So not too bad. Great question though! Hope this helps!

--Fitz

I Want My DTPtv...

Hi again...

So one of my presentations at EclipseCon 2009 was "DTPtv and Other Wacky Ideas."

Was this truly a wacky idea? Probably. Who would have thought to merge Eclipse, YouTube, and DTP in one go?

My goal was two-fold... First, I wanted to show that DTP can be used for something more than just databases. Far too often we're pidgeonholed as a provider of database tools. And though we do that, we do much more as well. Second, I wanted to do something out of the usual box, and YouTube is pretty dang far out of the box...

Basically I focused on three different things...

1. Make the YouTube APIs accessible and test them.
2. Create a YouTube Search connection profile in DTP so you could create and manage multiple searches in a variety of ways.
3. Create a viewer that would allow a user to take advantage of YouTube searches on the fly and see the results within the Eclipse IDE or in an RCP application.

Now why would you want to do this you might ask? Good question.

Image via CrunchBase

Let's say you write Eclipse RCP applications for a company with a healthy education department. They want to help out beginning users by recording tutorials and putting them up on YouTube. Not only will it help your users, but it works as a bit of helpful marketing for your sales staff. And now that they're doing this, they want a way to be able to take advantage of these YouTube videos right inside the application.

Maybe this isn't so far fetched after all?

In the next series of blog posts, I'll focus on the three steps I took to get this all working. I demoed working code at the conference and will clean it up a little, zip it up, and have it available on a file sharing site soon. I hope to also contribute it back to the Examples project at Eclipse, as it crosses project boundaries.

As always, please feel free to ask questions or make comments as I go through the process. Many of you will find this old hat, but I'm hoping someone can take advantage of the information. :)

Thanks!
--Fitz

Related articles by Zemanta

• EclipseCon 2009 Post-mortem (fitzdtp.blogspot.com)

EclipseCon 2009 Post-mortem

Hey all!

I'm still in the process of getting my act back together after EclipseCon 2009 last week in Santa Clara, California. Was a great conference as per usual, and a bit of an eye-opener for me at least as far as the excitement (or lack thereof) for DTP in the conference community.

View Larger Map

First, let me say that I am very excited about Amazon's announcement about the AWS Toolkit for Eclipse. Eclipse in the Cloud is a very cool concept and one that I think will prove to be fertile ground for all sorts of interesting applications. The exciting part for me was hearing that the AWS Toolkit team is still looking at using DTP to help them integrate with their SimpleDB database in the Cloud in a future release of the toolkit.

Anything we at DTP can do to help with that effort would be very educational and cool for us. And as always, we'll help wherever we can. :)

Second... There were a TON of great talks at EclipseCon this year. Though E4 seemed to get most of the buzz, I think that runtime talks were running a close second. After attending a number of talks in all parts of the spectrum (business, coding, new tools, and so on), I came away with a ton of cool ideas I'd love to explore if time allowed. Things from exposing DTP as OSGi services, to using the Data Binding APIs, to using Zest to possibly help to visualize a database schema and other things. I have a loooong list of things to explore.

And third, I was bummed about the lack of interest in our DTP talks this year. We had a few people show up at the DTP tutorial on Monday, and I had a few people at my DTPtv talk, but we ended up with an empty room for our set of short talks at the end of Thursday.

Now, I know that DTP is really more of a "plumbing" project and I get that plumbing isn't sexy. But when people put time and effort into creating presentations (and in one case fly from Germany to present), it's disappointing that nobody wanted to listen.

Yes, there were probably extenuating circumstances. I know that many people left the conference early to catch flights and there were other talks that were probably much more exciting (I missed John & Max's talk in the same slot and would have liked to attended). But still...

Even with that bummer deal, I still think it was an excellent conference. Scott Rosenbaum and the program committee (of which I was a very very minor part) did a great job recruiting amazing keynotes and cool folks to come talk about their technologies. And as I said, I came away brimming with ideas for DTP.

So maybe next year will be a better year for DTP... Who knows?

I'll be writing up one or two more posts about my EclipseCon experience this year, including a post about my DTPtv talk, which (though it went short), I had a great time doing.

Thanks to everybody involved in the conference this year. Let's do it again next year!

--Fitz

Related articles by Zemanta

• New AWS Toolkit for Eclipse (aws.typepad.com)
• Amazon stresses cloud opportunities for developers (infoworld.com)
• Cloud Day at EclipseCon (ianskerrett.wordpress.com)

Cool DTP Talks at EclipseCon 2009

Hey all!

I'm always amazed by the depth and breadth of talks at EclipseCon and this year is no different. In the Data Tooling category, we have a diverse set of talks (you can see the list here) on everything from some new tooling we've been working on, updates to the Graphical SQL Query Builder, how Ingres is rolling DTP components in new and unique ways, to using YouTube in DTP and how a commercial vendor (IBM) is using and extending DTP APIs for their PureQuery product.

I have to admit I'm sort of partial to the YouTube presentation I'm doing :), but I'm very curious to hear Ingres and IBM talk about their tooling and products and how DTP is playing a role in those.

For those of you just getting started with DTP, we have a tutorial scheduled for the Monday of the conference that's going to walk through a ton of topics from adding a new JDBC driver to the mix, to supporting a new database, and customizing SQL syntax and so on...

DTP is much more than just a great set of tools for data access... It's a great community. And EclipseCon is when that community comes together.

Come join our community!

--Fitz

Related articles by Zemanta

• Smoothly Blending Java and SQL with pureQuery
• New Open Source Database Offerings from Ingres and Sun
• Data Tools at EclipseCon 2009

Out of Memory Errors...

Image via WikipediaHi all...

I recently spent about a day and a half chasing my tail trying to track down some bizarre out of memory errors I was running into with Ganymede SR1 when debugging some DTP stuff and thought I'd pass along what I learned. I was continually running out of heap space while debugging, which was very troubling.

At first, I thought I had somehow corrupted my workspace. So I created a new workspace and tried it again (several times actually) and ended up with the same problem.

Next, I thought I'd try the Galileo Platform Galileo M3 build (along with the associated GEF, EMF, and DTP builds) and see if I got the error. At first I didn't, and I thought 3.5 might have fixed the problem... [sigh] No luck.

So then I started poking around on Google, trying to figure out how to bump up my heap space.

I kept running across this suggestion (which after thinking about it was kind a "well duh" kind of thing):

"Add -Xms256M -Xmx512M to the VM arguments for the runtime configuration or on the main Eclipse command line when you start it up..."

Lo and behold it works now in Galileo and Ganymede. Life is good.

But it was a frustrating end to the week, only to discover that I was just oblivious to the simple solution. :(

Hope this saves someone else from the pain and suffering!

--Fitz

Related articles by Zemanta

• New DTP Ganymede Video at Eclipse Live
• DTP T-shirts... Get your t-shirts here...

Creating the SQLite Connection Profile UI bits

Hi there!

So now we're almost to our first functional version of a SQLite connection profile in DTP. We have a driver-wrapper plug-in, a driver definition, an overridden catalog loader, and a connection profile with its associated connection and connection factory classes. What's next? Why, adding the UI so you can create the profile, of course!

With versions of DTP prior to Ganymede, this was a more difficult, but not horrible task. In Ganymede, we've reduced the amount of work to adding three extension points and writing four key classes in a org.eclipse.datatools.enablement.sqlite.ui plug-in (basically just extending these classes a tad, so little real work involved):

• A connection profile wizard extension that uses the org.eclipse.datatools.connectivity.connectionProfile extension and uses the newWizard node so we can define our wizard
• A property page extension (org.eclipse.ui.propertyPages) to define a property page so we can edit our SQLite conneciton profile instances
• A driver UI contributor extension (org.eclipse.datatools.connectivity.ui.driverUIContributor) to create a reusable UI component that gathers the information we need for our SQLite connections
• A connection profile wizard class that extends the org.eclipse.datatools.connectivity.ui.wizards.ExtensibleNewConnectionProfileWizard class
• A connection profile wizard page that extends org.eclipse.datatools.connectivity.ui.wizards.ExtensibleProfileDetailsWizardPage
• A connection profile property page that extends org.eclipse.datatools.connectivity.ui.wizards.ExtensibleProfileDetailsPropertyPage
• And a driver UI component that is used on the wizard and property pages that implements org.eclipse.datatools.connectivity.ui.wizards.IDriverUIContributor

It may look daunting, but really it boils down to a few extension points, a few extended classes, and an instance of poor man's inheritance (copying a class from another project).

So let's get started!

DTP's Been Babel-ized!

Image via Wikipedia Hi all!

Just thought I'd share some cool news from DTP land. I received word from Denis (via Bugzilla) that the initial translations for DTP's 1.6 (Ganymede) release have been added to Babel! Yay!

I would like to thank the folks who helped get us going with translations in Babel:

• Antoine Toulmé
• Yasuo Doshiro
• Denis Roy
• and Kit Lo

So thanks to everybody who helped get us started. Pretty soon we'll be able to have DTP in Klingon (probably not, but it's an interesting concept at any rate!)!

--Fitz

Creating an Actual SQLite Connection Profile (minus the UI)

Hi there!

So now we have the majority of our work done. We have a driver-wrapper plug-in, a driver definition, and an overridden catalog loader. What's next? Wrapping the functionality in a nice, easy to use connection profile!

Note: Previous articles in this series cover the following topics: Catalog Loaders, Driver Templates, and the Driver Framework.

Many moons ago, we talked at a high level about the driver template & driver definition frameworks. It's now time to talk briefly about the connection profile framework.

It all boils down to this... A connection profile manages a connection to something. Right now in DTP we connect to JDBC databases and file systems for the most part. But the Sybase WorkSpace product also uses DTP to connect to application servers, LDAP, UDDI repositories, and much more. So it's not limited in any way.

With that in mind, a JDBC database connection profile, such as the one we want to create for SQLite, just needs to manage a JDBC connection under the covers. We'll add a layer on top of that to attach the SQL Model to the connection so we can display the database specifics in the Data Source Explorer tree.

In the DTP Ganymede (1.6) release, we've really simplified creating a new connection profile if it's associated with a db definition vendor/version and a driver template. So we'll take advantage of that for SQLite.

To create a connection profile, we will go to the org.eclipse.datatools.enablement.sqlite plug-in project and create a couple of classes and two extension points. These steps are kind of chicken & egg - the order isn't really important so long as you get them all done.

Step 1: Create a new connection factory and connection class for SQLite. These are the actual raw connections that our SQLite connection profile will manage for us.

The connection class is pretty easy. We're just going to extend the Generic JDBC JDBCConnection class for SQLite so we have our own specialized version of it.

That code looks like this:

package org.eclipse.datatools.enablement.sqlite.connection;

import org.eclipse.datatools.connectivity.IConnectionProfile;
import org.eclipse.datatools.connectivity.db.generic.JDBCConnection;

public class SQLITEJDBCConnection extends JDBCConnection {

/**
* @param profile
* @param factoryClass
*/
public SQLITEJDBCConnection(IConnectionProfile profile,
                              Class factoryClass) {
super(profile, factoryClass);
}
}

The connection factory requires a little more work, but not much more:

package org.eclipse.datatools.enablement.sqlite.connection;

import org.eclipse.datatools.connectivity.IConnection;
import org.eclipse.datatools.connectivity.IConnectionProfile;
import org.eclipse.datatools.connectivity.db.generic.JDBCConnectionFactory;

public class SQLITEJDBCConnectionFactory extends JDBCConnectionFactory {

public SQLITEJDBCConnectionFactory() {
super();
}

public IConnection createConnection(IConnectionProfile profile) {
SQLITEJDBCConnection connection = new SQLITEJDBCConnection(profile, getClass());
connection.open();
return connection;
}
}

Basically in the connection factory, we're just creating one of our new SQLiteJDBCConnection class instances for the profile that's passed in.

Step 2: We want to add a new extension point to the plugin.xml in the org.eclipse.datatools.enablement.sqlite plug-in project: org.eclipse.datatools.connectivity.connectionProfile.
This extension point has a couple of nodes we're going to create beneath it: connectionFactory and connectionProfile.

Let's define our connectionProfile first, so we have the connection profile ID to use for the connectionFactory.



You can see from the screen that we're giving our connection profile the following properties:

• id = org.eclipse.datatools.enablement.sqlite.connectionProfile
• category = org.eclipse.datatools.connectivity.db.category (this ensures that our connection profile shows up under the "Databases" category in the DSE)
• name = SQLite Connection Profile
• icon = icons/jdbc_16.gif (you can copy this from the Generic JDBC connection profile plug-in)
• pingFactory = org.eclipse.datatools.enablement.sqlite.connection.SQLITEJDBCConnectionFactory (our new connection factory class we created in step 1)

Then we define our connectionFactory:



Our connectionFactory extension has the following properties:

• id = java.sql.Connection (this maps to the type of connection this connection factory/connection class maps back to -- in this case, a JDBC connection)
• class = org.eclipse.datatools.enablement.sqlite.connection.SQLITEJDBCConnectionFactory (our connection factory class)
• profile = org.eclipse.datatools.enablement.sqlite.connectionProfile (our SQLite connection profile ID from the connectionProfile extension)
• name = SQLite Connection Factory

So now we have a connection profile for SQLite in DTP. Now all we need is a user interface (wizard, wizard page, property page, and driver UI) and we'll be golden!

That's what we'll cover next time.
--Fitz

Related articles by Zemanta

• Adobe throws weight behind SQLite
• Adobe funds SQLite database

DTP SQLite support continued... on to Catalog Loaders...

Hi all...

Yes, it's been a while since I wrote the last article in this series. I apologize for that. This summer has been busy both at work and home (not like I got to go on a cruise like Ed Merks or anything, but we've been bouncing around!). So the series of articles fell by the wayside a bit.

That said... Let's get back to it. When we last left our intrepid coders, we were working through the steps of trying to get SQLite to connect and show its underlying model (schemas/tables/procedures, etc.) in the Data Source Explorer. We had just finished creating some driver templates (see that article here) and that didn't really buy us all that much.

The next step is to then create a custom catalog loader to take care of any shortcomings of the SQLite driver. We have a db definition (vendor/version) to hang the catalog loader from, which means we just have to choose which level to focus on first.

In this case, since we're interested in schemas as the highest level of the model for SQLite, we'll override the schema catalog loader.

To do this, we need to do a things.

1) In the plug-in manifest editor (opened either from MANIFEST.MF or plugin.xml) for the org.eclipse.datatools.enablement.sqlite plug-in project, we need to add a new dependency. Select the Dependencies tab in the manifest editor and add org.eclipse.connectivity.sqm.core. This will provide one of the extension points that we need to extend to override the catalog loader.

2) On the Extensions tab, add a new extension and select org.eclipse.datatools.connectivity.sqm.core.catalog. Right-click on the extension and select "overrideLoader". Then select the "overrideLoader" node in the extension tree. You should see something like the following:


Product and version equate directly to the db definition that we've already created. Provider is the actual loader class we'll override the default with. And eclass is the SQL model class that we want to override the loader for our SQLite databases.

In this case:

* product = SQLITE
* version = 3.5.9
* eclass = org.eclipse.datatools.modelbase.sql.schema.Schema
* provider = a new class we'll create called SQLiteSchemaLoader


The SQLiteSchemaLoader looks something like this:

package org.eclipse.datatools.enablement.sqlite.loader;

import org.eclipse.datatools.connectivity.sqm.core.rte.ICatalogObject;
import org.eclipse.datatools.connectivity.sqm.loader.IConnectionFilterProvider;
import org.eclipse.datatools.connectivity.sqm.loader.JDBCSchemaLoader;

public class SQLiteSchemaLoader extends JDBCSchemaLoader {

public SQLiteSchemaLoader(ICatalogObject catalogObject,
    IConnectionFilterProvider connectionFilterProvider) {

    super(catalogObject, connectionFilterProvider);

}
}

Since SQLite has no concept of a "schema", we need to dummy one up so that the model is satisfied. (Yes, this is one more case where the loose JDBC "standard" bites us in the rear when we try to adhere to it.) To do that, we really only need to focus on overriding a couple of key methods:

* protected void initialize(Schema schema, ResultSet rs) throws SQLException
* public void loadSchemas(List containmentList, Collection existingSchemas) throws SQLException

In the default JDBCSchemaLoader, it relies on the driver to provide a result set of schemas. In the SQLite case, since there are none, we need to change the behavior to just dummy up a schema object and pass it along.

So loadSchemas becomes:

public void loadSchemas(List containmentList, Collection existingSchemas)
throws SQLException {
Schema schema = (Schema) getAndRemoveSQLObject(existingSchemas,
        "DEFAULT");
if (schema == null) {
    schema = processRow(null);
    if (schema != null) {
        containmentList.add(schema);
    }
}
else {
    containmentList.add(schema);
    if (schema instanceof ICatalogObject) {
        ((ICatalogObject) schema).refresh();
    }
}
}

And initialize is changed to ignore the result set and just name the schema "DEFAULT":

protected void initialize(Schema schema, ResultSet rs) throws SQLException {
schema.setName("DEFAULT");
}

The last thing we have to do is make our catalog loader class actually run as an executable extension. If we don't add the following constructor, you get an InstantiationException, which is always a pain to track down:

public SQLiteSchemaLoader() {
super(null);
}

Once the dummy schema is in place, the driver actually does return the tables list correctly, as seen in the following screen shot.



However... It appears that the SQLite driver does not return getImportedKeys() directly (it throws a "not yet implemented" SQLException for me), so we will need to clean up the JDBCTableConstraintLoader before we're done, but we can do that during code cleanup. (We'll also have to remove some of the nodes that don't make sense for a SQLite database, such as Authorization IDs, Stored Procedures and User-defined Functions.)

So now we have a (mostly) working catalog loader that will connect to a SQLite database and allow us to drill in and see tables and columns.

In the next article we'll walk through the simplified process for creating a brand new connection profile (wizard, wizard page, property page, connection factory, and connection classes). And then we can talk about the clean up phases and some of the nice things we can do to help out our users and developers.

Hope that helps!

--Fitz

How are YOU using DTP?

Hi all!

Now that the Ganymede release has gone out the door, DTP 1.6 has been released into the wild.

We in the DTP project would like to know how you are using DTP -- either as an end user, an adopter, or an extender. Are you using it to help with day to day database development tasks? Are you integrating with it from different projects in Eclipse such as BIRT, WTP, or JPA?

Though we're working on our first maintenance release for September, we're also starting to plan our next major release due in June 2009 along with the rest of the Eclipse Release Train. So we want to know what you would like to see in the next major release.

What do you like? What don't you like? We're here to help our community grow and develop. But to help us do that, we need guidance from the very folks who are using our stuff or looking at using it.

Let us know! Either by leaving a comment on this blog entry or by posting a message to the DTP newsgroup or mailing list!

Thanks!
--Fitz

New DTP Ganymede Video at Eclipse Live

Hi all!

Just thought I'd let you know as part of the Ganymede release, Ian had most of the main projects create 15-20 minute videos to include on Eclipse Live to highlight what's new and cool in Ganymede.

Well, I'm happy to say that the DTP demo is live. :)

You can take a look at it here.

Please be gentle, as it's my first live demo video. :)

Thanks!
--Fitz

DTP T-shirts... Get your t-shirts here...

Hey all...

In honor of the Eclipse Ganymede release and DTP's 1.6 release, I put together a t-shirt design so you can share your DTP love with the world. It's not much, but it's colorful and after one revision yesterday (thanks Linda!) I think it includes a majority of DTP projects, components, and terms in a creative way.



Pick one up to share with your friends and coworkers. :)

Thanks to everyone involved with DTP's Ganymede release!

--Fitz