In this interview Richard Bair answers my questions on Swing, SwingX, JavaFX and Java on the desktop in general, saving some of the announcements for the next week’s Devoxx conference. We wanted to do this interview immediately after my thoughts on the state of core Swing, but the JavaFX release has postponed it up until now.

Tell us a little bit about yourself.

Like many developers I know, I started programming because I wanted to write games. Somewhere along the line my dad convinced me to write him a database backed application for his business, and that’s how I got started.

Since then I’ve been predominantly an application developer. It was in that role that I started working with open source software, and eventually started working with JDNC. I was subsequently hired by Sun and have been the SwingLabs lead since that time.

It hasn’t been all smooth, I learned a lot about open source community building as I went along, and certainly have plenty of faults in getting SwingX releases out regularly. But my overall goal and desire has always been to make building real world applications easier.

I know in the recent past I’ve been involved a lot in Java demos and such that are consumer in nature. I like building applications, either consumer or enterprise. Consumer applications are closer to what got me started in programming in the first place, while enterprise apps are really where I cut my teeth (one of a handful of engineers I’m sure who love both big iron databases and pushing pixels ;-)).

Why was SwingLabs created, how did it evolve and what is its current state?

SwingLabs was a bold attempt by the Java Client Group to engage the open source community long before OpenJDK came along. Originally the brainchild of Amy Fowler and Hans Muller (and probably others on the team), the idea was to have an open source venue for building APIs that fill in the missing pieces of what is shipped with Swing.

As any Swing developer can attest, you have to build a lot of plumbing to build real apps with Swing today as shipped with the JDK. In the last few years a great many open source projects have been created to help fill these gaps.

SwingLabs was also designed as a place where these APIs and experiments could be vetted and included in the JDK. Our score in this regard is not so good. We pulled in a lot from the JDIC project, and we pulled some of the filtering and sorting work in.

We will be pulling more for JDK 7 as well, expect to see 2 or 3 components at least and also one or two big JSRs.

SwingLabs is currently run predominantly by the community which is really working I think pretty well. The dedication of our team of open source developers is a credit to them and to the open source reality that we live in.

We do need more involvement from the Sun side, and will be making an announcement in this direction at Devoxx next week.

If you were to start over with SwingLabs / SwingX, what would be the most important parts in building and maintaining the community involvement?

Regular releases. Regular content added to the SwingLabs.org website. More responsive bug fixes (my fault entirely).

Would you like to provide your side of the story behind the decision on the layers and painters in SwingX?

The fundamental idea behind painters was to decouple the rendering of components from the components themselves, such that you wouldn’t have to subclass the component to change its rendering (or replace its LAF which is a much more complicated proposition). The corollary was that there could be a collection of default painters that people could reuse.

There was an argument made primarily by Mikael Grev who had a similar API in his MIG Calender component (if I remember correctly) that we should have components that support painters to have an arbitrary sequence of painters. Sort of like JLayeredPane but for painters. I know Scott, Romain, Josh, and I didn’t particularly like it for technical reasons (which are all documented in the forums), and there was both support and skepticism from other people outside Sun.

At the end of the day, the project lead has to make a call on API, and I made the call I felt most comfortable with.

How likely is it to see the SwingX components being part of core Swing in JDK 7 and beyond?

All of them? Unlikely. Some of them? Extremely likely.

Originally Nimbus was planned to be developed under the SwingLabs roof, but was later put into the internal repository. Would you like to see the development of new Swing-specific functionality in the open before it gets put into the core distribution?

Nimbus is a special case. To properly implement Nimbus required a significant number of fixes and enhancements to BasicLookAndFeel, Synth (lots of bug fixes for Synth) and other core APIs and implementations. We simply could not have delivered Nimbus in the open source without these fixes. Also, Nimbus went into Java 6 Update 10 which was not part of OpenJDK. When Nimbus is forward ported to Java 7, it will be in OpenJDK.

I would prefer that all development was done as part of a community process, be it SwingLabs or an open JSR or OpenJDK.

Do you think that maintaining backwards compatibility has unnecessarily restricted the Swing team in moving this UI toolkit forward?

No I don’t. If you gave up backwards compatibility you’d have a whole other set of issues that are many times worse. The level of risk one developer is willing to take isn’t the same as another. For every one person who wishes we’d ditch backwards compatibility there are another 50 that would be violently opposed when that advice rendered their applications (or development of future versions of their applications) inoperable. Especially with a technology as mature as Swing.

To those people who say we should make a Swing2 which is not backwards compatible, I would say, this is exactly what we’re doing with JavaFX.

I also want to emphasize that Swing isn’t going away. We have a tremendous number of customers building Swing apps and that will obviously be supported and extended going forward. Also, we will allow you to mix JavaFX both into Swing apps and mix Swing in JavaFX.

Both Microsoft and Adobe view “line of business” applications as a very important target segment for their offerings (WPF, Silverlight, AIR, Flex). Is JavaFX only targeting the rich UI development (graphics, effects, animations), or do you plan to include modern business components such as pivot tables, grid trees and date pickers?

JavaFX 1.0 is oriented toward creative content, very similar to Silverlight 1.0. Future versions of JavaFX will be designed to also handle full blown enterprise rich client application development.

What effort is planned to be put into the core Swing in 2009?

We’ll have announcements for these plans at Devoxx.

Anything else you would like to add?

As an application developer, and as a toolkit developer, JavaFX is very exciting for me. The first release is really aimed at providing the foundation for all subsequent work. We’re focusing on animation, graphics, media, and related APIs with an eye towards the future so that we have a solid foundation upon which to add user interface Controls, frameworks etc. JavaFX gives us the freedom to design the best possible design we can, without any API backwards compatibility constraints. It really is an exciting opportunity. We’re committed to providing interop between FX and Swing so that people can build the kinds of applications they want to, and can mix and match the two.

There’s nothing like the right link at the right time. Remy Rakic (Twitter alias lqd) has left a link to the apparent release jars of JavaFX Desktop 1.0 in the comments on the previous entry. So i’ve decided to see whether there is anything new in the JMC department since the last time i’ve looked at it.

In the JavaFX preview, the native code was shipped directly in the jmc.dll. The download folder has the native libraries available in three platform-specific jar files (one for Windows, Mac OS X and Linux each). For Windows, the jar contains four DLLs:

• jmc.dll as before
• on2_decoder.dll which would be presumably responsible for the new licensed decoders
• msvcp71.dll and msvcr71.dll which are parts of Visual C++ 7.1

Assuming that this download folder is going to be used as a centralized WebStart repository, i would venture to say that there is going to be some kind of built-in mechanism for extracting these DLLs at runtime and updating the java library path. However, in my experiments i had to extract them manually and point the -Djava.library.path to the folder that contains those DLLs. We’ll wait and see whether the final (and official) release will follow in the footsteps of SWT 3.3 that handles the locally run extraction automatically.

Apart from the new JMediaView class that is a lightweight Swing component I haven’t seen any new classes. However, this JMC drop can play all QuickTime content from the sample page, including H.264, MPEG-4, 3GPP and MPEG-2, as well as the full scale trailers from the Apple site (in H.264 format). The playback support goes a long way towards my number one wish for desktop Java in 2008, and it will be hopefully followed by the recording capabilities as well.

I’ve decided to dive a little bit deeper into the JMC internals and updated the JMCFrame class to show a skeleton implementation of a “home-grown” player. Instead of using ready-made classes such as JMediaPlayer, JMediaPane or JMediaView, i’m creating a MediaProvider from the specific URI and attaching a VideoRendererListener to its VideoRenderControl (i would guess that the JMedia* classes follow the same route, but without the source code it’s hard to tell). This certainly makes the code a little bit more complicated, but allows much greater flexibility as far as compositing and overlaying go. Here is a screenshot of the player in action:

What do you see in the screenshot? There is a JPanel with a JButton control in it. The JPanel overrides the paintComponent and paints the following three layers:

• Gradient checkered background.
• The contents of the current video frame with 80% opacity.
• Information on the current media location, total duration and rate.

Using a MediaProvider with a VideoRendererListener allows some interesting things – overlaying translucent controls and HUDs, cross fading between multiple video streams and rendering video streams on arbitrary Swing components.

The new implementation supports dropping two kinds of video content:

• Local file
• URL

If you run the sample code locally, drag a local video file on the panel and press the play button. The content should start playing. You can also drag a URL from your favourite browser. If that URL points to a video stream, the stream will be played – tested on streaming H.264 movie trailers from the Apple site :)

If i had to choose a particularly nifty API, that would be the MediaControl in the com.sun.media.jmc.control package. To get the specific control, call MediaProvider.getControl passing the class instance of the required control (such as VideoRenderControl). The MediaProvider.getControls API will enumerate the supported controls. Here is what i see on my machine when playing video files: play, audio, video, subtitle, video render and track. Hopefully we are going to see the record control joining this group soon.

Here are some Swing links that you might have missed during the last week:

• Jeremy Wood has been a busy blogger this week. First, he compares different approaches in using spinners for specifying length measurements in different units. Next, he presents a dialog to layout several tiles / cells on a print page. And finally, he closes the week with a partial implementation of Mac “Windows” menu that tracks the list of currently open top-level windows and their respective states.
• Olivier Gout has a prototype of WYSIWYG theme designer for Nimbus. The “official” Nimbus designer promised at this year’s JavaOne is nowhere in sight, and only time will tell whether this new attempt will outlive the growing list of Synth designers / builders mentioned in Swing Links of the Week edition of March 23, 2008.
• Patrick Ahlbrecht has a skeleton implementation of persisting window geometry and toolbar location between successive launches of an application window.
• Kai Toedter has written a demo application that shows how to use OSGi Declarative Services (DS) and Spring Dynamic Modules (Spring DM) together with Swing UI (and the Swing Application Framework, JSR 296). The Person Manager project is licensed under EPL.
• William Chen has announced his work on the Visual Swing Designer for Eclipse.
• Adam Bien continues to favor Swing over SWT in his two-part attack. Starting with how to make your Swing coding more efficient, he then continues to address the perceived Swing weak spots. From reading these and previous postings, looks like Adam holds a particular dislike towards SWT tables, and i couldn’t disagree more. Just like it takes quite some time to fully understand your way around Swing tables (including models, renderers, editors, listeners and much more), it takes a little bit more than a few suggested hours to switch to the SWT / JFace approach. This is not to say that one is clearly better than the other. It’s just that if you come with prejudice or a very strong knowledge in one, the other will seem clunky, over engineered and convoluted.
• Two interesting Swing-related threads on the Google group for Javaposse. The first addresses the misunderstanding of the painters issue and the community handling throughout the entire process, and the second calls for community questions on JavaFX.
• Andres Almiray has announced release 2.0.1 of JIDE Builder and 0.1 beta release of GfxPad.
• Roman Kennke continues his work with Java2D, writing about first Java2D pipeline implementation on top of the DirectFB hardware abstraction layer.
• Ken Orr has announced release 0.9.3 of Mac Widgets for Java. Major new addition in this release – iApp style scroll bars. Is it me, or is the reflection of the new scroll bars incorrectly flipped?
• Clemens Eisserer reports that his XRender pipeline has been integrated into the IcedTea fully open sourced port of OpenJDK.

In three days, JavaFX Desktop 1.0 will be officially released. I have been skeptical about JavaFX over the last few months, and while this may be attributed to general inclination of programmers towards being “entrenched” in the technologies that they know, there are other reasons as well. Aside from a few unnamed technology partners that were participating in shaping the capabilities of this release, the community has been effectively shut from lending its hand in commenting, testing and evolving the platform. We have long heard about “being blown away” by the new functionality being brought by JavaFX to the RIA market, and this week we will finally see if it is going to live up to the marketing hype.

Personally, i will be very glad to see at least one out of the following ten items being available for JavaFX, as well as cross-platform Swing applications come December 4:

1. 6u10 / 6u11 for Apple – [update] see bug 6761033 for some interesting news.
2. Full java lightweight rendering of SWF files even without installed Flash engine
3. Full java lightweight rendering of H.264 files even without installed codecs
4. Adaptive video streaming based on the client CPU and network utilization
5. Video capture and creation
6. Fully skinnable lightweight browser component based on WebKit or Mozilla / Gecko
7. Inclusion of form-oriented layout manager
8. Support for high level shader language (HLSL) for hardware accelerated custom effects
9. Nimbus designer
10. JavaFX.com with JavaFX-only applet driven content

And is it me, or are we going to see Sun ditching its own module initiative and move towards restructuring JDK as a collection of OSGI bundles?

Here are some Swing links that you might have missed during the last week:

• Ken Orr explores an interesting usability side of Swing buttons with popup menus. The solution involves using an unpublished “doNotCancelPopup” client property supported by the current implementation of popup handling in core Swing. This may be different in other JVM implementations, and may even break in the future Swing versions (although the later is quite unlikely). Thanks to this post i have also updated the Flamingo command buttons to dismiss popup menu / panel on the second click. The implementation does not use the above client property, since Flamingo has its own code to manage popups.
• Gregg Bolinger has kicked off a series of posts on blueprints for well-written Swing applications (part 1, part 2, part 3 so far). Project Maddie has been created to provide the collaboration grounds for interested developers, and it will be interesting to see how this pans out after the initial excitement is gone. I have talked about the lack of Swing blueprints in May 2006 (just after JavaOne 2006), and unfortunately nothing has changed as far as the proper platform documentation goes.
• Jeremy Wood delves into the intricacies of using existing alpha composites to do cross fades between two translucent images. After finding out that the existing alpha composites are not up to the job, he tries doing direct manipulations on the underlying buffer. As this turns off the hardware acceleration, the time and memory performance were disappointing.
• Gabriele Carcassi provides a solution for registering JavaScript functions to respond to Swing events on applets.
• Following his experimentations in the SwingX incubator, Matt Nathan has decided to create a separate project for scalable icons.
• Geertjan Wielenga writes about the JSyntaxPane project, mentioning the supported editor kits, find / replace dialog, code completion configuration, color styles and more.
• Gregg Bolinger vents his frustration over the opaque and very slow bug fixing process of core Swing bugs. Working on drag-and-drop between tables and lists, he has hit a bug reported in 2003 and not fixed since then. In the next post he presents (a rather poorly formatted) a solution that he found on JavaRanch came up with.
• Wolfgang Zitzelsberger has announced release 2.8 of Synthetica look-and-feel. New in this release are the SyntheticaSimple2D theme, improved translucency / border support for comboBox / spinner / text components, transparent window support for 6u10, improved RTL component orientation support.
• Jean Francois Poilpret has posted the slides to his Jazoon 08 presentation on the JSR 296 (AppFramework).
• Greg Brown shows two examples of building Pivot applications that are wired with JavaScript and Groovy.
• Jeremy Wood talks about his implementation of Aqua-like preference panel thats supports both single-row and multi-row layouts, along with cross fading between the components.
• And finally, Collin Fagan kicks off the new series on exploring layout prototyping. The first part documents his attempts to apply XML to the UI layouts. Layout managers surveyed in this part are absolute, table, grid bag and Mig. It certainly looks like the desktop heavyweights (Adobe and Microsoft) have adopted XML as the lingua franca to enable easier collaboration between designer and developer tools. And while the hardcore developers still prefer writing and tweaking those XML files by hand, both sides of the respective toolchains provide tools that are familiar to the target audience.