Native text rendering in Swing applications – how does it work?

December 5th, 2007

Two weeks ago i wrote about the first drop of Bramble plugin for Substance look-and-feel, which provides support for native text rendering in Swing applications. Today, it’s time to look at the actual implementation and see how it all works together.

Unlike SWT that passes the painting requests to the underlying native APIs, Swing handles all the painting itself (unless we’re talking about Windows and GTK look-and-feels under Mustang, which have their own set of problems). Swing’s greatest flexibility is in its “layered” approach to rendering the components (see this PDF presentation that shows one of the possible effects that are extremely easy to achieve with Swing UI delegates). Take, for example, painting a button. It’s all done in independent layers which are then composited together using the powerful Java2D APIs:

Background
Icon
Text
Focus
Border

What happens when you want to provide custom rendering for one of these layers (text in our case)? Just override the relevant paint method and provide the logic there (not touching any other pixels). Seems easy, but since SWT has its own painting model dictated by the OS, it’s not. The main difference between text rendering in AWT / Java2D and SWT is that the later expects the background pixels to be fully specified (as detailed in this discussion on the platform-swt-dev mailing list and this discussion on the eclipse.swt.forum).

And as it turns out, this complicates the logic of plugging the SWT text rendering into the Swing painting pipeline, but the end result is most certainly worth it. I’ve already posted a few screenshots, and here is one more:

The first two rows (labels and text fields) are rendered by Swing (Vista + Mustang), while the last two rows are rendered by SWT / native code. While for labels the difference might not be that big, for the digits it is simply staggering (at least for the default Vista Segoe UI font). So, what do we have to do in order to have the native text rendering in Swing?

The current implementation has three stages:

Filling the component bounds with the background color. The actual color depends on the component kind and component opacity (can come from the parent container).
Painting all the overlay / highlight layers. List cells have rollover / selection highlights. Text components have multiple highlights depending on the text selection mode.
Painting all the texts. There can be more than one with different fonts and colors. Menus have the menu text and accelerator text. Sliders have multiple labels. Text components have multiple selected and unselected sections.

The pure Swing implementation doesn’t have to be concerned with combining the results of these three stages together, since Java2D’s drawString method can operate on a translucent / transparent image (with questionable results in some cases, or otherwise there would be no need for this entry). The native text rendering expects all the background pixels to be filled before it can paint the text. Depending on the actual component, this can make the nice and structured Swing UI delegate implementation a little bit more coupled. This is especially true for such complex components as lists, trees, tables, combo boxes, sliders and all types of text components.

In addition to this limitation, there are technical issues with integrating Java2D and SWT rendering. What is the flow here?

Java2D code paints the component background and overlay / highlight layers on an AWT image
An SWT image is created from the pixels of the AWT image
AWT font and color settings are use to create the matching SWT text layout
SWT text layout is used to paint the text on the SWT image
Another AWT image is created from the pixels of the SWT image (which at this point contains the background, overlay / highlight layers and the texts)
This AWT image is painted back on the Java2D Graphics that is passed to the UI delegate

All these steps might add significant overhead to the overall painting performance, especially considering that text elements are pretty much everywhere in a common UI. The first drop that was available two weeks ago had some big performance issues on large lists and tables. This has been addressed by clipping the compositing area to only the necessary rectangle. In addition, there are minor differences between the two UI toolkits. For example, AWT uses pixels and SWT uses points. This means that the font size needs to be converted to maintain the visual results. In addition, most SWT resources need to be explicitly disposed (much like Java’s streams, for example, that need to be explicitly closed).

The components that are supported in the latest drop of Bramble plugin:

Buttons, toggle buttons, check boxes and radio buttons
All menu elements, including menus and menu items
Tabbed panes
Labels including the default cell renderers for trees, tables, lists and uneditable combo boxes
Text fields and formatted text fields with simple documents, including default editors for editable combo boxes
Decorated title panes for frames, dialogs, internal frames and desktop icons
Slider labels
Table headers
Tooltips
Components from the Flamingo component suite such as command buttons and ribbon

If you’re interested in testing this implementation, you need to do the following:

Download the substance-bramble.jar here and add it to the classpath of your application.
Add the org.eclipse.swt.win32 jar to the classpath of your application. Bramble distribution bundles this jar from Eclipse 3.2.1 in the ‘lib’ folder.
Extract the matching swt-win32.dll and add it to the java.library.path of your runtime configuration. Bramble distribution bundles this DLL from Eclipse 3.2.1 in the ‘lib’ folder.
Configure your application to run under the latest 4.2dev drop of Substance look and feel.

If you run into any issues with components mentioned above or custom cell renderers / editors, i would be more than interested hearing about it. The plugin is distributed under the Eclipse Public License (EPL).

Swing links of the week: December 2, 2007

December 2nd, 2007

Here are some Swing links that you might have missed during the last two weeks:

Continuing the theme of Swing-“powered” applications running on JVM but written in another language, Shawn Crowley addresses the viability of JRuby as a realistic candidate for authoring complex desktop applications. It’s great to see that his experience is largely positive and he has no regrets in choosing this platform.
Tim Dalton continues his series on building GUIs in Scala. This time, the “usual suspect” of UI demoes – a simple calculator.
And to top off this part, Andres Almiray revisits his library of Groovy builders as the Groovy 1.1 release approaches. The entry on GraphicsBuilder is talking about adding mouse and keyboard support, and the entry on JideBuilder makes sure that it is updated and functional. The last entry is on GraphicsPad, which begs just two questions – what is it, and where are the screenshots?
Jan Erik Paulsen continues his experiments with Photoshop Express clone, and is trying to come up with a new Substance-based skin. Dubbed “Titanium White”, it looks quite similar to the core Business skin from Substance with round button shaper.
Alex Potochkin has technical details on the Rainbow application that we used for our join JavaOne presentation last year. As you can see, this application is still alive (hint-hint), and is still used by both of us to improve the libraries that it is using (JXLayer for Alex and Flamingo for me).

And a few product announcements as well:

The JPen project is a library for accessing pen/digitizer tablets and pointing devices.
Alex Ruiz has announced the release 0.7.0 of the FEST-Swing UI testing library.
Dave Gilbert has announced the bugfix release 1.0.8 of JFreeChart charting library.
Wolfgang Zitzelsberger has announced the release 2.6.0 of Synthetica look-and-feel. If you thought your favorite library was customizable, take a look at the customization documentation – with ten new client properties, this release has broken the 300 mark :)

Native text rendering in Swing applications

November 21st, 2007

Yesterday i posted six screenshots divided in two groups and asked what is the difference. Most of the comments rightfully pointed out that the font rendering is different, and that the second group looks better in this area.

First, here is a screenshot that summarizes the difference in font rendering in those screenshots (the source code for this application is here, read the technical instructions below on how to run it):

The not-so-gentle hint in the frame title bears congratulations to Eugene Ryzhikov that said that the font rendering in the second group is “just like on Vista”. The first and third rows in the screenshot above are rendered by SWT (and hence by the OS), while the second and fourth rows are rendered byAWT / Java2D (applying the rendering hints as instructed by Chet Haase). The difference has been pointed out and discussed about eight months ago on this JavaLobby thread, but this specific screenshot doesn’t really do native rendering the justice it deserves.

First, note the stroke weights – SWT / native rendering is consistently heavier than in AWT / Java2D one. This puts less strain on the eyes in the long run. Second, pay close attention to the small letters ‘a’, ‘e’, ‘s’, pretty much all the digits and the symbol ‘@’. Java2D rendering is missing an entire pixel! And this specific screenshot is not even a good representation of a “real” UI, since it falsely skews the letter frequency and doesn’t show actual words.

Much can be said about statistics and how they can be bent both ways, but suffice it to say that, combined together, letters ‘a’, ‘e’ and ‘s’ account for more than 26% of an average English text. This means that on average, a Swing UI running on Vista and using the default Vista font (Segoe UI) has a lot of unnecessary inter-letter white space, which makes the texts less readable; the extra white space might cause the eye to perceive word boundaries where there are none. Combine that with lighter strokes and there – you have the reason why the WindowsLookAndFeel in Mustang under Vista still uses Tahoma for all the controls except menus. So much for the platform fidelity…

This issue has been on my mind ever since this comment by Karsten Lentzsch; i even had a chance to ask the Swing Java2D team about this during the Desktop Matters conference held on the next day of Karsten’s comment. Why not use SWT text rendering in Swing applications? Since the main strength of SWT is the native rendering (not only for texts), we will have the operating system rendering our strings, bringing the absolute fidelity in text rendering. And this is what is provided by the Bramble plugin for the latest 4.2dev drop of Substance look and feel (code-named Memphis).

Once you have the Bramble plugin in your classpath, along with the matching SWT jar and native library (see below for more technical details), Substance will automatically switch to native text rendering, turning this

into this

The actual implementation is a little bit more complicated than i originally thought. This is mainly due to the fact that in SWT you can’t paint anti-aliased text on a transparent background, like you can in Swing. See this discussion on the platform-swt-dev mailing list and this discussion on the eclipse.swt.forum for the reasons why. I’ll blog about this limitation and a few other differences between SWT and AWT text rendering later; in the meantime you can see the current implementation for more details.
Currently, the following components should have support for native text rendering:

JButton and JToggleButton
JRadioButton and JCheckBox
JLabel including the default cell renderers for JTree, JTable, JList and JComboBox
JTextField with simple view
JTabbedPane
Decorated title panes of JFrame and JDialog
JMenu and JMenuItem
JCheckBoxMenuItem and JRadioButtonMenuItem. Currently, the checkmarks are not shown (will be addressed)
Components from the Flamingo component suite such as JCommandButton, JToggleCommandButton and JRibbon.

The implementation requires JDK 5.0 to run, and has been successfully tested under Windows Vista (where it makes the biggest impact). In addition, it successfully runs under Windows 2003 and should also run under Windows XP. Since there are no Windows-specific classes used in the Bramble text painter, it should also run under other OSes, such as Mac OS X and Ubuntu (pretty much everywhere SWT is supported).

To give your application a ride, you need to do the following:

Download the substance-bramble.jar here and add it to the classpath of your application.
Add the org.eclipse.swt.win32 jar to the classpath of your application. Bramble distribution bundles this jar from Eclipse 3.2.1 in the ‘lib’ folder.
Extract the matching swt-win32.dll and add it to the java.library.path of your runtime configuration. Bramble distribution bundles this DLL from Eclipse 3.2.1 in the ‘lib’ folder.
Configure your application to run under the latest 4.2dev drop of Substance look and feel.

The next steps are:

Provide full support for all core Swing components, with possible exclusion of non-plain text components, such as HTML rendering
Provide full support for all SwingX components
Test and verify under Ubuntu 7.10
Extend the current Windows font policy to use Segoe UI font under JDK 5.0 running on Vista
Provide support for WebStart environment
Provide support for the NetBeans module

The native text rendering coupled with the automatic platform-specific font policy lookup should help Substance-powered Swing applications turn into the first-class citizens on the Vista desktop. Many thanks go to Karsten Lentzsch for his pioneering work in this field.

You’re more than welcome to leave comments with your thoughts, suggestions and bug reports (first consult the current list of supported components above). The plugin is distributed under the Eclipse Public License (EPL). Hope you like it, happy Thanksgiving and see you on the other side of our trip to Hawaii :)