The latest addition in the Flamingo component suite is support for key tips. This has been one of the items on the roadmap for version 4.0 (code named Fainnear), and is now available in the latest 4.0dev drop of Flamingo core and 5.1dev drop of Substance Flamingo plugin.
Key tips is one of the four supported ways to operate the Office 2007 ribbon with keyboard. Unlike the traditional menus-and-toolbars system, every command and control in ribbon can have associated key tip. This allows efficient access and activation of application menu items, task toggle buttons, task bar buttons, task expand buttons and other ribbon sub-components. As specified in the keyboard documentation, the key tip mode is initiated with either Alt or F10 key. Subsequent interaction will be illustrated in the screenshots below taken under the latest 4.0dev drop of Flamingo core and Synthetica Mauve Metallic skin.
Key tips are shown in groups (internally called chains). A key tip group is associated with the specific ribbon entity. When the key tip mode is initiated, the key tip group for the entire ribbon is displayed. It contains key tips for the application menu button, taskbar components and task toggle buttons:
Pressing the letter for taksbar button activates that button. Key tips for disabled buttons are painted with partial translucency. Pressing the letter for task toggle button selects the matching ribbon task. Pressing the letter for the application menu button shows the application menu:
Once the next key tip group is shown, the previously shown key tip group is hidden (no key tips on the application menu button and taskbar buttons in this screenshot). Pressing Esc dismisses the currently shown key tip group and shows the previous key tip group. When the initial key tip group is dismissed, the key tip mode is exited.
This screenshot also shows that the menu buttons can have two key tips associated with them – one for the action, and one for the popup. Pressing ‘D‘ (associated with the popup “Send” button) shows the second level menu, switching off the key tips for the first level and showing them for the second level:
Pressing keys ‘E‘, ‘H‘ or ‘W‘ will activate the relevant second level menu buttons. Pressing key ‘X‘ activates the popup menu associated with the “Wireless” menu button:
Going back to the first screenshot (that shows the top level key tip group). Pressing key ‘P‘ shows the key tip group for the first (and currently selected) ribbon task:
The key tips are shown for ribbon buttons, task expand buttons (look for ‘FO‘ and ‘FN‘), ribbon gallery expand buttons (look for ‘L‘) and buttons in button strips (in the “Font” ribbon band). As you can see, some key tips have two letters. This can be useful not only when there are not enough letters for the contents of the current task. It can also help in grouping key tips of related functionality (such as alignment and indentation command buttons in the “Font” ribbon band that start with A). In the present situation, when letter ‘F‘ is pressed, only key tips that start with that letter are shown:
As before, pressing Esc will show the previous key tip group – in this case, it would be key tips for the entire “Page Layout” ribbon task.
Key tips are also shown in popup menus. Pressing ‘V‘ when the key tips for the entire first ribbon task are shown activates the popup menu of the “Paste” button:
Pressing one of the relevant keys will activate the matching menu button, dismissing the key tip mode as well. Key tips are also shown for the menu buttons in rich popup panels:
Here, in order to activate the “Save Selection”, you would have to press the following sequence of keys:
- Alt or F10 to activate the key tip mode
- P to select the first ribbon task
- F, P to show the popup of the “Format” button in the “Clipboard” ribbon task
- S, S to activate the “Save Selection” menu button
As you can see, the sequence to activate a specific command can be quite long. As such, commands that are used often need to be identified and associated with shorter (and perhaps more memorable) sequences.
The APIs to associate key tips with different ribbon components are:
- AbstractCommandButton.setActionKeyTip
- JCommandButton.setPopupKeyTip
- JRibbonBand.setExpandKeyTip
- JRibbonBand.setRibbonGalleryExpandKeyTip
- RibbonApplicationMenuEntry.setActionKeyTip
- RibbonApplicationMenuEntry.setPopupKeyTip
- JRibbon.setApplicationMenuKeyTip
- RibbonTask.setKeyTip
If you want to see the scrolling in action under the core look-and-feels, run the following WebStart demo:
If you want to see this functionality under Substance, run the following WebStart demo:
If you want to test the new functionality in your applications, you would need the following (the last two only for applications running under Substance look-and-feel):
Your feedback is, as always, greatly appreciated.
In this interview Richard Bair answers my questions on Swing, SwingX, JavaFX and Java on the desktop in general, saving some of the announcements for the next week’s Devoxx conference. We wanted to do this interview immediately after my thoughts on the state of core Swing, but the JavaFX release has postponed it up until now.
Tell us a little bit about yourself.
Like many developers I know, I started programming because I wanted to write games. Somewhere along the line my dad convinced me to write him a database backed application for his business, and that’s how I got started.
Since then I’ve been predominantly an application developer. It was in that role that I started working with open source software, and eventually started working with JDNC. I was subsequently hired by Sun and have been the SwingLabs lead since that time.
It hasn’t been all smooth, I learned a lot about open source community building as I went along, and certainly have plenty of faults in getting SwingX releases out regularly. But my overall goal and desire has always been to make building real world applications easier.
I know in the recent past I’ve been involved a lot in Java demos and such that are consumer in nature. I like building applications, either consumer or enterprise. Consumer applications are closer to what got me started in programming in the first place, while enterprise apps are really where I cut my teeth (one of a handful of engineers I’m sure who love both big iron databases and pushing pixels ;-)).
Why was SwingLabs created, how did it evolve and what is its current state?
SwingLabs was a bold attempt by the Java Client Group to engage the open source community long before OpenJDK came along. Originally the brainchild of Amy Fowler and Hans Muller (and probably others on the team), the idea was to have an open source venue for building APIs that fill in the missing pieces of what is shipped with Swing.
As any Swing developer can attest, you have to build a lot of plumbing to build real apps with Swing today as shipped with the JDK. In the last few years a great many open source projects have been created to help fill these gaps.
SwingLabs was also designed as a place where these APIs and experiments could be vetted and included in the JDK. Our score in this regard is not so good. We pulled in a lot from the JDIC project, and we pulled some of the filtering and sorting work in.
We will be pulling more for JDK 7 as well, expect to see 2 or 3 components at least and also one or two big JSRs.
SwingLabs is currently run predominantly by the community which is really working I think pretty well. The dedication of our team of open source developers is a credit to them and to the open source reality that we live in.
We do need more involvement from the Sun side, and will be making an announcement in this direction at Devoxx next week.
If you were to start over with SwingLabs / SwingX, what would be the most important parts in building and maintaining the community involvement?
Regular releases. Regular content added to the SwingLabs.org website. More responsive bug fixes (my fault entirely).
Would you like to provide your side of the story behind the decision on the layers and painters in SwingX?
The fundamental idea behind painters was to decouple the rendering of components from the components themselves, such that you wouldn’t have to subclass the component to change its rendering (or replace its LAF which is a much more complicated proposition). The corollary was that there could be a collection of default painters that people could reuse.
There was an argument made primarily by Mikael Grev who had a similar API in his MIG Calender component (if I remember correctly) that we should have components that support painters to have an arbitrary sequence of painters. Sort of like JLayeredPane but for painters. I know Scott, Romain, Josh, and I didn’t particularly like it for technical reasons (which are all documented in the forums), and there was both support and skepticism from other people outside Sun.
At the end of the day, the project lead has to make a call on API, and I made the call I felt most comfortable with.
How likely is it to see the SwingX components being part of core Swing in JDK 7 and beyond?
All of them? Unlikely. Some of them? Extremely likely.
Originally Nimbus was planned to be developed under the SwingLabs roof, but was later put into the internal repository. Would you like to see the development of new Swing-specific functionality in the open before it gets put into the core distribution?
Nimbus is a special case. To properly implement Nimbus required a significant number of fixes and enhancements to BasicLookAndFeel, Synth (lots of bug fixes for Synth) and other core APIs and implementations. We simply could not have delivered Nimbus in the open source without these fixes. Also, Nimbus went into Java 6 Update 10 which was not part of OpenJDK. When Nimbus is forward ported to Java 7, it will be in OpenJDK.
I would prefer that all development was done as part of a community process, be it SwingLabs or an open JSR or OpenJDK.
Do you think that maintaining backwards compatibility has unnecessarily restricted the Swing team in moving this UI toolkit forward?
No I don’t. If you gave up backwards compatibility you’d have a whole other set of issues that are many times worse. The level of risk one developer is willing to take isn’t the same as another. For every one person who wishes we’d ditch backwards compatibility there are another 50 that would be violently opposed when that advice rendered their applications (or development of future versions of their applications) inoperable. Especially with a technology as mature as Swing.
To those people who say we should make a Swing2 which is not backwards compatible, I would say, this is exactly what we’re doing with JavaFX.
I also want to emphasize that Swing isn’t going away. We have a tremendous number of customers building Swing apps and that will obviously be supported and extended going forward. Also, we will allow you to mix JavaFX both into Swing apps and mix Swing in JavaFX.
Both Microsoft and Adobe view “line of business” applications as a very important target segment for their offerings (WPF, Silverlight, AIR, Flex). Is JavaFX only targeting the rich UI development (graphics, effects, animations), or do you plan to include modern business components such as pivot tables, grid trees and date pickers?
JavaFX 1.0 is oriented toward creative content, very similar to Silverlight 1.0. Future versions of JavaFX will be designed to also handle full blown enterprise rich client application development.
What effort is planned to be put into the core Swing in 2009?
We’ll have announcements for these plans at Devoxx.
Anything else you would like to add?
As an application developer, and as a toolkit developer, JavaFX is very exciting for me. The first release is really aimed at providing the foundation for all subsequent work. We’re focusing on animation, graphics, media, and related APIs with an eye towards the future so that we have a solid foundation upon which to add user interface Controls, frameworks etc. JavaFX gives us the freedom to design the best possible design we can, without any API backwards compatibility constraints. It really is an exciting opportunity. We’re committed to providing interop between FX and Swing so that people can build the kinds of applications they want to, and can mix and match the two.
There’s nothing like the right link at the right time. Remy Rakic (Twitter alias lqd) has left a link to the apparent release jars of JavaFX Desktop 1.0 in the comments on the previous entry. So i’ve decided to see whether there is anything new in the JMC department since the last time i’ve looked at it.
In the JavaFX preview, the native code was shipped directly in the jmc.dll. The download folder has the native libraries available in three platform-specific jar files (one for Windows, Mac OS X and Linux each). For Windows, the jar contains four DLLs:
- jmc.dll as before
- on2_decoder.dll which would be presumably responsible for the new licensed decoders
- msvcp71.dll and msvcr71.dll which are parts of Visual C++ 7.1
Assuming that this download folder is going to be used as a centralized WebStart repository, i would venture to say that there is going to be some kind of built-in mechanism for extracting these DLLs at runtime and updating the java library path. However, in my experiments i had to extract them manually and point the -Djava.library.path to the folder that contains those DLLs. We’ll wait and see whether the final (and official) release will follow in the footsteps of SWT 3.3 that handles the locally run extraction automatically.
Apart from the new JMediaView class that is a lightweight Swing component I haven’t seen any new classes. However, this JMC drop can play all QuickTime content from the sample page, including H.264, MPEG-4, 3GPP and MPEG-2, as well as the full scale trailers from the Apple site (in H.264 format). The playback support goes a long way towards my number one wish for desktop Java in 2008, and it will be hopefully followed by the recording capabilities as well.
I’ve decided to dive a little bit deeper into the JMC internals and updated the JMCFrame class to show a skeleton implementation of a “home-grown” player. Instead of using ready-made classes such as JMediaPlayer, JMediaPane or JMediaView, i’m creating a MediaProvider from the specific URI and attaching a VideoRendererListener to its VideoRenderControl (i would guess that the JMedia* classes follow the same route, but without the source code it’s hard to tell). This certainly makes the code a little bit more complicated, but allows much greater flexibility as far as compositing and overlaying go. Here is a screenshot of the player in action:
What do you see in the screenshot? There is a JPanel with a JButton control in it. The JPanel overrides the paintComponent and paints the following three layers:
- Gradient checkered background.
- The contents of the current video frame with 80% opacity.
- Information on the current media location, total duration and rate.
Using a MediaProvider with a VideoRendererListener allows some interesting things – overlaying translucent controls and HUDs, cross fading between multiple video streams and rendering video streams on arbitrary Swing components.
The new implementation supports dropping two kinds of video content:
If you run the sample code locally, drag a local video file on the panel and press the play button. The content should start playing. You can also drag a URL from your favourite browser. If that URL points to a video stream, the stream will be played – tested on streaming H.264 movie trailers from the Apple site :)
If i had to choose a particularly nifty API, that would be the MediaControl in the com.sun.media.jmc.control package. To get the specific control, call MediaProvider.getControl passing the class instance of the required control (such as VideoRenderControl). The MediaProvider.getControls API will enumerate the supported controls. Here is what i see on my machine when playing video files: play, audio, video, subtitle, video render and track. Hopefully we are going to see the record control joining this group soon.
