January 17th, 2017
Starting from OS X El Capitan (10.11), there’s a new default system font in town – San Francisco. And it came with a very big underlying change, as detailed by Craig Hockenberry:
Apple has started abstracting the idea of a system font: it doesn’t have a publicly exposed name. They’ve also stated that any private names are subject to change. These private names all begin with a period: the Ultralight face for San Francisco is named “.SFNSDisplay-Ultralight”. To determine this name, you need to dig around in the font instances returned by NSFont or UIFont; it’s not easy by design.
The motivation for this abstraction is so the operating system can make better choices on which face to use at a given weight. Apple is also working on font features, such as selectable “6” and “9” glyphs or non-monospaced numbers. It’s my guess that they’d like to bring these features to the web, as well.
Even though the underlying .otf files are still in /System/Library/Fonts, San Francisco is no longer exposed via the regular APIs that web and desktop developers have grown used to. Specifically for Swing developers (of which there may not be many, so at some point it will kind of take care of itself), passing “San Francisco” to the Font constructor ends up using the previous default – Lucida Grande.
JavaFX is already doing the right thing, using San Francisco as the default UI font on El Capitan and Sierra. Swing’s legacy is to have each look-and-feel decide which font to use, and I was expecting the “System” look-and-feel which maps to Aqua to be using the right font family on the latest OS releases. That is not the case as I’m writing this entry, and Swing apps on both El Capitan and Sierra are still using Lucida Grande on both 8u112 and 9-ea.
Last week Phil Race pointed me to this issue that tracked syncing up the internal implementation details of glyph mapping between JavaFX and AWT. That issue has been fixed in early access builds of JDK 9, and is slated to be available in JDK 8 u152 scheduled for October 2017. At the present moment there is no public API to get either a name or a font instance itself that will be mapped to Lucida Grande on 10.10 and earlier, and to San Francisco on 10.11 and 10.12. The only available solution is quite brittle as it depends on the internal naming conventions exposed by the underlying OS:
- .Helvetica Neue DeskInterface on El Capitan (10.11)
- .SF NS Text on Sierra (10.12)
Note that you need a leading dot in both cases, and that this only works on early access builds of JDK 9 at the moment:
In this screenshot the second button is using new Font(“.SF NS Text”, Font.PLAIN, 24) while the rest are rendered with Lucida Grande. The most noticeable differences are in the curvy strokes of “e”, “g”, “5” and “9”, as well as the straight leg of “a”.
Ideally, there’d be an officially supported way to use the right font on OS X / macOS, either in a form on some kind of a static Font API or a synthetic font family that maps to the underlying system font on all supported platforms. Phil has filed a bug to track the progress on that front.
December 28th, 2016
About five years ago I wrote about vector format, and how it is not quite well suited to be used for application icons. Five years is a long time in our industry, and the things have changed quite dramatically since then. That post was written when pretty much all application icons looked like this:
Some of these rich visuals are still around. In fact, the last three icons are taken from the latest releases of Gemini, Kaleidoscope and Transmit desktop Mac apps. However, the tide of flat / minimalistic design that swept the mobile platforms in the last few years has not spared the desktop. Eli Schiff is probably the most vocal critic of the shift away from the richness of skeuomorphic designs, showing quite a few examples of this trend here and here.
Indeed it’s a rare thing to see multi-colored textured icons inside apps these days. On my Mac laptop the only two holdouts are Soulver and MarsEdit. The rest have moved on to much simpler monochromatic icons that are predominantly using simple shapes and a few line strokes. Here is Adobe Acrobat’s toolbar:
This is Slack:
Line icons are dominating the UI of the web developer tools of Firefox:
They also have found their way to the web sites. Here is Feedly:
and Wikipedia’s visual editor:
You can also see this style on Airbnb listing pages (with a somewhat misbalanced amenities list):
and, to a smaller extent, on other websites such as Apple’s:
Google’s Material design places heavy emphasis on simple, clean icons:
They can be easily tinted to conform to the brand guidelines of the specific app, and lend themselves quite easily to beautiful animations. And, needless to say, the vector format is a perfect match to encode the visuals of such icons.
Trends come and go, and I wouldn’t recommend making bold predictions on how things will look like in another five years. For now, vector format has defied the predictions I’ve laid out in 2011 (even though I’ve ended that post by giving myself a way out). And, at least for now, vector format is the cool new kid.
December 22nd, 2016
This might be a bit of a surprise, but after almost six years of self-imposed hiatus I’ve decided to come back to some of the open-source Swing projects that have been frozen in time since late 2010. Part of it was a mild curiosity to see how much things have changed in the meanwhile, and part of it was somewhat of a challenge to get back to a code base that I once knew like the back of my hand.
Before delving into the rest of this rather lengthy post, a fair warning. The images seen here are only meant to be viewed on high-resolution / retina / 2x-density screens. Otherwise what you’re seeing is a scaled down version of the original images that has little to do with the actual visuals seen on those screens.
Having said that, what is you see above is what I saw a couple of months ago on my 2013 MacBook Pro that has a Retina screen. Having spent the last four years looking at crisp visuals of pretty much all the native apps (except for Eclipse that is just now starting to get there, but then again calling Eclipse a native app is somewhat of a stretch), this was painful. There are obvious line artifacts, with the two most noticeable being the light outline along left/top edges of the main window and a darker horizontal line across the selected tab. But other that that, everything is just too blocky, pixelated and way too fat.
Two months later, I’m happy to report that Substance is a fully fledged citizen of the Retina universe. Given the amount of work that went into making that happen, the next release will have a version bump to 7.0 code-named Uruguay. The three big themes of this release will be as follows.
Hi DPI support
Searching the web for the initial pointers on Hi DPI support in Swing got me to this post from 2013 by Konstantin Bulenkov. Konstantin is the team lead at Jetbrains – the company behind the IntelliJ platform and all the apps built on top of that platform. Did you know that IntelliJ IDEA is a Swing app? If you didn’t, now you know.
Lucky for me, the code referenced in that post is part of the community edition of IDEA and is available under the quite permissive Apache license. And even though some part of that code is using reflection to query the underlying capabilities, if it’s working for IDEA with its hundreds of thousands of active users, it’s certainly good enough for me.
This code now powers pretty much all the pixels you see in Substance 7.0 – once again, the same warning about images that are meant to be viewed only on retina / high-res screens applies to all the images in this post.
Continue reading »
June 6th, 2013
In my previous lifetime I was a Swing developer. And I liked shiny things. As a proof, here’s the pinnacle (or so I thought, at least) of my explorations in making shiny glossy glitzy buttons. That was around April 2006.
Different UI toolkits provide different capabilities that allow you controlling visual and behavioral aspects. Putting the technical details of styling aside though, UI control styling usually works at the level of an individual control.
And so as I was working on my own look-and-feel library, I heard more and more tidbits about Vista. It was released in January 2007, but it had a long [really really long] history. People kept talking about the three “pillars”, and I was mainly interested in the Presentation one. I don’t have a link, and I can’t even tell if it was a feature that was eventually shelved or just a rumor. But when I heard it, it made a long-lasting impression on me.
The gist of it was that entire UI is a 3D model. You know how they say that buttons should look like something that can be pressed. So you have some kind of z-axis separation. Drop shadows, bevels, some kind of a gradient that hints at the convex surface. And don’t forget to throw in the global lighting model. And so that bit of pixel feature rumor said that the entire UI – from the window level down to an individual control – would be an actual 3D model, with each object living in its own z plane.
So instead of styling each control to create an illusion of z separation (with whatever 2D images are backing each individual control), you would have a spatial model. Each control has its own 3D geometry. Now all you need to do is place the controls in the 3D space, create a few global lights, create a bunch of textures to use on the controls and voila – ship it over to the GPU to compute the final pixels. Want to restyle the UI? Supply a different texture pack and a different lighting model. All the rest is taken care of by the system. Have your own custom control library? Define the 3D meshes for them. All the rest is taken care of by the system.
Now imagine what you can do. If you place two buttons side by side, with just the right tweaking of the meshes and just the right amount of reflection on the textures you can have a button reflecting parts of other buttons around it. And the other way around. You know, all those shiny reflection balls from the early ray tracing demos.
Or, if you model the mouse cursor as an object moving above the window, you can have the back of it reflecting in those controls that it’s passing over. If your control mesh has some kind of a curved contour, the cursor shape would get distorted accordingly as it glides off of the edges.
Or, as you press the button, the press distorts the button mesh as the exact spot of the press, and the entire geometry of the scene reflects that.
I had serious thoughts of doing that. In Swing. That never happened though. Here’s why.
In my mind, there were three big parts to actually doing something like that.
The first one was relatively simple. It would involve transitioning from the point of view of looking at a single control at any point in time towards creating a global view scene that had the entire view hierarchy. There were enough hooks in the API surface to track all the relevant changes to the UI, and even without that you can always say that applications must opt into this mode and have to call some kind of an API that you provide that there are “ready” for you to build that graph.
The second one was also relatively simple. I would need to generate the meshes for all controls. Some are simple (buttons, progress bars), some might be trickier (check marks, sliders). But nothing too challenging. Mostly busy work.
But the last one was the effective non-start. How to actually create the final render of the entire window with acceptable performance? Doing my own 3D engine was kind of out of question. I knew just enough of what is involved to not even begin down that path. So that left me with OpenGL.
JOGL was around at the time, and had a nice momentum behind it. They were gearing towards providing bindings for OpenGL 2.0. There was a lot of activity on the mailing lists. Java3D was another alternative that was under similarly active development. There was even a talk of merging the two. And so I started looking into a simple proof of concept of making a simple JOGL demo on my trusty Windows box.
Around that time (early 2007) Ben Galbraith announced the first (and, posthumously, the only) Desktop Matters conference in downtown San Jose. I left a comment on that announcement. He asked me whether I wanted to make a short presentation on one of my projects. I was quite happy to do so. That was my first public presentation [thanks for the encouragement, by the way!]
It was a nice gathering. Around 100 people, I’d say. And they had quite a few people from the desktop client team at Sun available for informal Q&A. Chris Campbell was my hero at the time (no offense, Chet). The dude was slinging code left and right, showing a lot of great things that could be done with Java2D. He was also working on hardware acceleration of a lot of those APIs. If I remember correctly, he was talking a lot about doing various acceleration on top of OpenGL and Direct3D. Who would be better to validate the overall approach of doing this thing that I wanted to do than him.
I managed to grab him for a few moments. I outlined my thinking. He was polite. He said that it sounded about right. That was just enough encouragement for me.
So after the conference was over I got to actual work. My first private demo was to render a colored sphere. And it looked horrible. It had jagged edges all around it. And it also had visible seams running all over the sphere. I could see the tessellation model before my eyes. It was quite bad.
So I fired off an email to the mailing list. Not about my grand vision. But rather about this specific thing. How to make a sphere look like a sphere. With no jaggies and no tessellation. And they told me to get a “real” graphics card, because whatever integrated graphics card I had on the motherboard is no good for any kind of OpenGL work. And that’s where I stopped.
What’s the point of even thinking going down that road if you must have an expensive graphics card? It might be OK for a demo. It might be OK if I’m satisfying my own itch and showing off my skills with some kind of a thing that runs well on my machine [TM]. But if it can’t be used on “everyday” computers that don’t have those fancy hardware components, it’s a no-go for me.
You might say that I chickened out. I had this grand vision, and folded at the first sign of trouble. But that was – and still remains – my main issue with anything that ends with “GL”. Its never “quite there” promise of commodity hardware availability that is “just around the corner” – and in the meantime, you need this very particular combination of hardware components, drivers and other related software to run. And oh, even if you do have a beefy graphics card, unfortunately it has this driver bug that crashes the entire thing, so you might want to either bug the vendor to fix it, or just disable the whole thing altogether.
Things might have been different. I had really a lot of spare time back then. I might have went down the road of biting the bullet and buying that graphics card (although, as mentioned above, it was not about my own cost, but rather about the reach of the final library). I might have had this thing done in some form or another. Can you imagine buttons reflecting other buttons reflecting the mouse cursor passing above them and rippling as you press them? With the ripple reflected all around that button, and being reflected back in it?
So that never happened. And now it’s all about flat. Flat this. Flat that. Flat *ALL* the things!