Continuing the ongoing series of interviews on screen graphics and user interfaces in movie and television productions, it is my pleasure to welcome Robyn Haddow. While most of the recent interviews have focused on the work that goes into creating fantasy user interfaces for major sci-fi motion pictures, in this interview Robyn invites us into the frenetic world of episodic television. In the last few years she has worked on “The Flash” and “Arrow”, defining and evolving the world of screen graphics that follow characters through various major and minor sets across seasons that span 23 episodes each.

In this interview Robyn talks about the proliferation of screens around us and how that propagates into the make-believe worlds of movies and TV shows, the flow of work in episodic TV from fleshing out the initial brand to working on individual episode-specific sets with different directors and cinematographers, keeping up with interface trends in the realm of real-life software, her thoughts on the current generation of software tools she works with and how they can evolve, and the growing interest in human-computer interactions in augmented and virtual reality.

---

Kirill: Please tell us about yourself and your path so far.

Robyn: I am a fantasy user interface designer and motion graphics artist for playback in production and post production in Vancouver, BC. I work at Scarab Digital, which is a content creation house, as well as freelance doing screen graphics and gadgetry for film, video game trailers and in game cinematics. I love imagination, collaboration and storytelling. I completed my University degree with a double-major in film and theater. After that I received a scholarship to go to the Vancouver Film School where I completed a diploma in the digital design program. There I gravitated towards motion graphics and stumbled upon the work of Mark Coleran. I was so intrigued and fascinated by some of the work Coleran had created for films such as “The Island”, “Mr. and Mrs. Smith”, and “Lara Croft: Tomb Raider.” I loved the detail and technical aesthetic. In doing more research in the field, I discovered work that Jayse Hansen had done in “Rise of the Planet of the Apes” and work that Corey Bramall was doing on features that was very inspiring to me. I was curious about screen graphics and wanted to pursue fantasy user interface design. I really love technology, and understanding how things work. I feel that UI design is a good arena for me because it is a vessel for both creativity and logic.

Kirill: It would also appear to me that the proliferation of screens around us in our daily work and personal lives has to be reflected in those movie and TV productions set in either present days or some variant of future. You have to have those screens that support the story and extend the “cyber” abilities of the characters.

Robyn: Exactly. Screens can also serve as a conduit for telling a story in a way that you can’t do otherwise. Sometimes showing a screen with a high-tech system serves as an outlet to provide an explanation of what happens, it’s a convenient way to get a story point across.

Kirill: And as we experience assistive screen based systems on our phones, tablets, cars and even in our homes, viewers come to expect to have those capabilities taken advantage of in the movie and TV productions.

Robyn: I think it also helps support the environments and the worlds set in films. It is an extension of believability of the story. If it’s set in the future, set in some sci-fi universe, as an audience you want to buy into their reality more. The technology in that world only helps to enforce that.

Screen graphics for Palmer Technologies in Arrow. Courtesy of Robyn Haddow.

Kirill: When did you switch to doing more work on screen graphics?

Robyn: Danny Ho, the Creative Director of Scarab Digital, reached out to me. He saw some of the work that I was doing, and thought I might be a good fit. He hired me to develop a lot of the screen graphics for the pilot of “The Flash” and I continued on with the success of that and the other shows at the time, season two of “Arrow” and “Tomorrow People.”

Kirill: What do you work off of when you work on a TV show? Do you get some kind of an initial brief that outlines the universe, and how much freedom you get to explore that universe within a certain set of story constraints?

Robyn: I get briefed from the concept and playback meetings where specifics from the production designer and director get flushed out, I read the script, go over any concept drawings and have a creative session with our team about how best to telegraph the script and make sure we are supporting the story the best way possible. We have earned a lot of trust and creative freedom of the course of the shows, and I feel very lucky for that. Our team is very collaborative and we work very closely on everything. Danny works very hard to maintain a positive energy and supportive environment in our team. It is so important as production moves so quickly.

Screen graphics for Palmer Technologies in Arrow. Courtesy of Robyn Haddow.

Kirill: What’s interesting in this for me is that most of the interviews on screen graphics I did so far were with people doing them for movie productions. There you have a known, constant set of people you talk to – producers or director or production designer or visual effects supervisor. And on a TV show you have the production designer and the show creators, but then each two-episode block is done by a different team of a director and a cinematographer. How do you work in this much more dynamic work setting where each pair has their own take on how to tell the specific part of the story?

Robyn: Well, it has been my experience that in the pilot you sort of brand everything. The look of the main set screens in particular are established at the beginning, and it is over the course of the season where things evolve and get built upon. I feel the influence of those involved is injected in each episode through knowing their taste and how they like to shoot things. Of course, there are new sets being introduced frequently in episodes, and again, influences change. Its like an evolving creative energetic force of those involved per episode.

Kirill: We have these big software tools we use in our daily work and personal lives – Adobe Creative Suite for you, some programming tools for me, and also browsers, office tools etc. They all evolve over time to meet new users’ requirements or, in some cases, anticipate and shape new requirements that haven’t even been “articulated” by the intended user base. Is that also expected in the world of these fantasy tools, to have them evolve between seasons or even throughout the storyline of a single season? What kind of a discussion you have in your team to convey such an evolution?

Robyn: Good question. I’d say, yes. It’s about being tuned in to design trends and real world usability as well as going beyond that into a more imaginative sphere. It’s twofold. You want to create pieces that look future forward and fresh, yet are also familiar enough to people that they can relate and will believe in the world you are creating graphics to support. I use the Creative Suite and Cinema 4d and love the software. My colleagues and I have an ongoing discussion about what’s going on with the software and how we use it. We translate some of the improvements we notice and appreciate in the software that we use into some of the graphics that we’re creating. For example, notifications and how they appear / affect the screen for such instances as a virus detection / hackers, or a completed simulation. I think it’s important to be reflective on the software that you are surrounding yourself with and inject some of those nuances into our imaginative worlds.

Screen graphics for Palmer Technologies in Arrow. Courtesy of Robyn Haddow.

Kirill: Because you don’t want your next 2016 season look like Windows 98.

Robyn: No, not at all. It’s funny going back, for example in flashback sequences, that’s exactly where we can do – let’s get reflections and heavy drop shadows. Let’s not use any flat colors. People are familiar with visual cues which become a signifier of time and era.

Kirill: I’d say that it has become so accelerated over the last few years. It used to be that major releases of leading operating systems were literally years apart. And now companies that are still in the OS business are almost expected to do yearly upgrades to their offerings, if not revolutionary then at least markedly evolutionary. It’s becoming very easy to spot the exact year that the specific app design was created in.

Robyn: Yes, for sure. You have to keep up with the evolution of usability and the appearance of it, otherwise people aren’t interested.

Kirill: So that’s on the software side. What about the hardware side. Do you keep track with what’s happening with the physical manifestation of human-computer interaction, from hiding the screen like in the movie “Her” to elevating assistive technologies into being almost the “driver” in that interaction, with Jarvis being the prime example?

Robyn: I gravitate naturally towards learning about innovative developments and what people are doing on that front. I love looking at new technology and learning about what it does and how you can interact with things. I was looking at Project Soli recently that uses radar for touchless interactions – like, seriously?! I love learning about how things work, and the strong direction in which it’s becoming a more interactive world.

I took a golf lesson awhile back, and my instructor brought out his iPad and was looking at the angles of my swing and the trajectory of the ball. It’s so interesting to see how people want to bring technology into their world, and the direction its moving. People want technology immersed in our environments. The video I’ve sent you about Oculus Medium for example, taking virtual reality and immersing yourself into the world of sculpting and painting. It’s fascinating.

Atom HUD sequence from Arrow. Courtesy of Robyn Haddow.

Kirill: You mention HoloLens on your portfolio site and I’d like to talk to you about that a bit later. I want to bring you back to what you said earlier about working in parallel on two productions. So the second one is “Arrow”?

Robyn: Yes, that’s right.

Kirill: How does that work for you – does the work overlap and where do you find the time?

Robyn: It can be tricky… the work does overlap according to shooting schedules and we work hard to build ahead as much as we can so we are prepared for scheduling changes etc. It’s tough. There are a lot of long days.

Kirill: How different is it for you to work on a production that had the universe established? I’m not a designer myself, but I’m very fond of the remark that I’ve read a few times, that a true designer’s mark is in working within a tightly built set of constraints and use those to his or her advantage.

Robyn: It’s like a brand. You learn a brand, and you become familiar with the characters. Take, for example, one of the sets. You learn how the character operates within that OS, talking in relatable terms. Or when a character is searching for something, this is what happens, and that’s what’s already been established. You want to maintain that, yet keep things fresh. You can slowly begin to take more liberties once the foundation of things have been established.

Atom HUD sequence from Arrow. Courtesy of Robyn Haddow.

Kirill: How does it work for you schedule wise? Are you 365 days a year on some part of these two shows?

Robyn: We have down time in the spring which is nice, that’s when I jump into work with other post houses or get involved with video game trailers.

There’s an upcoming spinoff of the two shows that has been announced – Legends of Tomorrow. So that’s three shows that all come from the DC universe.

Kirill: You mentioned that some of the elements you’re designing and building are interactive. So the script that you get – or perhaps notes – provides information on how is going to be interacting with those screens and what they’d be doing, so that you know how the screens react to those taps, touches, flings etc.

Robyn: That becomes apparent during the meetings and conversations. You tease out which builds are intended to be interactive, and on top of that, our team sometimes imagines something that can become a cool moment and incorporates key strokes for that even if it isn’t written in the script as a 100% interactive scene, it can be kind of cool if we give them actions, if we give them moments. We try to draw out the beats so that it gives the actor some business should they want the option. If the director doesn’t want it, then we can opt out and it’s fine – which I think is the beauty of playback. You’re live on set, you’re in production, and you can provide more creative freedom.

Atom HUD from Arrow. Courtesy of Robyn Haddow.

Kirill: Your many work hours become “compressed” to a few minutes of screen graphics in a single episode. I’d imagine that it’s a very rare thing where the screen is the main focus of the scene for any significant amount of time. Do you wish sometimes that certain elements would get more screen “love”, or is it just, if I may weave your film education background, that the work that you do is just one storytelling tool at the director’s disposal?

Robyn: I think most artists would say that it’s a little bit disappointing when that’s the case. But unless it’s an established hero moment, it’s almost expected. It’s just dressing. It’s just graphical dressing on the set for the environment.

Kirill: Maybe 20 years ago computer screens were still a novelty and could be their own characters. But nowadays everybody has screens in their lives, so those can’t be a pillar of a story.

Robyn: Not at all. Regardless, I pour a lot of effort and energy into the screen graphics that I do create, and I also inject things for selfish reasons. Sometimes I want to exercise a new technique or try something new that I’ve been thinking about. For me, It’s definitely about the process and what I learn along the way. You want something to look awesome, yes, and it has to always support the story, but maybe I wanted to learn this new toolset or technique as well, so I created this gacky nerdy little thing in a composition just for me.

Traffic cameras sequence from The Flash. Courtesy of Robyn Haddow.

Kirill: On the record, is there a chance that I pause one of the episodes that you’ve worked on and I’ll see your initials or your full name on one of the screens, hidden as a message to your fans?

Robyn: [Laughs]. I’ve definitely squeaked in my initials.

Kirill: Do you get to enjoy watching the shows that you’ve been involved with after working all those long days and reading through a whole bunch of episode scripts? Does it lessen the enjoyment? I’m in the middle of rewatching the first season of Mr Robot and I find these little pieces that click so much better together even if I know the overall plot. Is there any impact on you as a viewer of these shows?

Robyn: I would say that there’s definitely an element of surprise. By the time I get to watch the shows, so much has happened that I don’t remember certain things. It’s familiar of course because I know the story so well, but at the same time everything moves so fast. It’s still a pleasure to watch.

Individual screen elements and hacking sequence from The Flash. Courtesy of Robyn Haddow.

Kirill: Maybe it belongs more in the realm of visual effects, but my impression is that the software tools are now at the point where you can do practically anything that you’re asked to given enough time and money. Would you agree with such an assessment?

Robyn: I would say, largely, to a certain extent. The key part is time. You’re racing against the clock before you even show up.

Kirill: So you come back to the director or the production designer and tell them what you can realistically fit into the short production schedule?

Robyn: I don’t think that conversation actually happens. You basically come up with a solution. You just get more creative. You might get asked to create a certain thing, and your first thought is to create it in 3D and fly through it from a certain perspective, and then you do a test frame and it takes 2 minutes- we’ll you just don’t have that sort of time. So you ask yourself how can you approach the situation in a different way, perhaps it’s more graphically. Let’s eliminate the whole 3D aspect of what we were going to do, and we’re going to tell the story with way points and trackers. Let’s render a still from 3D and maybe use it to tell one aspect of the story, but get creative in doing other things. It’s not saying “no,” it’s getting creative in coming up with a workable solution.

Cortex medical sequence from The Flash. Courtesy of Robyn Haddow.

Kirill: Are you worried about advances in 4K display where you’d have to be even more precise and cut fewer corners. Everybody has high-definition TV screens and BluRay players, and Netflix is already doing 4K shows, and some of the hardware manufacturers are already working on 8K. That must keep you awake.

Robyn: For sure. Everything just grows exponentially. Higher resolution increases your render times. Everything goes up together, and now you need a bigger graphics card, more horse power, and a mini render farm etc. From my point of view, it’s coming up with solutions that you can make work. Like I said earlier, you come up with a solution with the tools you have. That’s part of being a motion graphics artist. It’s thinking on your feet, quickly and all the time.

Kirill: And old solutions never apply to new problems. You have newer versions of the tools you’re working with, new sets to dress, more rows of pixels to fill.

Robyn: Right, but I think your experience applies. You can rationalize that in older systems this thing sort of worked, so maybe if we try doing this, it might work. I definitely think that it’s a person’s whole oeuvre of experience that has value and builds up your problem-solving toolkit.

Kirill: So what you’re saying is that, for now at least, there is no computer powerful enough to replace you and take over what you do.

Robyn: If there was, I would shell out a lot of money to have such a system beside me.

Screen graphics for Palmer Technologies in Arrow. Courtesy of Robyn Haddow.

Kirill: On a more generic level and not specific to your productions, although it does appear in some of the screencaps on your site. What are your thoughts on the gigantic flashing red “Access Denied” box? How do you reconcile the way such warnings are displayed in real-life systems and the need to support the specific story point in a quick and visually consumable way?

Robyn: You have an instant. You have a millisecond for somebody to look at it and immediately get it. You rely on conventions that have been established very heavily. Colour for example, red is bad, green is good etc. I tend not to over think these sorts of things.

Suit failure in Flash. Courtesy of
Robyn Haddow.

Kirill: Because the alternative would be to make it much less prominent on the screen and have a character explicitly those words into the camera.

Robyn: Exactly, and why would somebody voice that? It’s cheesey.

Kirill: You meet a new person at a party. What’s your 10-second description of what you do for living?

Robyn: Hmm. I create screen graphics for film and tv that are played back on set and appear as if the actors are actually interacting with them. In reality, they aren’t. We pre-program the graphics to be operated by someone on set behind the camera in order for the actor to focus on delivering the scene. You know, Ironman style.

Kirill: Do you think that people understand the need for those screens to be designed explicitly? People are surrounded by screens everywhere in their lives and may take some of that technology for granted. And yet, when I sit down to watch a movie or a TV show episode, I don’t necessarily want to see the same exact things I see around me every day. So on your side you, perhaps, would want to design something that doesn’t exist yet, to push the envelope forward.

Robyn: Yes, I prefer to work on designs of things that don’t exist yet. It is much more interesting to me and the process is more rewarding I find. I really love the element of creativity that exists in this realm as opposed to building something thats supposed to mimic something that has already been done.

Kirill: Let’s talk about the term FUI itself, which started as Fantasy UIs and is sometimes referred to as Fictitious UIs or Faux UIs. Do you think that it captures what you do?

Robyn: I think it’s very fitting. I say Fantasy User Interface because it is fantasy. We’re making it up. It doesn’t actually work. It just has to look like it, look functional. And at the end of the day, that’s where the creative fun is. And for the user interface part of the equation, that is exactly what you’re designing, and how the character is to interact with it.

Kirill: This brings me into the last couple of areas that I wanted to explore. Earlier you mentioned that you love Adobe tools and then almost immediately said that they have their own set of problems. You create these distilled idealized fantasy user interactions, and you create them using real-life software that can sometimes be quite far from ideal as far as how easy it is to do certain things. Are you happy with the way things are at the present in the software you’re using? How would you like them to evolve?

Robyn: Hmm, it’s interesting. In a dream world I would make a gigantic superstar app where you have all of the go-to features of vector points from Illustrator, bitmap manipulation from Photoshop, animation curves from AfterEffects, 3D creation from Cinema 4D and programmatic capabilities from Director and Flash. To have them all in one mega-app [laughs] that takes the best of the best and is local to one world where you’re dealing with one coordinate system, one universal language, one set of key commands- that would be awesome.

Singularity simulation from The Flash. Courtesy of Robyn Haddow.

Kirill: What about the complexity of such an app? How complex its UI is going to be to expose all of those capabilities to a varied set of users?

Robyn: It’s a challenge, a very big challenge. It would be very interesting. I use those programs a lot every single day and I will say that there’s still a huge side of all of them that I don’t know about.

Kirill: There’s this saying that everybody is using 20% of Microsoft Office features, but everybody is using a different 20% subset.

Robyn: I believe that. You go in there and get out of it what you need. I am not going to go into Cinema 4D and understand the whole character-rigging architecture. I don’t need to articulate limbs every single day. I just need a pose. On that, I think that you can integrate different modes and create your own user profile in that mega-app. It’s about how you want it to work for you and about increasing the level of customization.

Talking about usability and where it’s headed with a lot of the touch ideas that are already out there, I don’t see much practicality to these yet in the everyday work environment. When I’m working in Illustrator with a bunch of layers, I don’t want to touch the screen. I can’t do that for 10 hours a day.

Brain scans from The Flash. Courtesy of Robyn Haddow.

Kirill: We would be in a much better shape though.

Robyn: We would, I agree with that [laughs]. I do like that idea.

I can see for example, when you’re in a concept meeting or a pitch that you’re going to want to throw information from a tablet up onto a big screen or table so that everyone can see it, interact with it, brainstorm etc adding a tactile nature to the meeting. But for now in everyday work, I don’t think it’s quite there yet. I’m excited to see what further developments will come.

Kirill: And outside of your “work” interactions with computers we have the “personal” ones, from browsing the web to looking at photos to texting and video chatting to watching movies. All of these feel, at least to me, very disjointed right now. They are very different ways for me to immerse myself into that digital world.

Robyn: I still find that I want to have a more immersive mode as well. We were talking about HoloLens for example. You still want to bring yourself into a virtual world and have it encroach into your physical environment. HoloLens had an interesting prototype for working in apps.

Wormhole warning from The Flash. Courtesy of Robyn Haddow.

Kirill: It was kind of a blend of physical environment with the virtual environment superimposed on top of it. It was a pretty impressive demo where they started moving “virtual” doors and walls that were displayed on top of a barely-started construction set.

Robyn: It was amazing. There still has to be a lot of studies and trial-and-error conducted about how people interact and integrate this kind of technology. I read about a study in Beijing maybe? where they had holograms projected on sidewalks because so many people walk looking down at their phones. You don’t look up, you bump into people or obstructions. They had all these arrows projected on the floor for where people’s eye lines are. It improved flow and way-finding.

Kirill: Sounds like what they had on the floors of the Battle School in Ender’s Game.

Robyn: It also reminds me of a concept video that Microsoft came up with on sustainability a few years back. It portrayed this world with information UI everywhere. Really cool.

Kirill: On the topic of HoloLens, there’s a lot of money being invested by quite a few major tech players into the world of virtual reality, literally billions of dollars. It feels like this is the big thing for the next 10-15 years, and I’m not sure that this is the right path for us as a social species, the path where you put some kind of a box on your face and you detach from the real world around you.

Brain scans from The Flash. Courtesy of Robyn Haddow.

Robyn: It’s a bit sad when you think about that narrow field of view physically. It’s depressing in a way. I think eventually with all these concept videos that we’ve mentioned they want to break out and away from those headsets. It’s about being able to access huge amount of information and being able to localize it into one area, to customize that information for you in a localized spot so that you can go through it quickly and get what you need.

You talked about the movie “Her” and the technology there is not even seen anymore. It’s just in the world. Ideally, the direction moves more in this vein.

Kirill: There are these extremes. One extreme is that screens are everywhere and every surface can be an interaction. Another extreme that you wrap your head in a box and that’s your world. And another extreme is that technology is not there at all because it’s so miniaturized to become effectively invisible. You can discuss what is the most viable option that is being explored today, or you can just “jump” forward 30 years and imagine something that simply doesn’t exist today. So our kids will be looking back at where the tech giants of today were spending their billions and simply laugh at how absurd the current explorations are.

Robyn: You can even look at the evolution of the last 10 years in cellphones. It’s hilarious that the keyboard was a separate thing and not on screen. But if nobody explores the idea, how do we know what works? For example, early ideas of having a few preinstalled apps on your phone, calendar, address book, each with its own unique button to launch it, actually transcended into the iPhone for one, where we have all these apps that can be installed on demand. It’s survival of the fittest, I suppose, where the notion of a keyboard survived but now it is touch-screen based. And the screen is so much bigger than it used to be.

Suit from The Flash. Courtesy of Robyn Haddow.

Kirill: I certainly don’t have a better idea for interacting with computers than what I have now – keyboard, mouse, trackpad – but it feels that it cannot be the last step in the world of human-computer interaction. And this is what interests me in these various productions – what take each one has on these interactions, and what can possibly seep back into the world of real-life interactions.

Robyn: For sure. It’s interesting to see what parlays into the real world, what inspires what. There’s definitely a strong crossover from some of the things that started as fiction and have transcended over to real world.

Kirill: I wouldn’t mind having Jarvis in my lifetime.

Robyn: Ditto. I think we’re living in an exciting time where a lot of imagination is crossing over very rapidly. At every major tech conference I’m always excited to see what new prototypes are being developed, or where things are going. Some day there probably will be a Jarvis.

Cortex gadgetry from The Flash. Courtesy of Robyn Haddow.

And here I’d like to thank Robyn Haddow for graciously agreeing to answer a few question I had about the art and craft of screen graphics and user interfaces for episodic television, and for sharing background materials. You can find more of Robyn’s work at her main site and on Twitter. Stay tuned for more screen graphics interviews in 2016!