Continuing the ongoing series of interviews on fantasy user interfaces, it gives me great pleasure to welcome Carly Cerquone. In this interview we talk about how much (or little) time screen graphics get in feature films, designing elements to be seen by camera, fantasy user interfaces as a storytelling device, and how she approaches the task of creating interfaces that target specific characters. As we discuss all this and more, we dive deeper into Carly’s work on the recently released “The Fate of the Furious” and “Spider-Man: Homecoming”.

Kirill: Please tell us about yourself and your path so far.

Carly: Growing up, I don’t think I could have ever considered myself “bored”. I spent most of my free time playing sports, watching movies, drawing, painting, and participating in a variety of extracurricular activities. While I enjoyed the fine-arts very much, I was also attracted to science, math, and technology. In middle and high school I enrolled in as many art and AP classes as my schedule would allow, and by my junior year I made the decision to pursue a career as an artist in the film industry.

After a lengthy search and application process, I was accepted into the Motion Picture Science program at the Rochester Institute of Technology. In addition to my coursework, I held jobs as a motion graphics designer, and a live graphics operator for Sportszone Live, our broadcast channel.

The summer of my sophomore year I was offered an internship at Cantina Creative. That year, I spent the two months in Los Angeles and earned my first feature film credit on Need for Speed. I returned the following summer for a second internship, and accepted a full-time position with Cantina immediately after graduation.

I’d be remiss if I didn’t credit my parents. I’m not sure where I’d be if it weren’t for their unwavering guidance and support. They’ve encouraged me in every aspect of my life and I consider myself extremely lucky to have them in my corner.

Screen graphics for “The Fate of the Furious”. Courtesy of Carly Cerquone.

Kirill: You’ve graduated the Motion Picture Science program at RIT. What is special about that program, and what are the challenges that the education system needs to meet in the world that is undergoing such dramatic changes in consumer technology?

Carly: Motion Picture Science (MPS) is a truly unique program. It provides a science and engineering based education in the fundamental imaging technologies used for the motion picture industry. It straddles two colleges – the College of Imaging Arts and Sciences, and the College of Science, and joins a core curriculum in practical filmmaking and imaging science. When I enrolled, MPS had just entered its 5th year of existence, and is still the only program of its kind in the country.

I don’t think it’s realistic to expect the education system to keep up with technology by way of lesson plans or curriculum updates. It’s definitely important for students to be knowledgeable of current industry standards and practices, but that alone won’t be enough to carry them through their careers. I’ve met artists that have graduated from rigorous design programs, others that are completely self-taught, and others still that are educated in an entirely different field! The one thing that each of these individuals have had in common is a drive to learn and continue learning at every opportunity. If an institution can provide their students with the tools and motivation to learn outside of the classroom, they will have created an education to last a lifetime. Students who are fortunate enough to graduate with their appetite for knowledge intact, and the ability to gain it on their own will be able to adapt no matter how frequently the technology changes.

Kirill: There are so many screens in our lives, and so much software that we interact with every day. When you talk with people about what you do for a living, do you think they are surprised to hear that everything has to be explicitly designed?

Carly: Absolutely! Most people know that holograms and other obviously-futuristic elements are created in a studio. They’re often surprised to find out that even the most basic phone screen and computer monitors have been designed, animated, and replaced by an artist.

Kirill: What are your thoughts on the amount of time screen graphics get in feature films? How does it feel to see your work appear and then be gone in almost the blink of an eye?

Carly: Hahaha, well when it’s worded like that it doesn’t feel great! But in all honesty, I don’t mind. Screen graphics exist to support the story. Sometimes their only function is to add to the tone of the film. Other times, they are needed to deliver important information to the audience. As long as my work is visible long enough to perform its intended function, I am happy. I can indulge in the intricacies of the design and animation in the showreels.

Screen graphics for “The Fate of the Furious”. Courtesy of Carly Cerquone.

Kirill: What was the most surprising thing for you when you started to work on your first feature film production?

Carly: My first feature film was “Need for Speed”. At this point in my career, I was new to the entire process. I had done solo and small team projects, but nothing of this scale. I wouldn’t say I was surprised, but I was certainly impressed by how efficiently the team was run. From where I was sitting, the communication between the producers, supervisors, and editorial was seamless.

Kirill: When you design specific elements and animations, do you think about how camera will see them, or do you think about the viewer? Is there a tension around how the camera captures those graphical elements?

Carly: Camera first. If I design without the frame in mind, I place myself at a disadvantage. I could spend days designing something truly exceptional, but if it is covered by an actor for the majority of the shot, I haven’t done myself or the production any good.

If I’m working on a hero shot, I take the viewer into greater consideration. As you mentioned before, these graphics are only featured for a moment. It’s important that they deliver the information to the audience quickly and clearly.

Kirill: Do you think it’s harder these days to keep the movie audiences engaged, because we can do so much with our everyday screens? How do you keep the screen graphics half a step ahead of where we are right now with our screens?

Carly: I do think that the audience has become desensitised to on-screen technology, but i don’t place the blame on the screens they have at home. FUI graphics have exploded in popularity over the past few years, but were being used in films well before I even considered a career in the field. By now, the audience has come to expect them as a storytelling device. Because audiences are so accustomed to FUI graphics in their films, no particular care needs to be taken when introducing them into the film. They are easily accepted, and allowed to exist without explanation. This is great news when I’m looking to integrate the graphics seamlessly, but not-so-great news when I want to captivate the audience with imagery alone.

Because I am rarely designing with consumers in mind, I don’t often think of my screen graphics and our everyday screens as competitors. The biggest challenge for me is giving each new design its own identity that makes sense within the context of the story. That being said, there are cases in which staying a step ahead of our current screens is necessary. The Netflix show “Black Mirror” does an excellent job creating a world in which technology is just advanced enough to be considered “futuristic”, but not so outrageous that the viewer is unable to suspend their disbelief.

If I were to design consumer based FUI graphics, I would approach it as a problem to be solved. I would first need to consider the functionality necessary for the story. If technology to satisfy this need already exists, that current system could be used as starting point. From there, I would need to think about how it could be made more efficient for the consumer if our current technical limitations were not an issue. This would hopefully result in a design that could very well come into existence within the next decade or two, but isn’t feasible just yet.

Screen graphics for “The Fate of the Furious”. Courtesy of Carly Cerquone.

Kirill: Speaking of specific feature films that you’ve worked on, how was “Fate of the Furious” for you? How much of it was the extension of the established language, and how much did it push things forward?

Carly: F8 was Gr8! (haha). I came into the project about one month into the design phase. Much of my job consisted of animating elements that had been designed by other artists, and creating new elements that fit into the already established visual language.

Kirill: On the same movie, the screen graphics are very detailed and dense. How would you compare this approach with the current trend that we see on our real-life devices that try to remove as much data complexity as possible when it is presented, even though all these sensors collect a lot of data?

Carly: While both designs most likely started out with functionality in mind, they meet very different goals once completed. Real-life devices need to be uncomplicated and easy to use for all individuals. A phone app, for instance, can be downloaded by any number of people with any variance of technical understanding. The UI/UX designer will need to put themselves into the shoes of a potential consumer with no prior knowledge of applications or their workings. If the app is unusable by that consumer, the design has failed.

On the other hand, FUI graphics only need to appear easy to use by a select person or group of people. For example, we designed an entire God’s Eye system for Cipher and her team in The Fate of the Furious, and two very different HUD systems for Spider-Man and Vulture in Spider-Man: Homecoming. When the audience sees a character interacting with the graphics, it’s easy for them to assume that the system is functional. The fact that only a single character (or small group of characters in the instance of Cipher’s team) is able to use the system suggests that it is too complicated for the layman, but still straightforward enough to be understood with the proper training or background.

Kirill: How do you approach designing within a system that needs to scale across dozens of screen (such as in the plane’s command center)?

Carly: When designing for an unusual shape, I try to first take into consideration the system’s user. What do they need to see most prominently? Where should the interactive elements be placed so as to be close-at-hand when needed? Are they even able to manipulate the design, or does it exist as a single piece?

For the command center on Cipher’s jet, we had to consider more than one user. The main screen graphics are located on the monitor in Cipher’s personal station. This works well for her when she is sitting in front of the monitor, but if she needs to see it from another position in the room, or a member of her team needs to reference the screen, they are out of luck. Placing duplicate graphics and video feeds on the large wall in the front of the room solved that problem. Sometimes, even if the design works well for the system’s user, it does not work well for the camera. In cases like this, a compromise needs to be struck. There is often a lot of re-organizing elements, and redesigning screens to find that perfect balance.

Screen graphics for “The Fate of the Furious”. Courtesy of Carly Cerquone.

Kirill: With all those screens in the background, what do you think is the right amount of animation? Too little, and the system feels lifeless. Too much, and everything screams for viewers’ attention.

Carly: You’re absolutely right, a lot of my job is to toe that line between “too much” and “not enough”. It all depends on the shot in question. A successful design will direct the viewer’s attention to the most important element in the frame. If the graphics are the most crucial element, I can catch the viewer’s eye with a more complex animation and ensure that the information is delivered effectively.

On the flip side, if an actor is delivering an important line, the screen graphics should either fade into the background, or frame the actor in such a way that he/she is the center of attention. I wouldn’t want to place anything flashy in the frame that could distract from the performance. That is, of course, unless the story called for it.

Kirill: If you look at “Fate of the Furious” and “Spider-Man: Homecoming”, and the current crop of sci-fi movies in general, would you say that there are certain common aspects to screen graphics dictated by the militaristic structure of the organizations behind those technologies?

Carly: Absolutely. I always try to design systems with both the manufacturer and the user in mind. In Spider-Man: Homecoming, Vulture’s HUD is based heavily on military drone displays. We know that Adrian Toomes is behind the making of his own system, and would have drawn from other sources when developing it. As a blue collar salvage manager turned weapons manufacturer, Adrian lives for efficiency. He would design for functionality, and nothing else. A military drone seems the most obvious candidate to source his design. Another character in the MCU, War Machine, has a similarly based display. Because Rhodey was formerly a pilot in the United States Marines, it stands to reason that his display would be organized by Tony in a way he could easily understand.

Designing for films is interesting in this way. If the system that I am creating is meant to have originated from an organization or character in the film, my personal style and preferences should have little to do with the final product. Instead, I need to put myself into the shoes of the “designer” responsible for the UI in the film, and imagine what they would create.

Screen graphics for “Spider-Man: Homecoming”. Courtesy of Carly Cerquone and Cantina Creative.

Kirill: Staying with “Spider-Man: Homecoming”, what were the particular challenges of designing graphics for a system that doesn’t use traditional screen rectangles? How does that figure out into the interactive aspects of that technology when you don’t use keyboard, mouse or stylus?

Carly: I love Spider-man’s HUD for exactly this reason. Unlike Stark-tech we have seen in the past, Peter Parker and his HUD elements are never seen in the same shot. Spider-Man isn’t encased in a helmet. Instead his eyes act like camera lenses, helping him focus, narrow, and expand his visual field. When laying out the screens, I imagined that eye-tracking software would have been a necessary aspect of his suit. It would allow him to both interact with, and see past his HUD elements without the use of outside devices. The ferry scene does a good job of showcasing this subtle piece of technology. While Spidey is honing in on the criminals and their dealings, the pop-ups used to display information within the HUD are constantly floating and re-adjusting to maintain a clear view of his targets.

This, of course, posed a challenge. Each shot had to be treated differently depending on the footage being analyzed. The “off limit” areas were rarely in the same place twice, making each HUD shot its own puzzle to solve.

Screen graphics for “Spider-Man: Homecoming”. Courtesy of Cantina Creative.

Kirill: What were the discussions around the evolution of the Spider-Man’s HUD as the hero leaves behind the training mode and transitions into deeper, more powerful capabilities of the suit?

Carly: In the film, Peter Parker removes the “training wheels protocol” before he is given permission from Tony Stark. This gives him access to hundreds of web-shooter combinations, and various suit modes. Because Peter gains access to his full capabilities before he has been cleared, it was important that the boot up feel complicated and overwhelming. The HUD rifles quickly through new nodes, data tables, and image analysis functions before settling into its default mode. From this point forward, Peter is bombarded with visual information that can sometimes be more distracting than helpful.

So often in films involving FUI, the audience can be made to feel out-of-touch with the technology being used. Peter’s obvious unfamiliarity with his suit’s functions brings him down to the same level as the audience, and makes him a much more relatable character. He’s just as confused as we would be!

Screen graphics for “The Fate of the Furious”. Courtesy of Carly Cerquone.

Kirill: Speaking of your work as a designer in general, is it always sort of a fight against the restrictions placed on your work – such as very short time for those hero screens that we see in sci-fi feature films?

Carly: Hahaha, yes, for sure. Time constraints are a considerable challenge for hero screens. When the graphics are given that much real estate in the frame, they are there to drive home a point. Screens of this nature often go through many rounds of revisions before striking the proper balance between interest and clarity. Finding that compromise is always a little bittersweet. I’m thrilled to have created something that successfully enhances the story, but sad to see bits and pieces of my design dropped in the process.

Kirill: If we’re speaking about the world of real-life technology, what are the main pain points for you? Not to single out any particular product, but some of the high-end professional tools at the level of complexity of Microsoft Office or the various Adobe offerings have so many features that navigating their interfaces becomes a very hard task to accomplish.

Carly: When working with a new or unfamiliar tool, I’ll sometimes find myself thinking: “You know what, if I were in [this program] I’d know exactly how to execute this”. I’ve become so proficient in my most frequently used programs that attempting to complete a simple task using an alternative poses a challenge. My frustration is not with myself for being unfamiliar with the program, or with the program for being foreign, but with the idea of wasted time. Thankfully, there are a lot of resources available to learn learn from, and once I have familiarized myself with a new program or tool, I am happy to have gone through the trouble to add a new skill to my repertoire.

Kirill: Is there anything particularly exciting for you in the world of technology in our everyday lives, or perhaps something that is promised to be just around the corner?

Carly: Augmented reality! I feel like we are right on the cusp of something truly revolutionary. Advances in AR technology will impact the entertainment, medical, military, tourism, gaming, and advertising industries to name only a few. I get giddy just thinking about all of the possibilities.

Screen graphics for “The Fate of the Furious”. Courtesy of Carly Cerquone.

And here I’d like to thank Carly Cerquone for taking time out of her busy schedule to talk with me about the wonderful world of screen graphics / fantasy user interfaces for feature film. You can see more of Carly’s work on her portfolio site, and you can also find her on Instagram and Twitter. And, if you’re interested to read additional interviews about the wonderful world of screen graphics and user interfaces for film and TV, click here for more.