The art and craft of screen graphics – interview with Ryan Jefferson Hays

June 14th, 2017

Continuing the ongoing series of interviews on fantasy user interfaces, it gives me great pleasure to welcome Ryan Jefferson Hays. Over the last twenty years his career took him through the worlds of multimedia, web design, TV, video games and, most recently, feature films. At Territory he worked on “Guardians of the Galaxy” and “Jupiter: Ascending”. At SPOV, in addition to a variety of game cinematics for major titles, he worked on “Mission Impossible: Rogue Nation”. And as the creative director at MPC his job was to oversee everything that went into making the digital worlds on “Passengers” and “Ghost in the Shell”.

Kirill: Please tell us about yourself and your path so far.

Ryan: I’m from Australia originally, a little place in Western Australia called Perth. I went to an art school there, majoring in multimedia. During my last year, I joined a company called Max Multimedia that specialized in music and sports. Their focus was on multimedia and CD-ROMs, all the way back in 1992-93. The dot-com boom was in full force, and there was a lot of money thrown around, with young kids and fast cars. And then the company went bust after six months [laughs], and I was one of the first ones to fly the coop.

I moved to Sydney in 2000, joining a web company called Massive. They did traditional web sites, and I got bored really quickly. At that point I was into Flash and Director. Director used a programming language called lingo and it was very easy to come up with weird animations. At the time I was part of a design collective called Australian Infront, which grew from a few people to host design meets and doing the conference circuits over time. Through Infront I met Dom Bartolo who worked as a broadcast designer, and after seeing my work he said that I should probably get into TV.

Through that connection I started at Foxtel which is a cable television company that had around 120 channels at the time. I worked on movies, documentaries and other productions, with my main focus on rebranding the sports channels. Then, around 12 years ago, I met my wife Jasmine. We couldn’t afford a house in Sydney so the idea was to move abroad for 2 years and save for a deposit, the aussie dollar at the time was very low so the exchange rate would help when returning to Aus. The choices were New York or London, we chose London as Jasmine has family there. Initially I started working for various television production companies then after a few years in TV I moved into video games, which I guess was the start of the move into feature film work.

Screen graphics for “Mission Impossible: Rogue Nation”. Courtesy of Ryan Jefferson Hays.

I worked for a company called SPOV for about five years, on game titles such as Call of Duty: Black Ops 1 & 2, Modern Warfare 3, Advanced Warfare, Resistance 3 and Titanfall. I was working on in-game cinematics with Spov, which incorporated UI and 3D narrative work. Then I got a call from David Sheldon Hicks. He used to work at SPOV before starting Territory so new of the work I had produced. I joined them, and my first job at Territory was on Killzone: Mercenary as the 2D lead. The very next production at Territory was my first feature film – Marvel’s “Guardians of the Galaxy”.

Me and another designer, Yugen Blake were tasked with pitching on the project for Territory based on the early script we received. We covered a load of different design aspects in that pitch and worked our asses off which ultimately helped win the job for Territory. As you can see it’s been a linear progression in my career, starting in multimedia, then web sites, then TV, then video games and finally film. I am not sure where the next step is to be honest.

Screen graphics for “Jupiter: Ascending”. Courtesy of Ryan Jefferson Hays.

After Guardians I moved onto “Jupiter Ascending”, working on the set with Wachowskis. It was a great experience to be on that set, even though that was the moment I lost the idea of the ‘magic of film’ [laughs]. You see people with fire extinguishers, pushing the side of a cardboard box with actors inside it imagining it’s a spaceship of some sort. You spend 12-14 hours a day, on a film lot in the middle of nowhere, because all the films that are filmed in London are actually filmed on a lot an hour or two outside the city. You get there when it’s dark, and you leave when it’s dark.

But it was fun project to work on and sitting next to the Wachowskis on set was a great memory. I worked with Nik Hill who was a junior designer at that point, and it was a huge learning curve for the both of us. You need to work very quickly in pre-production, banging out designs all day and not really seeing any of it until it comes out in the theater.

Kirill: As we’re talking about your first couple of productions, do you remember what was the most surprising or unexpected part of it?

Ryan: On Guardians I was not expecting the huge work load needed for the film, it was immense. There was a moment when we first walked into the boardroom filled with concept art. They had hundreds of images along the walls, and miniature sets on the desks. The amount of work and detail that went into the pre-production before they started building anything was incredible. That was the moment when it hit home how big that production was. It wasn’t just a few people working on it before we joined. They had a load of people working on it for years before we had a chance to see it. There was so much hard work gone into it, and it’s really great to see that.

Screen graphics for “Mission Impossible: Rogue Nation”. Courtesy of Ryan Jefferson Hays.

Things in TV production seem a bit more guerilla but you work it out, more or less. You don’t have a lot of time in TV, whereas what I saw on Guardians was a big eye-opener. I didn’t have any problems with that job at all, and it was great experience. The script was so open and vivid, and you could just do whatever you wanted. We were always in discussion with the production designers about the right direction for the film, and they told us to go with it. Usually when you work with UI, you only have a certain amount of colors. You don’t do pink or orange, for example. But they didn’t care.

The world was so vivid in color and there was so much going on, and someone made the comment that the world was vomiting color [laughs]. The Marvel franchise was coming from Iron-Man and Avengers, and I think that’s why they let us go wild with the look-and-feel. And it worked. Probably the only place that had structure was the prison. Those screens reflected that corporate entity inside the world, while everything else was all over the place. Territory is really great at doing those kinds of screens, where everything needs to be based on a strong grid because there’s a lot going on.

Screen graphics for “Jupiter: Ascending”. Courtesy of Ryan Jefferson Hays.

Kirill: You mentioned that your second film was “Jupiter Ascending”. How different was that in the sense that most of the screens were for aliens?

Ryan: Yeah the major challenge for “Jupiter Ascending” was to establish a look-and-feel for the alien interfaces that made it clear what was going on, but also make it feel very alien for the viewer at the same time. One of the screens we needed to work up was the navigation screens used in the small jump ship. The problem we needed to solve was there’s so much space traffic going on and between hundreds of worlds, and there must be a simple but elegant way to navigate this big mess. We came up with the idea of mapping possible routes for star constellations with flowing line work built from thousands of particles for the main interface, these particles could then reform into different maps once the ship had finished its warp jump. The idea was we were mapping out trade routes and paths for the different spacecrafts in different constellations at the same time. In the end it was very basic, but I think it got the point across.

Screen graphics for “Jupiter: Ascending”. Courtesy of Ryan Jefferson Hays.

Kirill: Do you have any trouble telling people what you do for a living, to explain what it is that you do and why everything needs to be explicitly designed?

Ryan: Oh yeah totally, early on when I was a broadcast designer I would tell my family back in Australia I was designing for TV. For years a few family members of mine thought I actually designed television sets… Now I just tell them I am a graphic designer as it’s a hell of a lot easier.

Kirill: What happened for you after these two films?

Ryan: My last project at Territory was working on “Hitman: Agent 47”. I then jumped back to SPOV for six months as lead designer on “Mission Impossible: Rogue Nation”, which was a more traditional UI project. It was a nice change after doing loads of fantasy screens, designing in the world of more realistic technology.

During that project I was approached by MPC, I went for an interview and that turned into a full-time job. They had a design department, but their focus was on the advertising side of things. The department did work on a few small film projects, and they wanted to keep that moving forward. They needed someone to come in and lead that department on film graphics work. The first six months was all about meeting with people internally and telling them that we could do that sort of work.

The way MPC is structured is that there’s a film department, an advertising department, and a creative department which is the production studio. MPC design sits sort of in the middle of all three. Those first months finally paid off when the head of the art department Ravi, reached out and requested our help on some design work for ‘Passengers’. At first it was set design changes. It started with concepting up new ceilings for one of the communal areas, then moved into new interiors and exteriors of the ship, and it kept on going for a few weeks.

Screen graphics / GFX for concourse on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Then I met up with the VFX supervisor on the project Erik Nordby, I showed him my reel and asked him if he had anyone doing the screen graphics in post. At the time they had people doing screen graphics on set, including Chris Kieffer and Vince Parker. A lot of the time, that work is rushed and is not always what the Director had imagined but it does help the actors a fair bit.

Once we had the sign off from both Morten the director and Erik we launched into doing the hero shots. At first the shot list was around 215 shots, the majority being hero screens focusing on the narrative, as well as few bits and pieces for backgrounds. After a few months of deliveries the director wanting to pull everything together asked us to redesign some of the work that was done on set. Out of roughly 500 shots with graphics in them, we ended up completing 425 in the end, over a period of 5-6 months.

The project was quite a challenge for the department we had some teething issues at the start, but we got there in the end. It was great to have Erik as the VFX supervisor as his experience helped a lot, and it was a great learning process for everybody involved. A big hick up was around two months before we finished the script had changes and we were asked about how we could use graphics to help further this new story line. It was a huge task to change and incorporate the graphics to relay those changes. The film didn’t get great reviews, but I think a lot of people judged it even before the film came out, but everyone that I know that has seen it thought that it looked great and that’s all that matters in my books.

Screen graphics / GFX for communications on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: Even though the technology on the ship is effectively frozen in time since everybody’s in some sort of cryogenic sleep, the technology there is still much more advanced than what we have now. What kinds of discussions did you have around that?

Ryan: We had to come up with two strands of looks. One was for the people who were traveling on this adventure to another planet. It had to be user-friendly, and easy to use by anyone on the ship no matter who they are. And the other strand was the engineering technical type look-and-feel.

For the user-friendly designs we wanted it to look slick and be understated. We looked at Microsoft’s HoloLens, Google Glass and other technologies, and how simplified they make their interface work. The color palette needed to be appealing and friendly, differentiating it from the more technical interfaces for the everyday user. It was all soft shapes and soft colors, with larger text for warnings.

Screen graphics / GFX for infomat on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

If you look at “Prometheus”, that’s quite tech-y and overloaded with info. You look at a screen, and you need to spend time to dissect it in order to know what it does. A lot of our stuff was straight to the point. If it’s a map, it’s a huge map of that concourse with simple info tags depicting where things are on the ship. We also did the designs for the talking info mat that answers questions. All those icons needed to be very simple, friendly and easy to understand even from a distance.

Morten was going with a classic Danish style for the interiors. We also worked on all the info boards running along the concourse using large numbers and icons for where things are. Where are the bathrooms? Where is the gym? Where is the basketball court? Everything was represented by simple icons. It was simple, basic, elegant design. And even the technical interfaces on the engineering devices were straight to the point but using a more sterile look and feel.

When I started at MPC, I had to set up a team quite quickly. That’s one of the core things you need – a good team behind you. The turnaround for the work is so quick. You want to work with people that don’t need constant direction all the time. This is the script, this is what the basic design looks like, take that structure and that look-and-feel, and build a screen that they can fill in with information. Otherwise if you have to sit with someone and design everything, you’re not getting anywhere.

Screen graphics / GFX for office screens on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: Do you find that productions keep on demanding more because the audiences want to see something that is more believable every time as years pass by? Would you say that creating the worlds of “Passengers” and “Ghost in the Shell” was more complicated than your previous work?

Ryan: Everybody wants to push themselves. You always want to do something new. When “Prometheus” came out, everybody wanted to do a holotable, but a much more advanced holotable.

We were trying to push things on “Passengers”. You have to work with the time frame, and that becomes a huge hindrance when you’re trying to push the envelope. Sometimes you need to go back to what you know best. In the end good design always comes through. When I work with other designers, that’s my emphasis. Don’t overcomplicate it. The director is always busy so his/her time is precious and may not understand what you are trying to convey in the review if you have overcomplicated the design. If it’s good design, the point will come across in a second. Do the simple things first, and if the idea is approved, you can try and work on it a bit more.

With the timelines on feature films especially in the post production period, there is not a lot of time to explore. You have to do that in your off time, between productions. You look up new reference materials that could be a great jump off spot for the next project.

Screen graphics / GFX for tablets on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: With all the highly-paid actors in these sci-fi productions, screen graphics don’t get a lot of screen time as an art form of its own. Jumping to “Ghost in the Shell”, you tweeted that you’ve worked on it for seven months. How does it feel to see those long months compressed into those few precious minutes on screen?

Ryan: It really depends on the project, with ‘Ghost in the Shell’ MPC Design took on a large portion of the work involved in each scene. Everything from design, 3D build, VFX work and into the final comp. Our screen time was a lot in the end so that was a big bonus.

From the very early on in the project the director Rupert Sanders was adamant that ‘Ghost in the Shell’ needed to be on another level with its visual appeal. Bringing on Ash Thorp and the team Ash had working with him was a brilliant move. Rupert is a commercial director, and I think he has a great eye for detail. Sometimes it’s a hindrance and other times it’s not. As the director, you always need to have one eye on the overall flow of the look-and-feel for the film. There was a lot of studios involved, and every studio had its own details to work on, and he was always on point with it.

“Ghost in the Shell” was a huge project for the studio, we worked on close to 250 shots throughout the film. We had so much work on our plate with this feature that we had to turn down other shots Rupert and his team requested us to work on, I felt taking on these additional shots would eventually impact on our quality of output. It’s always nice to have the faith of a director where he or she knows they can throw us any challenge and they know we we will get it first try. The design department had about 40 people working on the film over that 7 month period, with MPC as a whole having 600 people involved with the project. Territory were also one of the vendors involved, David and his team focused mainly on the signage through out the world of ‘Ghost in the Shell’.

Screen graphics / GFX for conference room on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

I was able to view a lot of the work that was coming from Territory and it was all on point. It was great to see their work as I got to work out their approach to it all. The project had two VFX supervisors, Guillaume Rocheron from MPC and John Dykstra. We worked closely with Guillaume at the start, and then John was brought on to be our main point of contact as it was becoming clear this project was massive.

We did the majority of the holograms work through out the film. The first shot we did concept work on was for the conference room, we started establishing early on what the holograms would look like and how they were built. After this look was signed off by Rupert we then moved onto the rest of the conference room scenes, the hologlobes, Batou vision, the hotel scan at the start of the film and then the tricky hologram disintegration for the Major, Kuze and Cutter. With this amount of work we had on our plates we needed to build a set of bespoke tools that were designed specifically to meet the requirements for each scene.

You see the difference between MPC Design and the other studios who do film graphics work is we can fulfil all aspects of a project, design, build, comp and along side that integrate smoothly in the film pipeline. As a studio we have full flexibility on any of the work we produce for film. This flexibility helps take a lot of the workload off the production company, so instead of a 1,000 shots, now they were down to around 750. It helps everyone in the end.

There’s a lot of design work out their in the film world if you can find it. The major problem I had was you come in early on a project in the pre-production stage and work on set, and then you don’t see your work until it comes out in the theatre. I would sometimes feel a little let down about how it looked on screen and it was not what I had expected at all. The team on ‘Ghost in the Shell’ worked on it from the concept stage to the finished delivery. MPC Design were involved in the whole process of designing, building and compositing of our shots. It’s a different approach to film work these days. It’s design-led visual effects, I would rather give the film makers problems than solve all their problems. It’s more fun that way.

Now that we have completed “Passengers” and “Ghost in the Shell”, we have proven that we can do it, and that we can do it on budget, and that is the major thing about film work these days.

Screen graphics / GFX for hotel scan on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: How did you approach defining the interactions between synthetically-enhanced humans and the technology around them?

Ryan: In the world of “Ghost in the Shell” everyone is hardwired into the tech. There are not many gestural interactions. If they get a call on a hologlobe, they plug themselves in and get on that call. Everything is visualized in front of them. I’m a big fan of the originals, and we’ve always tried to stay true to that look-and-feel, but also push that tech forward.

The original idea that Rupert had for the hologlobe was more of a larger well type device, with information spilling out of it and filling the entire room. We found out very early on that was going to be way too much work, and that’s when it turned into the hologlobe. All the information is contained in that globe and the outer layers become the viewport and interface. We used the shot at the start of the film as our test piece, where Aramaki is watching Major jump off the building and he is interacting with the playback. The globe becomes a live feed of the operation, cataloging all the information as the operation unfolds. Once we had a workflow for that initial hologlobe we utilised the same techniques covering all the hologlobes through the film. Some of the globes were used as communication devices, others used as personal home computers. The one overriding theme was that all the globes had streaming realtime data running through them, but there is no interactional aspect to it. All the characters are hard-plugged into their tech.

The conference room was another major part of the film we worked on. We based the look and feel of the conference room on the original as it’s too iconic not to. We came up with systems to animate content on and off, adding small pieces of streaming data all over the scene to add a real sense of depth. All the characters are hard-plugged into this room, with heaps of data flowing in continually. We had eight actors in those scenes standing around the holo conference room. You simplify it, and keep it looking nice. It’s a hard balance, but I think we got it in the end.

Screen graphics / GFX for hologlobe on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: Now that you’re working at a much bigger studio, does it help to have more direct communication between different teams working on this one big project?

Ryan: Majorly, and that was one of the reasons why I wanted to do it. I’ve been a freelancer for 12 years, and it was time to set down some roots and try to establish something. I found when you’re working as a freelancer, most of the time the company that you’re working for is not going to put you in front of the client to discuss projects. Which I totally understand but it’s always been a concern of mine, and especially on film.

It’s all down to politics between the company, the full-time staffers and yourself. Some people may worry that you’re doing such a good job that the next time a job pops up it will go straight to you as the freelancer instead of back to the company. And even if you’re the art director leading it, you’re still going through someone who’s hiring you for that job, which is understandable. The lines between you and the studio are blurred. You’re trying to understand what that director really wants, and you want to go to straight to either the VFX supervisor or the director. But you’re rarely going to do that as a freelancer.

I did have a chance to do that on Mission Impossible, and I really enjoyed that and wanted to do it more often. You also get a chance to tap into the world of the larger studio. As a freelancer, you’re working with your specific motion tools like Cinema 4D and the Adobe set. But in a larger post-production studio you’re working with people that are experts in so many different fields. I’d either have no time or no desire to look into all that as a freelancer.

Screen graphics / GFX for holo disintegration on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

On “Ghost in the Shell” I was coming up with complex ideas, and then passing them on to the 3D effects team to run with it. We were working on a specific shot for instance to visualise Kuze’s head in the conference room. We thought about how would section 9 get all this info on Kuze if he was so mysterious. Well you have these CCTV cameras all over the city, which have small glimpses of him, and that’s what we were trying to visualize in a holographic form. We were coming up with different architectural references, pulling ideas from various places. I asked if they could come up with pointillization-style composition of fifteen heads, always morphing and never being one solid face.

The big plus of working in a large production company is you always have someone in the building that you can turn nudge to get their opinion of the best way to execute any of your crazy ideas. London has three buildings MPC filled with these very talented people.

Screen graphics / GFX for holo disintegration on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: You’ve been involved with both “Passengers” and “Ghost in the Shell” from the beginning through the post-production. Do you get to enjoy such a movie when it’s out in theaters, or do you look at how your work was integrated into it?

Ryan: I was lucky enough to attend the premiere of “Ghost in the Shell” with the cast and crew in LA, and that was a great experience seeing all our hard work on screen. The majority of times I don’t get to go and see movies in the cinema, as I have two young children. But when I do it’s always a very enjoyable experience to be sitting there with the people who have helped make the film, rather than sitting down and critiquing how your graphics have been implemented.

It was a big film and a lot of the people at the studio were huge fans of the originals. When we just got the job, we had a small team of around 10 people. As the word got around the building that we were working on “Ghost in the Shell” on the fifth floor, people would come up and say that they’ve just finished their job and they wanted to join ours. That was a good feeling.

Screen graphics / GFX for hotel scan on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: Do you think about how your work will be perceived by the next generation in, say, 20-30 years? Or how much will have changed not only in how stories are told on these screens, but also where the technology will actually go?

Ryan: There will always be design work, and it always goes through your head what will be next. A lot of the work that we are doing now is being mimicked by technology anyway. When a film is out, the software stuff that we did always finds its way as an interaction based idea 3-4 months later out there in the world. That’s always nice to see.

It’s not something that I really worry about. I just want to make good work, to be honest with you. It’s making sure that you do the right work for the right people, and that’s all that matters to me at the moment. I’m not worried about how it will be perceived in 10-15 years. I’m a fan of old films, anyway. One of my favourites is “Escape from New York”. I love the design work in that film. It’s basic, and I would love to do some basic stuff like that, rather than overcomplicate some screens that don’t need to be.

Screen graphics / GFX for basketball court on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: My favorite example is the first “Alien” movie, and how well it has survived over the decades. The design is so coherent, and is not trying to be too futuristic. It’s this hardcore industrial ship, and it has hardcore analogue technology that is meant to last for however many decades the ship is in space.

Ryan: Exactly. It’s a workmanship, rather than a fancy stylised ship. We’re actually working on a short film for Alien at the moment. We based a lot of the graphics on the original but because it’s part of the Covenant, we had to match some of the graphics in the new feature. Toby Dye who is the director for RSA said that if we wanted to pull references from the original, we should run with it. Then Ridley Scott saw it and he liked it as well.

When we work on films, we try to pull little things into it that hint at the stuff that we like. “Ghost in the Shell” has bits and pieces all over that hint at the original film. The logo for Section 9 was based off of the original one, for example. It’s tying all the different elements together that makes it cohesive and understandable to the viewer.

Screen graphics / GFX for hologlobe on “Ghost in the Shell”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: You’ve been working with design software for a while now. Are you happy with those tools, and the complexity that they bring with them?

Ryan: AfterEffects could get a good update [laughs]. Shape layers are crazy, and they need a big update. My problem is mainly with rendering and render farms. It’s never what you think it’s going to be, especially if it’s a time-based thing. You get a call in the morning, and they need something by that afternoon. So you start thinking that you have 3 hours of design, 2 hours of build and 3 hours for rendering. That’s one thing that I wish I could work out. Even after all these years I can’t figure it out. It’s never what I think.

And then you get into 3D. On “Ghost in the Shell” we had a lot of files that came from other vendors. Some of those were made in Octane and other tools. Octane is not in any film pipeline as it’s not CPU based. It’s not a piece of software that film companies use, Octane shaders do not translate over to Arnold. That was a huge problem.

Screen graphics / GFX for spacesuit on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

Kirill: Would you say that the render times go hand in hand with how elaborate these productions are, year after year? I could trace the complexity of robot transformations in the Transformers franchise, from fully-blurred transitions in the first one to progressive refinements in each consecutive iteration.

Ryan: We push the software to the limits. We are always trying to come up with new ways to render and animate. Computers are getting faster with more memory and more processing power. But then you’re just pushing it more, and it’s a never-ending loop of frustration. I’d rather keep on pushing that envelope.

Screen graphics / GFX for sleeping pods on “Passengers”. Courtesy of Ryan Jefferson Hays. See more of MPC’s work on the movie here.

And here I’d like to thank Ryan Jefferson Hays for taking time out of his busy schedule to talk with me about the wonderful world of screen graphics / fantasy user interfaces for feature film. You can also find Ryan on Twitter. MPC’s main site has more of the work the studio has done so far on major feature film productions. The most recently published reels for “Passengers” and “Ghost in the Shell” dive deeper into MPC’s screen graphics and visual effects work for these two productions.

Finally, if you’re interested to read additional interviews about the wonderful world of screen graphics and user interfaces for film and TV, click here for more.