'The Falcon And The Winter Soldier' Weta Digital Visual Effects Supervisor On Bringing Superhero Action To Life On The Small Screen [Interview]

Charles Tait joined Weta Digital in 2000 as a Senior Compositor on The Lord of the Rings: The Fellowship of the Ring. In 2004, he was promoted to Compositing Supervisor and spent a year designing the look of King Kong's rampage through New York in King Kong. Now, he has over 50 credits to his name, including Avatar, Rise of the Planet of the Apes, The Adventures of Tintin (during which he became Head of Compositing), and Prometheus. Charles transitioned into a Visual Effects Supervisor role on The Hobbit trilogy and worked on Deadpool, Avengers: Infinity War, Alita: Battle Angel, Game of Thrones, and, most recently, Marvel's The Falcon and the Winter Soldier.

I recently had the chance to speak to Charlie about Weta Digital's work on the Disney+ series and how they brought Sam Wilson's high-flying action scenes to life.

So WETA Digital handled the premiere's epic opening sequence, the canyon chase. Can you take me through the creation of this sequence? I imagine you start with the script, storyboards, and concept art — is previsualization the next step?

Yes. We previs'd it, which gave us a rough idea of what the sequence needed to be. We'd already seen some plate shots and some live-action footage, like inside the C130 plane and the helicopter, so we knew what we had to do, but we did start with the previs and make it our own. We had Sidney Kombo-Kintombo (Animation Supervisor at Weta Digital) do that, and he did an amazing job, and people got more and more excited about the sequence. While that was happening, it gave us time to develop the CG assets and develop our method for creating the canyon.

So, you're blocking out the scenes and finding the story beats with previs. As far as the canyon environment itself, is that based on an actual location? Or is it assembled using bits and pieces of different real locales to create a new space?

It's exactly that. It is a real canyon: Paria Canyon, which starts in Utah and ends in Arizona. It was a great canyon because it starts very narrow and widens out at the end, which was perfect for us. We used that real-world canyon model to give us a very natural realistic form, and the previs and animation were staged in that geometry. We took a terrain model of the Libyan desert and the canyon environment and joined the two together to make the one thing. So, we had a natural form, which was definitely a good way to go, and then we added additional geometric detail to it by using small bits of rocky terrain.

So, you're building this digital environment, creating the C130 plane and helicopters, and then you've got digital doubles for Falcon (Anthony Mackie) as well as the wingsuit jumpers. I know real footage was shot with professional jumpers, so how are you integrating that with the digital environment? 

They didn't fly through the canyon — the canyon only existed in our world, but they did some pretty great flying and it was so good to start with some real material. They were jumping over flat terrain from a very high altitude, but they did do the stunt with the Captain Vassant character, so that was pretty great. We got some nice footage of that, but they jumped out of a light aircraft so we had to remove that and swap it for a C130.

The door flies past the camera as the camera pulls out backward and we see them diving. In those shots, there were a handful of shots we'd basically rotoscope the jumpers out so that we could put them into the digital terrain. Another thing we had to do, for continuity reasons, was figuring out the altitude that each shot would be taking place at because it had to feel like they were getting lower and lower and lower and closer to the canyon before entering the canyon. All the real skydiving footage was from multiple jumps happening at different heights. If we just replaced the terrain, it wouldn't have cut together right, so that was an interesting exercise.

I can imagine. How are you accounting for the depth of field, and how are you capturing that sensation of the speed of moving through that canyon?

That was something we thought about right from the beginning. If you make something go really fast, but what's behind it is far away, it doesn't look fast, so this was sort of a tricky puzzle to work out. We decided to base the speed of all of the flying, the Falcon, the wingsuit jumpers, on how fast an NH90 helicopter flies. We settled on 350 km an hour as an easy figure to work with. The idea was that all of the animation would be based around that, so if something needed to appear to be faster we'd move it closer to the terrain.

For some of the scenes, you've got Anthony Mackie on wires in front of a blue screen. Is he able to see the previs or get real-time feedback from a monitor? 

He would have probably liked that a lot [laughs] but the timeline of the production didn't allow for that. That footage was already shot before we started actually. So, in fact, when they shot the blue screen material for him they did it outside, which is always good, we want natural light, that's always going to help if we're supposed to be putting him in an outside scene. But unfortunately for them, it was pretty overcast and wet when they were shooting him, and we've got to put him in a very sunny, dry-looking environment so that presented some problems for us.

We also noticed that they were shot like a kind of hero portrait of a person. He was out of focus apart from his face because he's lying down, you know, his feet are furthest from the camera, so we had too shallow a depth of field to be able to work with. Compositing that with the background would mean we have to make the background all out of focus as well, and then we've got something that is a bit too portrait-y and not really action-camera, not the super wide-lens action camera look we wanted, so we got around that by using a digital double. The face is Mackie, we matched it with our digi-double and lined it up perfectly.

So it's still his face, his performance, but we helped make him look like he was more in the world by adjusting the lighting effects.  When he rotates, the sun moves around him, because he's rolling away from it, and that just wasn't there before. We were able to light it in our sunny environment and give it that really hard key look for bright sunshine.

What is the revision process like? Do you get so far down the path on a sequence and then suddenly you realize something has to change, or you have to adjust where a scene begins or ends? Or, because of the previs work, are those things pretty well ironed out?

I would say we're always ready for that eventuality, but most of the revision process is done in previs. We knew what we needed to do, and once you get into a groove and everybody's happy with the way things are looking, making those shots later in the schedule is often easier. Even though it looks like we're sort of ramping up or really pushing out finished shots at the end, it's because all the questions have been answered by then, you know? Things have been revised and you really know what you're doing and what you're going to get.

You're mixing so many techniques to accomplish this one sequence.  I think a lot of people that don't have knowledge of the filmmaking process take for granted the ability to just make the impossible look real. On a daily basis, we see things that aren't possible, but look effortless in the way they were achieved. How many hours of work, how many artists are involved in just this sequence?

It's hard for me, looking out, to see how people, as you described it, how they perceive what we do. I've been doing it for a long time, so I don't really know how people think of it. The number of artists ramped up with the shot schedule ramping up but, I'd say most of the way through the project we would have had a team of 50-60 people. Maybe a 100-200 people touched the sequence at one point or another. We worked on this for a year, and that's the real thing people don't realize. The exact number of crew, because they come and go, they're required at different times on different parts of the process, the total number ebbs and flows. But it's really the time, the overall time. It's an exciting process, it's good fun all the way through, it really is, but it does take a long time.

For the finale, you also did some work with Sam's new Captain America suit. Is there anything that was particularly challenging about that Captain America wingsuit versus his previous suit?

What was most important is that it had to be striking and exciting. It was a really important shot when he smashes through the window and rolls and stands up heroically, showing that he's the new Captain America. It was a big deal. It was a big deal for Marvel, it was a big deal for Eric [Leven], and for all of us. We knew all eyes were on it and it had to look great, that was the main thing.

Obviously, the actor's wearing a costume, and he rolls and stands up and we did a digi-double handoff so he's CG guy flying up to the window and he smashes through the CG window and rolls and hands over and stands up as Anthony Mackie in the suit. That's a hard shot, but not something we haven't done before, but I think what made it hard was how important it was. You know, it's important that it looked really spot-on. I watched a lot of those fan reaction videos on YouTube and I really think we nailed it because they were so excited when they saw it. It was really gratifying when we saw how much fans liked it.

Absolutely. That sequence, and the way you revealed the suit and then the shot of him landing, angelically, felt like a splash page right out of a comic. I thought it was great. OK, final question. How did COVID affect working on this streaming series? You've worked on Marvel films in the past, but I imagine this was a bit different.

It affected everybody's work, but WETA did an amazing job of getting everybody working remotely. Within just a few days, 1500 of us were working at home. We started on the show working remotely for quite a few months as we developed things and to be honest it was fine. Everybody was really surprised at how well it worked. It affected the schedule, there were changes at Marvel's end with when things were going to come out, the releases of their different projects, but for us, it didn't have a huge effect.