Less is more – that’s how VFX supe Hal Couzens approached the visual effects in The Bourne Legacy.
Coupled with a push towards getting as much ‘in camera’ as possible,
the film’s effects in some of the major scenes were always secondary to
the story point. With Legacy soon to be released on
DVD/Blu-ray, we break down a few key shots – including the drone, the
wolf, the burning house and the Manila chase.
Directed by Tony Gilroy, the film was shot on 35mm by Robert Elswit and ended up with about 800 shots, with the lion’s share completed by Double Negative under visual effects supervisor Mike Ellis. Additional work was handled by Level 256, Rhythm & Hues, Phosphene, Somnyo Films and Lola VFX, with Legacy Effects contributing the animatronic wolf.
A different drone used to deliver blood samples and supplies is first seen taking off from a snow-covered field, achieved with a full sized art department model and then a digital takeover. Cross later flees the cabin as a missile streaks past him and destroys it, killing Number Three. Managing to shoot down the aircraft with a sniper rifle, Cross then draws another drone to strike at a wolf (see below for how that was accomplished), now carrying his tracking device as a diversion.
Double Negative approached the drone shots early in production by heavily previs’ing the sequence and referring ideas to the director and to Couzens for practical aerial plates to be filmed near Calgary – also relying on Google Earth data to scout mountains. “We did a lot of previs to help tell the pilots what we needed and what would work in the cut,” says Mike Ellis. “The editors cut our previs into their scenes, and would test block-outs. Once we’d got that previs nailed, there was a direction and a road-plan for how to get there.” Most of the final drone work was completed by the studio’s Singapore office.
For the shot of the drone missile destroying the cabin, DNeg combined plates of Renner running with a whip pan to the cabin before it explodes. SFX supervisor Garry Elmendorf rigged the structure as a full-sized explosion, enhanced only with the CG drone swooping past, the missile trail, CG snow and just a few frames of Number Three in the doorway. DNeg also added the antenna array and storage box exploding to the side of the cabin.
“The attack is a bridge of two plates and a panorama shot on location and a face replacement,” explains DNeg CG supervisor Pietro Ponti. “When Jeremy ran out of the cabin, he pretended to see the missile and turned to see it hit the cabin. The camera then whips through 90 degrees. Before the motion blur stops in the plate, we pick up with a panorama shot on location and we continue the swap and then land exactly at the location of the house as it explodes – which was the practical explosion. To bridge the three plates we brought LIDAR scans of the environment and cabin and the panorama into Nuke and re-aligned them. Then we created the missile from the drone as a CG render with CG smoke trailing behind it.”
For aerial views showing the drone returning to hunt Cross, and for monitor screens seen in a control room, the production chose to acquire helicopter plates. “We started with the previs as an aid for the director, editor and a guide to the chopper pilot and aerial DOP,” explains Couzens. “And up in the air you get happy accidents. That gave us a library of back plates but also enabled us to build some camera moves into the plates during the shooting stage. We went for realistic camera positions and movement, sticking to how fast a real drone would fly compared to a camera helicopter too.”
Armed with the background plates, DNeg set to work augmenting them to match the previs and required drone movement and to add in realistic layers of snow haze, as the original plates were shot in the clear, dry Calgary air. “The final approach,” outlines Ponti, “was to take the plate and get our matchmove department to work out the shapes of the mountains and create very simple geometry for the mountains that we would pass on to comp with depth passes rendered in RenderMan. With that as a guide they could create a hazing effect. The good thing was that the trees were so green and the ground so white they were able to luma key the trees and do a treatment to make them full of snow even if they weren’t.”
A snow simulation effort for the drone scenes, and for the wolf shots and for Cross in the mountains, was orchestrated by effects supervisor Viktor Rietveld. “He actually modeled crystals of snow and then wrote a little bit of software to clump them together in actual flakes,” says Ellis. “What you’re seeing are actual real crystals clumped together, with real dynamics of wind and turbulence added in. And when you see falling snow in a forest or around trees, it’s very difficult to place it in the right depth, otherwise it just looks like it’s falling over everything. So we used LIDAR for that, particularly in the wolf scene for the ground and backgrounds. We had all these trees in LIDAR and used that as holdouts.”
The monitors in the control room featured both live playback and bluescreen inserts (comp’d by Level 256) showing the drone’s view. Live playback graphics were created by Jasen ‘Jaz’ Nannini of Somnyo Films. “One of the story points was the snowstorm reducing the pilot’s vision,” says Couzens. “So where we used live action helicopter plates we had to treat them to look like the rest of the snowy drone command shots created by Jaz.”
Aware of the second drone, Cross removes a tracking device in his
torso and manages via a phenomenal wrestling scene to feed it to
‘Alpha’, a wolf – which bears the brunt of a missile – and the agent is
assumed dead. A combination of real and hybrid wolves, animatronic and
stuffie work by Legacy FX and CG creations from Double Negative were
relied on for the wolf sequence.
“We had real wolves and we were going to do as much as we could with them,” recalls Couzens. “But we found early on that what we needed was a furtive, sly, sneaky wolf. Wolves are very timid creatures, so they’re hard to train in one sense. Also, they’ll always approach another animal or person very cautiously. So what we found was that we couldn’t get our trained wolves to do anything slowly. We could get them to stand and growl and get them to come forward and move really fast, but we couldn’t get that middle ground of them coming forward slowly.”
Close-up shots on the wolf standing were mostly the real wolf. On set, too, hybrid wolves (half-wolf, half-dog) were filmed, but used only as animation reference thanks to multiple witness cameras. Shots of the animal approaching slowly, lunging or breaking away were generally the full CG DNeg creatures, “but we would start with the first five frames of the real wolf for real-world line-up and lighting then do a take-over to full CG,” says Couzens.
In the sequence, Cross traps the wolf; as it gets thrown up, a CG wolf is seen, with the suspended version a Legacy Effects animatronic. Shots of Cross wrestling with the wolf were filmed with Renner interacting with a soft animatronic. “That gave us the interaction between hands and fur,” adds Couzens. “We then replaced the head and teeth and eyes, and legs where necessary. There are 15 CG wolf shots and 12 enhancements to the animatronic or real wolf. So it ended up being fairly substantial, but the shots are fairly short and sharp.”
Still, full frame views of the wolf were required, and DNeg relied on
the real-world reference to model and animate the CG version. That work
began with an early Los Angeles visit by Pietro Ponti to Steve Martin’s
Working Wildlife, which would be supplying and training the lead wolf
for the sequence. “We took a ton of photo reference, the same I would do
for a digi-double of an actor,” says Ponti. “I set up this gray back
stage and the trainers were really good in trying to keep the wolf
steady. I shot at different heights, first off looking down at it and
the looking up from below with 100mm and 50mm lenses. I also asked the
trainer to hold open the mouth for reference. Then we went outside and
let the wolf run free in his pen and I took videos on two different
cameras.”
DNeg then used this data to build a CG version in Maya, grooming with Shave and a Haircut and the studio’s proprietary fur tool. A promising test was presented to the director while the sequence was being filmed. Animation supervisor Stephen Enticott was also on-hand in Canada to see the actual wolf in action. “In our favor was that we had the real wolves on set and the real environment,” says Ellis, “so we knew exactly what the wolves looked like. They weren’t necessarily doing the right thing but they were in the right place, in the right lighting. We could see all the right nuances, the twitching, the hair blowing.”
After the shoot, DNeg refined its CG wolf from the fresh reference
and stepped up the modelling of its underlying muscles and tissue.
“Anything that was not in line with that reference, well, we tried to
change it,” says Ponti. “One issue with the wolf was that from one
camera angle it looked great, but from another it might not look the
same – it was really important to understand how he was made underneath
the fur.”
The reference also, of course, drove the style of animation. “A lot
of times when you’re doing CG wolves you want to over-animate them, but
instead we took out things,” notes Couzens. “We kept the small
subtleties – small movements in the ears and eyes, not a lot of
secondary animation in the fur, a little bit of wind blowing – not a lot
of what you’d describe as traditional jiggle or deformers going on,
though. I think the result is you get a sense of calmness, so there’s
slightly more menace from the growling.”
Ponti took HDRs of the location, as well as gray ball and color chart reference to suit DNeg’s newly adopted physically-plausible and image-based lighting approach to rendering – although the wolf also used the studio’s existing lighting techniques. “For the fur,” says Ponti, “we didn’t have indirect diffuse or indirect specular, we just cheated an area light to get bounce from the snow. But we really wanted to control the difference between root and tip on the fur – you get a lot of difference between the colors around the coat. In the end we did add some cinematography-like lighting, such as a rim light or what not to enhance the fur, but it was very subtle.”
Snow simulations (described above) were an important part of integrating the wolf into the scene, too. 2D supervisor Peter Jopling oversaw the snow interaction on the wolf, taking two different approaches depending on whether the animal was live action or CG. “Where we wanted snow to land on a static or nearly static wolf we found we could track a point of fur and then stick individual flakes,” says Jopling. “The same applied to Aaron. It was amazing how few flakes were required in order to sell the idea of being integrated, sometimes only one or two.”
“For the CG wolf,” continues Jopling, “we had more options. We could use the p world pass in Nuke to select a very specific area of the wolf, down to a few pixels and then stick a snow flake to him, regardless of his movement. We also had a full bells and whistles approach for a few hero shots where we would create custom sims to create full fx snow renders that would fully interact will Aaron and wolf which would require a full body track of the characters in order to create proper hold outs for the snow.”
Cross now seeks out the help of Dr. Marta Shearing – who in the past
had carried out tests on Cross – to wean him off the
chemically-engineered meds on which he relies. Having survived an attack
on an Outcome lab, Shearing is confronted by CIA assassins at her
house. Cross rescues her and the two make a getaway, setting fire to the
house to cover their escape. Double Negative contributed crucial
invisible extensions and digital flame/smoke effects within the
sequence.
“For the house,” recounts Couzens, “we were originally going to shoot
at a somewhat dilapidated premises and we were going to contribute
money to restoring it. But for some very significant safety reasons, it
was really too dangerous to shoot there. So all of the interior is on a
greenscreen stage as a three-storey house. The basement is a location
somewhere else. The exterior shots were filmed on Staten Island, where
we built the driveway, a couple of the out-buildings and the porch
itself as a full-sized set. That was digitally topped up as a CG house
for wide shots.”
The reference location, Oliver Bronson House in Hudson, New York, was featured in one shot of Cross climbing parkour-style on the exterior before entering. “We were able to get Jeremy to climb the outside of the house and enter a window. This window was replicated on the third floor set on the greenscreen stage,” says Couzens. “It was a seamless one shot combined from two camera moves, forest replacement, greenscreen window, CG room and set extension and some wire and rig removal. The lens flare in the shot is in fact real – our takeover is as Aaron goes through the window.”
DNeg previs’d the one-shot prior to filming and worked with the camera department to plan the move. “We had witness cameras around so we could track the camera and track Jeremy,” explains Ellis. “We built a rough piece of building in 3D so we could reverse engineer a new camera move going through the window. We showed it to the camera department and said, ‘Can you match that?’ They said, ‘Yep’. And off they went! We didn’t do motion control – they just eye-matched it and it worked really well.”
Still, careful 2D re-projections and reconciliation work was required to fuse the plates together. “It became clear that the best transition for Aaron would be when he jumps down onto the landing interior where a mini-morph on him from plate A to plate B would disguise the switch,” says Jopling. “For the building, however, that would remain the A plate until we entered completely through the window, thus avoiding awkward mismatching of window and frames. Once inside, the other windows were shot with greenscreen and were treated as similar to all the other Marta house interior shots. In addition to the invisible transition through the window there was also a huge wire and rig removal job on Aaron, as his climb was not only potentially dangerous but also physically impossible for all accept perhaps a parkour specialist!”
The fire is only initially suggested to the audience after Shearing
and Cross make their escape, rather than as a major reveal. “This is
something that happens quite a lot in the film where fairly big moments
are kind of thrown away,” notes Couzens. “There are a lot of moments
where we don’t treat the visual effects as the story point – they happen
a bit in background while our focus is on the FG actors. So when Cross
shoots the drone down and it crashes, what we are really interested in
is that he knows he’s shot it down so he’s wasting no more time and is
just packing his gun and moving on. So in the same way with the fire, we
didn’t want to have so many views of it – the first time the agent
drives up and sees the house on fire, he sees it from far away. And we
see it also just in the background behind the trees as they’re running
away. But we also do give the audience a couple of full views of the
house engulfed in flames.”
For the drive-up to the fire with the agent, DNeg had previs’d the shot to determine whether a large bluescreen could stand in for the structure (and help with roto down the track). “The bluescreen turned out to be too expensive and impractical,” recalls Ponti, “partly for the size of it and also having fire in front of it – we would have had to place it away from the flames at a safe distance. We also learned later that they planned on hiding all the technical equipment for the shoot behind the porch where the bluescreen would have gone. So our final idea was to strip out the greenery, all the bushes and leaves and sets of branches directly in front of the house, to make the job of the roto artists easier, and then we replenished that with CG trees later on.”
The Bronson location was also LIDAR’d and areas around it photographed for stitching together plates, while only the porch section of the house was constructed on the Staten Island location – actually a scout camp clearing surrounded by trees. Extra photo reference and panoramas were taken here to aid in the CG house extensions and fire shots. “We knew there would be a lot of interaction with the bottom story, standing on the porch area,” comments Ellis. “We were very keen for them to build that lower section as a grounding point. We knew we could set that on fire and get real flames and real light bleeding around the actors for that low level – the most effective, realistic look.”
For the fire itself, Double Negative relied on its own Squirt tool and Maya effects after referencing footage of real house fires for the sequence. Says Couzens: “We paid a lot of attention to smoke, interactivity with smoke and fire, and how smoke would billow out from the eaves. Every now and then something would fall – it was a really ‘less is more’ approach – not going crazy and showing off.”
“We had this great artist, Victor Wagner, our effects guy for the house and he built all these different sims,” adds Ponti. “We only had a few camera angles so that meant we could build a massive library of fire burning and smoke billowing with different levels of turbulence. Then the shots were lookdev’d in comp. We would render the house as a normal CG object, all the burning baked into the textures. The comp artists would then build the layouts for flame and smoke. We’d get that approved by the client and once we had that we’d go back in CG to do interactive lighting – especially towards the end we added fire inside the house burning which really added depth to it. And we also had some 2D elements tracked in. We had that thing you always hope for as a VFX artist when the director asks you to remove a certain bit of fire, and he’s actually asking to remove what was shot on the day!”
Cross and Shearing travel to Manila, the location of the pills
factory that provides the medication so important to maintaining Outcome
Agents’ enhancements. Here they continue with the meds ‘viral-off’
process for Cross. Later, the two are confronted by police around their
hotel room, who, along with a brainwashed supersoldier Larx-03 (Louis
Ozawa Changchien), pursue them over rooftops and then on motorcycle
through the Manila streets.
“The rooftop chase was filmed almost entirely practically with Jeremy and Louis running and jumping around on those roofs,” says Couzens. “A lot of our work there was taking out the safety lines, wirecam rigs and then all the safety and truss rigs on location. There were probably 50 wire and camera removals, carried out mostly by Level 256.”
The subsequent motorcycle chase, which begins as Cross and Shearing steal a bike, featured stunt car work filmed in the Philippines – including a stand-out shot of Renner ‘grinding’ down a set of stairs on a motorcycle stunt rig. Some further action was acquired in LA and on a greenscreen stage, with occasional shots of stunt riders supplemented with digital face replacement for the principal actors by Double Negative. DNeg also enhanced scenes with CG traffic.
For the face replacement shots, a four-camera ARRI Alexa setup was devised in which the principal actors were filmed adopting relevant facial poses on an upright chair with diffused polarized lighting in order reduce specularity so that DNeg could produce photoreal CG heads. Earlier, the stunt riders had been captured, where possible, with tracking dots on their faces. DNeg would later body track the stunties’ performances and track on the CG heads of the principal actors. “It was a lot of brute force tracking,” says Ellis, “but it really paid off in the end. Close face replacement is such a challenge – there’s something very deep in the human psyche which recognizes faces and if it’s a few pixels off, you’re screwed, it doesn’t work.”
To create a sense of the immense traffic in Manila, DNeg also
augmented the motorcycle chase with digital cars, vans and converted
jeep-like minibuses known as Jeepneys. “We photographed HDRs of cars and
absolutely everything in Manila,” explains Ellis, “and did lots of
LIDAR scans for ground surfaces of the road so we knew exactly where
everything was sitting, and to help us with tracking.” Again, DNeg’s new
physically-plausible shading pipeline came into play for the traffic
enhancements.
The chase ends when Larx-03 crashes his commandeered motorcycle into a warehouse pillar in the Navotas Fish-Port. Here, DNeg helped with set dressing of Fish-Port furniture in the area by referencing panoramas and HDRs of the area, before adding a CG Larx for the crash. “Tony Gilroy was pretty clear straight away that he wanted it to be a ‘background’ shot – it wasn’t a huge in your face moment,” says Ellis. “It was all about Aaron and Matar escaping on the bike. We talked about it being a dummy originally, although more for reference, but we always knew we’d have to shoot the bike clean.”
Stunt co-ordinator Dan Bradley rigged an empty bike on rails with pulleys which completed the flip into the pillar. Jeremy Renner and Rachel Weisz then performed their foreground action on a greenscreen bike setup, leaving DNeg to add the hurtling Larx-03 digi-double and comp the plates together. Changchien was present at the location in Larx-03 clothing and make-up where he was photographed sitting on the bike, standing in the lighting environment and laying on the ground in roughly the position he would end up in. DNeg also had the character’s clothes sent to them for scanning so that an accurate human model could be built.
For animation of the tumbling Larx-03, artists looked directly to reference from the Internet. “There is an unhealthy amount of bike and car crashes on YouTube,” admits Ellis. “People either kind of crumple to the ground in an undramatic way, or do crazy windmilling things. We ended up looking at some slower crashes and things that had more tumbling in them – people on skateboards or people falling over. What we wanted was the guy coming off the bike, being spun by the great force and trajectory of the crash and landing in a way that he was clearly dead. A lot of that fell to us doing keyframe of the final motion.”
As mentioned, Double Negative Singapore took a lead role in the drone
work for the film, and delivered numerous falling snow shots. Having an
equivalent studio set up in a different timezone has proved extremely
helpful for DNeg, but visual effects producer Tracey Leadbetter says the
managing of work is done almost as if the Singapore team are just in a
different room of the studio’s London building. “We run dailies, monitor
delivery schedules et cetera in the same way that we would in London,”
she says, “just at different times of the day due to the timezone. It’s
all about communication and planning. If we’ve got a clear plan in place
and communicated it to our teams, then things will run well wherever
they are.”
Leadbetter notes also that the type of invisible effects work in the film did not tend to affect the way DNeg planned their approach for a sequence or shot. “For example,” says Leadbetter, “there are sequences which have a more subjective content, such as the wolf, whereas others are invisible effects, such as face replacements. We tend to show a lot more of the work in progress for the heavily subjective content in order to set the foundations before layering on all the finishing detail, for example, showing the blocking animation, then muscle simulations later on. With the work that just has to be ‘correct’ we’ll often plan differently and show for comment more at a stage when the work is looking invisible without showing as much of the under the bonnet work; our client’s time is limited so we always want to get their feedback where it’s most appropriate.”
Rhythm & Hues also came on late in the production to complete several environments, including an early scene of Cross jumping across a chasm in Alaska. “The scene was shot in Canada and supervised by R&H’s John Heller,” says Couzens, “with the final shot combining plates, greenscreen stage and wire work and great matte paintings.”
Ultimately, Legacy was designed to combine real photography, practical stunts and both practical and digital effects to complement the previous Bourne films which strove for the same authentic aesthetic. “We were always going to attempt things practically if possible,” says Couzens of the style of effects in The Bourne Legacy. “And then, wherever we required some kind of effect, we’d go one step beyond photo realism to what I call ‘photo naturalism’. It meant that any effect, like the wolf or the burning house, had to be something that would be achievable on the planet today, and that fits the real-world philosophy of the Bourne franchise.”
Directed by Tony Gilroy, the film was shot on 35mm by Robert Elswit and ended up with about 800 shots, with the lion’s share completed by Double Negative under visual effects supervisor Mike Ellis. Additional work was handled by Level 256, Rhythm & Hues, Phosphene, Somnyo Films and Lola VFX, with Legacy Effects contributing the animatronic wolf.
The drone
In the film, Aaron Cross (Jeremy Renner) plays a physically, chemically and mentally-enhanced black ops member of Operation Outcome. In Alaska for a training assignment, he happens across an exiled agent, Number Three (Oscar Isaac) at a remote cabin. But with previous black ops projects exposed by Jason Bourne, it is determined the Outcome ‘assets’ are to be eliminated – via a Predator drone sent to destroy the cabin and wipe out Cross and Number Three.
The first drone takes off – original plate.
Final shot.
A different drone used to deliver blood samples and supplies is first seen taking off from a snow-covered field, achieved with a full sized art department model and then a digital takeover. Cross later flees the cabin as a missile streaks past him and destroys it, killing Number Three. Managing to shoot down the aircraft with a sniper rifle, Cross then draws another drone to strike at a wolf (see below for how that was accomplished), now carrying his tracking device as a diversion.
Double Negative approached the drone shots early in production by heavily previs’ing the sequence and referring ideas to the director and to Couzens for practical aerial plates to be filmed near Calgary – also relying on Google Earth data to scout mountains. “We did a lot of previs to help tell the pilots what we needed and what would work in the cut,” says Mike Ellis. “The editors cut our previs into their scenes, and would test block-outs. Once we’d got that previs nailed, there was a direction and a road-plan for how to get there.” Most of the final drone work was completed by the studio’s Singapore office.
A missile flies past Aaron – original plate.
Final shot.
For the shot of the drone missile destroying the cabin, DNeg combined plates of Renner running with a whip pan to the cabin before it explodes. SFX supervisor Garry Elmendorf rigged the structure as a full-sized explosion, enhanced only with the CG drone swooping past, the missile trail, CG snow and just a few frames of Number Three in the doorway. DNeg also added the antenna array and storage box exploding to the side of the cabin.
“The attack is a bridge of two plates and a panorama shot on location and a face replacement,” explains DNeg CG supervisor Pietro Ponti. “When Jeremy ran out of the cabin, he pretended to see the missile and turned to see it hit the cabin. The camera then whips through 90 degrees. Before the motion blur stops in the plate, we pick up with a panorama shot on location and we continue the swap and then land exactly at the location of the house as it explodes – which was the practical explosion. To bridge the three plates we brought LIDAR scans of the environment and cabin and the panorama into Nuke and re-aligned them. Then we created the missile from the drone as a CG render with CG smoke trailing behind it.”
Original plate.
Final shot.
For aerial views showing the drone returning to hunt Cross, and for monitor screens seen in a control room, the production chose to acquire helicopter plates. “We started with the previs as an aid for the director, editor and a guide to the chopper pilot and aerial DOP,” explains Couzens. “And up in the air you get happy accidents. That gave us a library of back plates but also enabled us to build some camera moves into the plates during the shooting stage. We went for realistic camera positions and movement, sticking to how fast a real drone would fly compared to a camera helicopter too.”
Armed with the background plates, DNeg set to work augmenting them to match the previs and required drone movement and to add in realistic layers of snow haze, as the original plates were shot in the clear, dry Calgary air. “The final approach,” outlines Ponti, “was to take the plate and get our matchmove department to work out the shapes of the mountains and create very simple geometry for the mountains that we would pass on to comp with depth passes rendered in RenderMan. With that as a guide they could create a hazing effect. The good thing was that the trees were so green and the ground so white they were able to luma key the trees and do a treatment to make them full of snow even if they weren’t.”
Original plate.
Final shot.
A snow simulation effort for the drone scenes, and for the wolf shots and for Cross in the mountains, was orchestrated by effects supervisor Viktor Rietveld. “He actually modeled crystals of snow and then wrote a little bit of software to clump them together in actual flakes,” says Ellis. “What you’re seeing are actual real crystals clumped together, with real dynamics of wind and turbulence added in. And when you see falling snow in a forest or around trees, it’s very difficult to place it in the right depth, otherwise it just looks like it’s falling over everything. So we used LIDAR for that, particularly in the wolf scene for the ground and backgrounds. We had all these trees in LIDAR and used that as holdouts.”
The monitors in the control room featured both live playback and bluescreen inserts (comp’d by Level 256) showing the drone’s view. Live playback graphics were created by Jasen ‘Jaz’ Nannini of Somnyo Films. “One of the story points was the snowstorm reducing the pilot’s vision,” says Couzens. “So where we used live action helicopter plates we had to treat them to look like the rest of the snowy drone command shots created by Jaz.”
The wolf
Original plate.
Final shot.
“We had real wolves and we were going to do as much as we could with them,” recalls Couzens. “But we found early on that what we needed was a furtive, sly, sneaky wolf. Wolves are very timid creatures, so they’re hard to train in one sense. Also, they’ll always approach another animal or person very cautiously. So what we found was that we couldn’t get our trained wolves to do anything slowly. We could get them to stand and growl and get them to come forward and move really fast, but we couldn’t get that middle ground of them coming forward slowly.”
Close-up shots on the wolf standing were mostly the real wolf. On set, too, hybrid wolves (half-wolf, half-dog) were filmed, but used only as animation reference thanks to multiple witness cameras. Shots of the animal approaching slowly, lunging or breaking away were generally the full CG DNeg creatures, “but we would start with the first five frames of the real wolf for real-world line-up and lighting then do a take-over to full CG,” says Couzens.
In the sequence, Cross traps the wolf; as it gets thrown up, a CG wolf is seen, with the suspended version a Legacy Effects animatronic. Shots of Cross wrestling with the wolf were filmed with Renner interacting with a soft animatronic. “That gave us the interaction between hands and fur,” adds Couzens. “We then replaced the head and teeth and eyes, and legs where necessary. There are 15 CG wolf shots and 12 enhancements to the animatronic or real wolf. So it ended up being fairly substantial, but the shots are fairly short and sharp.”
Another view of the wolf.
DNeg then used this data to build a CG version in Maya, grooming with Shave and a Haircut and the studio’s proprietary fur tool. A promising test was presented to the director while the sequence was being filmed. Animation supervisor Stephen Enticott was also on-hand in Canada to see the actual wolf in action. “In our favor was that we had the real wolves on set and the real environment,” says Ellis, “so we knew exactly what the wolves looked like. They weren’t necessarily doing the right thing but they were in the right place, in the right lighting. We could see all the right nuances, the twitching, the hair blowing.”
Original plate with real wolf.
Final shot.
Ponti took HDRs of the location, as well as gray ball and color chart reference to suit DNeg’s newly adopted physically-plausible and image-based lighting approach to rendering – although the wolf also used the studio’s existing lighting techniques. “For the fur,” says Ponti, “we didn’t have indirect diffuse or indirect specular, we just cheated an area light to get bounce from the snow. But we really wanted to control the difference between root and tip on the fur – you get a lot of difference between the colors around the coat. In the end we did add some cinematography-like lighting, such as a rim light or what not to enhance the fur, but it was very subtle.”
Snow simulations (described above) were an important part of integrating the wolf into the scene, too. 2D supervisor Peter Jopling oversaw the snow interaction on the wolf, taking two different approaches depending on whether the animal was live action or CG. “Where we wanted snow to land on a static or nearly static wolf we found we could track a point of fur and then stick individual flakes,” says Jopling. “The same applied to Aaron. It was amazing how few flakes were required in order to sell the idea of being integrated, sometimes only one or two.”
“For the CG wolf,” continues Jopling, “we had more options. We could use the p world pass in Nuke to select a very specific area of the wolf, down to a few pixels and then stick a snow flake to him, regardless of his movement. We also had a full bells and whistles approach for a few hero shots where we would create custom sims to create full fx snow renders that would fully interact will Aaron and wolf which would require a full body track of the characters in order to create proper hold outs for the snow.”
Marta’s house
Original plate with partially built house.
Final shot.
The reference location, Oliver Bronson House in Hudson, New York, was featured in one shot of Cross climbing parkour-style on the exterior before entering. “We were able to get Jeremy to climb the outside of the house and enter a window. This window was replicated on the third floor set on the greenscreen stage,” says Couzens. “It was a seamless one shot combined from two camera moves, forest replacement, greenscreen window, CG room and set extension and some wire and rig removal. The lens flare in the shot is in fact real – our takeover is as Aaron goes through the window.”
DNeg previs’d the one-shot prior to filming and worked with the camera department to plan the move. “We had witness cameras around so we could track the camera and track Jeremy,” explains Ellis. “We built a rough piece of building in 3D so we could reverse engineer a new camera move going through the window. We showed it to the camera department and said, ‘Can you match that?’ They said, ‘Yep’. And off they went! We didn’t do motion control – they just eye-matched it and it worked really well.”
Still, careful 2D re-projections and reconciliation work was required to fuse the plates together. “It became clear that the best transition for Aaron would be when he jumps down onto the landing interior where a mini-morph on him from plate A to plate B would disguise the switch,” says Jopling. “For the building, however, that would remain the A plate until we entered completely through the window, thus avoiding awkward mismatching of window and frames. Once inside, the other windows were shot with greenscreen and were treated as similar to all the other Marta house interior shots. In addition to the invisible transition through the window there was also a huge wire and rig removal job on Aaron, as his climb was not only potentially dangerous but also physically impossible for all accept perhaps a parkour specialist!”
The agent finds the house on fire – original plate.
Final shot.
For the drive-up to the fire with the agent, DNeg had previs’d the shot to determine whether a large bluescreen could stand in for the structure (and help with roto down the track). “The bluescreen turned out to be too expensive and impractical,” recalls Ponti, “partly for the size of it and also having fire in front of it – we would have had to place it away from the flames at a safe distance. We also learned later that they planned on hiding all the technical equipment for the shoot behind the porch where the bluescreen would have gone. So our final idea was to strip out the greenery, all the bushes and leaves and sets of branches directly in front of the house, to make the job of the roto artists easier, and then we replenished that with CG trees later on.”
The Bronson location was also LIDAR’d and areas around it photographed for stitching together plates, while only the porch section of the house was constructed on the Staten Island location – actually a scout camp clearing surrounded by trees. Extra photo reference and panoramas were taken here to aid in the CG house extensions and fire shots. “We knew there would be a lot of interaction with the bottom story, standing on the porch area,” comments Ellis. “We were very keen for them to build that lower section as a grounding point. We knew we could set that on fire and get real flames and real light bleeding around the actors for that low level – the most effective, realistic look.”
Original plate of house on fire.
Final shot.
For the fire itself, Double Negative relied on its own Squirt tool and Maya effects after referencing footage of real house fires for the sequence. Says Couzens: “We paid a lot of attention to smoke, interactivity with smoke and fire, and how smoke would billow out from the eaves. Every now and then something would fall – it was a really ‘less is more’ approach – not going crazy and showing off.”
“We had this great artist, Victor Wagner, our effects guy for the house and he built all these different sims,” adds Ponti. “We only had a few camera angles so that meant we could build a massive library of fire burning and smoke billowing with different levels of turbulence. Then the shots were lookdev’d in comp. We would render the house as a normal CG object, all the burning baked into the textures. The comp artists would then build the layouts for flame and smoke. We’d get that approved by the client and once we had that we’d go back in CG to do interactive lighting – especially towards the end we added fire inside the house burning which really added depth to it. And we also had some 2D elements tracked in. We had that thing you always hope for as a VFX artist when the director asks you to remove a certain bit of fire, and he’s actually asking to remove what was shot on the day!”
The Manilla chase
Level 256 performed a number of rig removals for the film.
Final shot.
“The rooftop chase was filmed almost entirely practically with Jeremy and Louis running and jumping around on those roofs,” says Couzens. “A lot of our work there was taking out the safety lines, wirecam rigs and then all the safety and truss rigs on location. There were probably 50 wire and camera removals, carried out mostly by Level 256.”
The subsequent motorcycle chase, which begins as Cross and Shearing steal a bike, featured stunt car work filmed in the Philippines – including a stand-out shot of Renner ‘grinding’ down a set of stairs on a motorcycle stunt rig. Some further action was acquired in LA and on a greenscreen stage, with occasional shots of stunt riders supplemented with digital face replacement for the principal actors by Double Negative. DNeg also enhanced scenes with CG traffic.
For the face replacement shots, a four-camera ARRI Alexa setup was devised in which the principal actors were filmed adopting relevant facial poses on an upright chair with diffused polarized lighting in order reduce specularity so that DNeg could produce photoreal CG heads. Earlier, the stunt riders had been captured, where possible, with tracking dots on their faces. DNeg would later body track the stunties’ performances and track on the CG heads of the principal actors. “It was a lot of brute force tracking,” says Ellis, “but it really paid off in the end. Close face replacement is such a challenge – there’s something very deep in the human psyche which recognizes faces and if it’s a few pixels off, you’re screwed, it doesn’t work.”
A breakdown of a Dneg traffic shot. Clockwise from top left: bluescreen car plate, background 1, background 2, final shot.
The chase ends when Larx-03 crashes his commandeered motorcycle into a warehouse pillar in the Navotas Fish-Port. Here, DNeg helped with set dressing of Fish-Port furniture in the area by referencing panoramas and HDRs of the area, before adding a CG Larx for the crash. “Tony Gilroy was pretty clear straight away that he wanted it to be a ‘background’ shot – it wasn’t a huge in your face moment,” says Ellis. “It was all about Aaron and Matar escaping on the bike. We talked about it being a dummy originally, although more for reference, but we always knew we’d have to shoot the bike clean.”
Motorcycle flip – background with motorcycle on rails.
Foreground plate.
Final shot.
Stunt co-ordinator Dan Bradley rigged an empty bike on rails with pulleys which completed the flip into the pillar. Jeremy Renner and Rachel Weisz then performed their foreground action on a greenscreen bike setup, leaving DNeg to add the hurtling Larx-03 digi-double and comp the plates together. Changchien was present at the location in Larx-03 clothing and make-up where he was photographed sitting on the bike, standing in the lighting environment and laying on the ground in roughly the position he would end up in. DNeg also had the character’s clothes sent to them for scanning so that an accurate human model could be built.
For animation of the tumbling Larx-03, artists looked directly to reference from the Internet. “There is an unhealthy amount of bike and car crashes on YouTube,” admits Ellis. “People either kind of crumple to the ground in an undramatic way, or do crazy windmilling things. We ended up looking at some slower crashes and things that had more tumbling in them – people on skateboards or people falling over. What we wanted was the guy coming off the bike, being spun by the great force and trajectory of the crash and landing in a way that he was clearly dead. A lot of that fell to us doing keyframe of the final motion.”
Invisible effects across the globe
Original plate.
Final shot by Dneg.
Leadbetter notes also that the type of invisible effects work in the film did not tend to affect the way DNeg planned their approach for a sequence or shot. “For example,” says Leadbetter, “there are sequences which have a more subjective content, such as the wolf, whereas others are invisible effects, such as face replacements. We tend to show a lot more of the work in progress for the heavily subjective content in order to set the foundations before layering on all the finishing detail, for example, showing the blocking animation, then muscle simulations later on. With the work that just has to be ‘correct’ we’ll often plan differently and show for comment more at a stage when the work is looking invisible without showing as much of the under the bonnet work; our client’s time is limited so we always want to get their feedback where it’s most appropriate.”
Rhythm & Hues also came on late in the production to complete several environments, including an early scene of Cross jumping across a chasm in Alaska. “The scene was shot in Canada and supervised by R&H’s John Heller,” says Couzens, “with the final shot combining plates, greenscreen stage and wire work and great matte paintings.”
Ultimately, Legacy was designed to combine real photography, practical stunts and both practical and digital effects to complement the previous Bourne films which strove for the same authentic aesthetic. “We were always going to attempt things practically if possible,” says Couzens of the style of effects in The Bourne Legacy. “And then, wherever we required some kind of effect, we’d go one step beyond photo realism to what I call ‘photo naturalism’. It meant that any effect, like the wolf or the burning house, had to be something that would be achievable on the planet today, and that fits the real-world philosophy of the Bourne franchise.”