Jack climbs the mighty beanstalk in Jack the Giant Killer. All images courtesy of Warner Bros. Pictures.
Director Bryan Singer not only got more kid-friendly with Jack the Giant Slayer (riffing on The Princess Bride and Clash of the Titans), but he also fully embraced virtual production and 3-D for the first time. Indeed, he played in the volume during the mocap sessions right along with the actors: choreographing the action and making selects right on through principal photography using the Simul-Cam system for integrating live action and virtual components.
VFX vet Hoyt Yeatman (The Abyss) oversaw Jack as the visual effects production supervisor. He and the filmmakers worked with Digital Domain and Giant Studios, which collaborated on the virtual production process. Meanwhile, The Third Floor provided previs of action sequences and MPC created the crucial CG beanstalk, sharing assets with DD when necessary.
"When we started three years ago, we talked about Avatar-like characters but this would be in real environments and Bryan wanted athletic and smart giants," Yeatman explains. "When the idea of Simul-Cam was first proposed it was an optical tracking system, but we shot Jack on location and very often in historical locales such as a cathedral that had restrictions for how you could shoot inside. We ended up shaft encoding, which came from the old days of motion control, and that worked really well and since the 3-D camera rigs [the movie was shot natively on Red Epic cameras] almost always lived on the cranes. It was a perfect platform to work on, so the guys at Giant Studios wrote some code and built a virtual model of the cranes accurately used for geometry reasons. So we were able to Simul-Cam anytime we wanted into those scenes and we brought Giant Studios out to the UK and built a mocap stage at Shepperton Studios."
Fye
It was critical for the filmmakers to have an accurate understanding of the 20-to-24-foot-tall CG giants' performances, position and timing while shooting scenes where giants drove the primary action and interacted with live actors; as well as for all-CG or mostly-CG sequences featuring giants. As Fallon, the two-headed giant, was performed by two actors simultaneously, Bill Nighy and John Kassir, the use of virtual production was particularly important in that it enabled both of them to pre-visualize the final product and base their performances around this unique character trait.
In a mocap shoot prior to principal photography, sequences driven by giants were choreographed and directed by Singer; Giant Studios capturing all body performance and Digital Domain’s virtual production team capturing all facial performance (using four mounted face cameras), simultaneously as a single, cohesive performance, in real-time in 3-D.
Fumm (center)
Digital Domain’s virtual production team then created "Kabuki": projecting video from its face cameras onto the giants’ characters so Singer and editorial could view the actors’ performance quickly and cost-effectively within the real time version of the performance captured. Digital Domain delivered the Kabuki asset to Giant Studios where it was integrated it into the real-time scene, which could then be viewed in MotionBuilder or played out as QuickTime files for editorial. This process enabled Singer and the editorial team to make performance selections from all options gathered via real-time render of the body performance on set plus witness camera material from the face cameras. To ensure all motions maintain the weight & scale of giants, most motions were slowed down by 15-20%.
Those cohesive giant performances were then used for on-set references during principal photography where they were viewed by Singer, cinematographer Tom Sigel and VFX supervisors through the Simul-Cam or viewed as QuickTime references, so they could see the giants’ performance, scaled correctly (at a factor of four) alongside live actors, within the CG environment. This additionally created great time and cost efficiencies in the post-visualization stage as Digital Domain was able to respond quickly to directorial and editorial changes in giants’ position, timing, performance and eye lines.
Fallon
"We had to raise the bar for virtual production for these actors to become these characters," remarks Stephen Rosenbaum, who came over to Digital Domain from Weta Digital after working on the Oscar-winning Avatar. "And that was huge because when you see Fallon delivering dialogue or emoting, you pick up Bill Nighy's expressions and mannerisms that come through in the character. And it gave the editors a lot to work with as well early on in the production. They didn't have to wait months for us to develop the blocked animation and then get down into dialogue later and ultimately everyone wondering whether or not the performance was going to come through."
The challenge of the giants from a design perspective was that Singer wanted them to look unique as well as individually distinctive -- and not only the eight hero characters. For example, since the giants were sprung from the landscape, their faces resembled rock formations with boils and other blemishes on their skin. At the same time, Fee, Fye, Foe and Fumm were different from one another. Fee was tall and strong with stringy hair; Fye was tall and bald and triangular; Foe was short, rotund and bald.
But Fallon, of course, proved the most daunting and time consuming character. During script development Singer presented the idea of a two-headed giant, which he got from looking at Jack the Giant Killer illustrations and recalling the interactive humor from How to Get Ahead in Advertising. "Design wise, it proved to be a significant challenge," Rosenbaum continues. "You can't just stick another head on his shoulder. So we went through numerous iterations and came up with a kind of cystic growth but with his own personality. In terms of virtual production, you see Bill Nighy and John Kassir as his grunting side-kick coming together to create this character."
The Cook
According to DD's animation director, Jan Philip Cramer, the action-packed kitchen sequence in which Jack (Nicholas Hoult), Isabelle (Eleanor Tomlinson) and Elmont (Ewan McGregor) interact with the Cook giant (Philip Philmar) was the most complex. "It involved all the players: Digital Domain, The Third Floor, Giant Studios and our in-house facial capture for Kabuki, and it all came together," Cramer says. "They have different eye lines and different actions throughout the sequence. And it shows up the scale really nice. We planned it well and communicated well between the departments to make sure all the right assets were there. Once we finished mocap, we were able to cut it up and find good angles for principal and they were matched tightly with the previs and with the performance from the actors."
But things changed in production. Isabelle became a virtual character in the cage in the kitchen and rather than having to go back and mocap that, they had already done the sequence in the volume and had her data and could cut to that.
MPC's digital beanstalk.
Meanwhile, MPC (under the supervision of Greg Butler) was responsible for the massive digital beanstalk. The production design and special effects teams created several 30-foot models. These were then animated, digitally enhanced and extended by MPC. DMP’s of the countryside below and digital cloudy skies above were later composited in.
The beanstalk was created using animated curves that had sections of interlocking beanstalk geometry pieces. This was then rendered in RenderMan, with the team relying heavily on ray tracing. Key framed digital doubles were used for the majority of the characters ascending the stalk.
Having created the beanstalk itself, the teams were then charged with its destruction. A combination of CG and practical elements were used to fill in the shots where support vines fly to the air and the earth explodes as roots are ripped out of the earth. For the wider shots showing the beanstalk falling across the landscape, procedural modeling and rendering techniques had to be developed.
The beanstalk's destruction was done with a combination of CG and practical elements.
But it's the evolution of virtual production that most excites Yeatman, who began with motion control. "Mocap offers new layers of directing and creativity -- it's more interactive. For me, it's getting back to using your imagination and that's the fun part of it."