Weta Digital has grown to now define state of the art in visual effects worldwide. Its dedication to storytelling and realizing director’s visions has lead to the company working with some of the best directors in the world. For The Hobbit: An Unexpected Journey, Weta Digital continued to employ its exquisite attention to detail into helping to tell the story of the fantastical world of Middle-earth.
To do that, the studio created numerous creatures, environments and relied on cutting edge-technologies that allowed actors to work together on sets even where the final characters would be different scales. Masters of transferring an actor’s intent into fully believable digital characters, the Weta Digital team welcomed back some old digital friends, and introduced us to a whole new host of creatures. They also completed the work for stereo 3D and inside the new paradigm of high frame rates (48fps).
To do that, the studio created numerous creatures, environments and relied on cutting edge-technologies that allowed actors to work together on sets even where the final characters would be different scales. Masters of transferring an actor’s intent into fully believable digital characters, the Weta Digital team welcomed back some old digital friends, and introduced us to a whole new host of creatures. They also completed the work for stereo 3D and inside the new paradigm of high frame rates (48fps).
Watch a behind scenes look at the VFX in The Hobbit: An Unexpected Journey.
Heading up Weta Digital is visual effects supervisor Joe Letteri. “We now have a much better understanding of what the causes and effects are of what we’re seeing on the screen,” says Letteri, reflecting on the growth of the studio since he came to Wellington to work on The Lord of the Rings. “We’re more able to really cut to the heart of the matter, whether it be the performance or creating a landscape or some piece of architecture or some clouds.” Letteri’s core effects team was made of visual effects supervisors Eric Saindon and Chris White and animation supervisor David Clayton.fxguide spent a week in Wellington talking to the filmmakers and artists behind An Unexpected Journey. In this article we take a look at just some of they key landscapes, creatures and incredible tricks Weta Digital used to bring Middle-earth to life, plus we explore the new cinematography challenges and the practical effects behind the film.
You can also check out our exclusive fxguidetv interview with visual effects supervisor Joe Letteri, and soon our audio fxpodcast with animation supervisor David Clayton.
Cinematography: stereo and 48fps
Director Peter Jackon and DOP Andrew Lesnie on the set of An Unexpected Journey.
Lesnie had to balance spectacle and wonder while being focused on what is important. “It’s always about story and performance. No matter how much preproduction work has been done, all drama scenes were approached in the normal way.”
As with his non-effects work, Lesnie always blocks and choreographs the scene with the cast before deciding or confirming the coverage. Ever the craftsman, Lesnie has this time captured Middle-earth in high frame rate digital stereo, a first for any major motion picture.
“We have a certain mandate to remain faithful to the look of Middle-earth, while acknowledging that Lord of the Rings was shot on film and now we are in a fully digital environment. We’ve been successful with softening the curve and giving the material a gentle, textural finish. Sometimes the digital capture feels a bit thin or sharp, so we take great pains to give the material some softness and body.”
Lighting interior and exterior
The sheer range of sets and environments on The Hobbit: An Unexpected Journey was incredible. Lesnie carefully crafted different color palettes and textures for each. From Bag End / Hobbiton which were rendered as he had in Lord of the Rings as “warm, romantic, inviting – the most ideal place to live in the world – innocent, natural, simple, breathtaking,” to Gollum’s cave, which is in fact a vast underground lake. It is “the calm in the middle of a storm. Its stillness only enhances the strange, eerie quality that exists in this cavern. It reeks of loneliness and the desperation of lost souls. Although it is below the Goblin world, I kept a cool, distant feel to this netherworld,” Lesnie adds.
Some locations will be familiar to audiences, as they were to Lesnie. “I always think of Rivendell and Lothlorien as the seats of higher learning. I kept this magic by portraying the realm always in dawn or dusk light. At night, Rivendell still has a magical glow. This mystical kingdom almost transcends reality.”
Jackson and Lesnie with the RED EPICs and 3Ality stereo rig.
Technical considerations
“We rated the RED EPICS at 800ISO. Shooting with mirror rigs loses 1 stop (400ISO), and shooting 48fps makes that 200ISO. Which meant that we were shooting at a lower image speed than Lord of the Rings over 10 years ago! In preproduction I tested every camera speed against every shutter angle and we determined that 270 degree shutter gave us the most comfortable viewing experience. That finally landed me at 250ISO for the project.”
“Stereo is another addition to the cinematographer’s palette,” adds Lesnie. “I always stage a scene with priority to the storytelling, always looking for ways to illuminate the subtext of a scene. If 3D can enhance that prospect, then it can be put into play. Like all techniques, it should support but not dominate the drama.”
To tell the story the script demanded, at the level the director needed, Lesine used dollies, sliders, steadicams, went handheld, “whatever it took…we even used tripods!” he jokes. Shooting primarily on Ultra Primes on 3Ality TS-5 stereo rigs, with Optimo short zooms on the TS2 rigs, Lesnie kept the camera fluid and the production moving swiftly while filming in advanced stereo. Lesnie extensively used Technocranes sporting Flight heads.
Although different characters were required to be rendered in different sizes, any method that would allow the whole cast to perform together was preferred. “Everything we learnt from Lord of the Rings we tried to apply to The Hobbit. Some tricks don’t work so well in 3D. However, some simple sleight-of-hand forced perspective tricks still work, as well as split screens, digital scaling in postproduction and our pet project, the on-set slave motion control system which allowed us to design some amazingly complex choreography.”
High Frame Rate
Lesnie and Jackson set up a shot.
“The Hobbit is my first 3D shoot, my first major digital shoot and at a higher frame rate. The learning curve has been enormous,” explains the award-winning DOP. “The issues unique to The Hobbit included shooting at 48fps, using slave motion control rigs and motion capture live on set. All these elements were developed and tested in preproduction.”
“In 2010 we’d already decided to shoot 3D at a higher frame rate,” says Lesnie. “So when we started prep, the digital camera that accommodated these requirements was the RED EPIC. Other shoots around the world were about to use the same technology, and all the DPs stayed in close contact throughout. The RED cameras and 3Ality 3D rigs were still a work in progress, with software and design upgrades coming in on a regular basis, covering everything from the cameras to rigs to wireless systems. Both RED and 3Ality put a great deal of effort into helping get the issues sorted. Our head of technology Dion Hartley and 3D camera supervisor Gareth Daley also tailormade a lot of additional hardware. Our infrastructure, put together by 2nd Unit DP Richard Bluck, had the personnel, equipment, engineering and post facility to investigate every new challenge.”
The camera department needed cameras to not only be synchronized at 48fps, but 72fps, 96fps and 120fps for slow motion work. “I wanted space for filtration in the 3D rigs,” notes Lesnie. “We wanted to reduce the weight and size of the 3Ality TS5 rigs for the steadicams.” The team were constantly making demands on the new technology to make it less of a presence. By pushing the technology, they could in fact make the set more focused on the performances, and not about the equipment. They pushed the requirements of the gear, especially in pre-production, so they could move beyond focusing on it, and not worry about it keeping up. The production used cutting edge technology for one reason only, to tell a better story.
Watch behind the scenes on the set of The Hobbit: An Unexpected Journey.
“I staged a series of tests in December 2010,” Lesnie says. “We staged full events in the two sets that were built; Bag End and Gollum’s Cave. We were testing the cameras, rigs, remote heads as well as the slave mocon rigs and on-set motion capture. We were looking for the most comfortable 3D viewing experience, so we tested every speed at every shutter angle, while Park Road Post worked to accommodate the massive influx of data. Everything from drama scenes to still lifes, to raiding the stunt department and staging swordfighting scenes in water to watch motion blur and strobing as well as getting a handle on the Red Epic’s dynamic range. We eventually settled on 48fps at 270 degree shutter as our standard shooting speed.”“I was keen to get things sorted before we started principal photography. I wanted to shoot 3D on a 2D schedule. The reality was we never completely locked off the tech development in these cameras and rigs. Day to day use educates you in ways that are irreplaceable.” With a breakthrough camera workflow designed for The Hobbit, the whole team had to deal with a monumental amount of data being shot in camera, checked, processed, backed up and made ready for post production. During production, Lesnie tried to make sure that the creative team viewed 3D dailies in a cinema at least three times a week.
“For the demands we had to devise new peripherals to achieve the shoot and to make the rigs work with the cameras – sync, timecode, video reference, control, making the systems wireless and managing the wireless spectrum to allow multiple units to run multiple rigs at the same time. Because we were planning on running so much wireless technology, transmitted picture delay became a real issue. Making rigs waterproof, sharing metadata to other departments, getting our physical lens changes down to about two to three minutes, getting stereo down to another couple of minutes, all of these issues we resolved,” Lesnie explains, allowing the director and the entire camera department to film for months on end in studio and in harsh exterior locations all over New Zealand.
Locations ranged from interiors to the wide expanses of New Zealand.
Although Lesnie had some very large sets to light, sometimes the smallest sets become the most ambitious.
Bilbo’s story begins at Bag End when the Hobbit’s peaceful life-style is shattered by thirteen rambunctious Dwarves and a Wizard he barely knows. “What’s great about Bag End is you meet every pivotal character that’s going on the quest,” he says. “What’s not so great is that they were all stuffed into the one set. But even if we’re jammed in a closet, I believe if it’s the shot you need to tell the story you just have to work out how to light it. Finding hidey-holes for small lighting units was a big deal in Bag End’s curved wooden tunnels. And even more challenging in the scaled down Bag End set. That one was a real headbanger. The other big help was a computerized lighting log, kept by a dedicated crew member on each unit. All the information was uploaded to a secure website, so both units had access to everything. Sometimes we were doing pickups to a scene we’d shot 15 months before!”
A new paradigm for visual effects
Shooting in stereo and at 48fps presented new challenges for Weta Digital, including for animating characters such as Gollum.
However, the extra information meant that Weta Digital could work with the benefit of higher resolution, even in roto and paint. “It was a lot more work but the great thing about 48fps was there was half the motion blur,” notes Saindon. “And the big hard thing to do with roto and paint is painting back motion blur. Because it’s lowered the amount a little bit it’s actually made the job easier.”
“The great thing about the 48fps on the animation side was that animators no longer had 24 frames to do a move,” adds Saindon. “They had 48 which means they can do much smoother moves. They can do much better animation so the subtleties of animation are brought through the 48, whereas the 24 we didn’t have quite the ability to do that. It sounds like a very little amount but it’s amazing the subtleties you can add by just having a few extra frames there.”
The stereo nature of the production also meant that miniatures were not part of the visual effects mix, with the worlds of Middle-earth instead realized using Weta Digital’s finely detailed CG modeling techniques. “When your shooting miniatures in two dimensions as we did on Rings,” says Joe Letteri, “if you make a change to the camera you can usually cut and paste the miniature elements around it and still make them fit the camera. That doesn’t work in stereo. So for reasons of preserving flexibility, we wanted to make everything digitally – that way we can make everything be open right until the very last minute.”
On-set surveying
A dedicated crew of artists oversaw the intense visual effects on-set surveying and data recording for The Hobbit trilogy’s 18 month shoot. It is a crucial aspect of the visual effects work – tying in the three dimensional world with the two dimensional (albeit) stereo film plates, and helping to enable CG characters and environments to be added into the scenes with real-world lighting. “We really went over the top and scanned every single set that it was shot on, which was absolutely beneficial,” says Eric Saindon. “We had 3D scans of everything – we knew the placement of everything when we got it back in the 3D world. Because everything’s in stereo and at 48fps we needed a lot more detail.”
What’s inside Weta Digital’s on-set VFX kit bag
Theodolite – measures three dimensional points in space and makes a cartesian plan of the environments.
LIDAR scanner – a Lieca ScanStation C10 capable of scanning 50 million points in 30 minutes.
Cameras – Canon 5Ds, plus a 1DS Mark III for HDRs since it provides a higher bracketing range. Also a variety of kit lenses from 8mm to 200mm.
HDR tools – pano head from Really Right Stuff, plus mirror and chrome balls.
Tracking markers – orange and colored tape, different colored LED lights for darker situations, tent pegs, nylon
Measuring tape – measuring camera heights and plotting out three dimensional scenarios
Hardware – laptops, desktops and 3D monitors.
“A notebook is always handy too!” – Stan Alley
Senior on-set visual effects technical director Stan Alley was on second unit for most of the shoot and relied on several methods to acquire set measurements and camera data. LIDAR scanning became an important part of scanning both large external environments and sets and set-pieces (other smaller props were handled by the models department). “We generally don’t use the LIDAR while shooting – it takes half an hour to capture one full dome and that’s time we don’t have on set,” says Alley. “It’s also a very expensive piece of machinery that we don’t like to put in a chaotic environment. We generally do our LIDAR scanning at the end of the day, mornings or lunchtimes when there’s no-one around.”Theodolite – measures three dimensional points in space and makes a cartesian plan of the environments.
LIDAR scanner – a Lieca ScanStation C10 capable of scanning 50 million points in 30 minutes.
Cameras – Canon 5Ds, plus a 1DS Mark III for HDRs since it provides a higher bracketing range. Also a variety of kit lenses from 8mm to 200mm.
HDR tools – pano head from Really Right Stuff, plus mirror and chrome balls.
Tracking markers – orange and colored tape, different colored LED lights for darker situations, tent pegs, nylon
Measuring tape – measuring camera heights and plotting out three dimensional scenarios
Hardware – laptops, desktops and 3D monitors.
“A notebook is always handy too!” – Stan Alley
“One of the challenges with LIDAR scanners is that you get an enormous dataset,” adds Alley. “It’s 50 million points per scan and you might do ten scans per set. This is obviously too much information to load into 3D software, so we’ve had to develop tools to use that data. We bring in a low-res proxy mesh and from there you can select regions and then bring those into the application at high resolution and work from there.”
The sets were also extensively photographed – both as HDRs and as photographic reference – with images processed and color-corrected on the go. “They all have to be processed and go into the internal Weta side of things – called Shotsub – which is a way of accessing all files,” explains Alley. “We have to label them all with the correct date and slate, which goes into the database and makes them easily searchable.”
To deal with HDRs captured on greenscreen sets, where normally compositors might have to go in an key out the green, Alley says that on set they would take “quite a large bracket of about seven stops – 2 exposures apart – the lights will be fine pricks at one end and totally blown out at the other – somewhere in there they’ll be no green.”
For tracking markers, Weta Digital tends to use orange tape and then anything and everything to make tape stick in different conditions. “Sometimes you have to stick to the ground so we have super glue,” says Alley. “We also carry small creme brulee torches to dry out areas that are wet and stick things down that way.” LED lights are used instead for nighttime shots or scenes with deep focus (here the LED remains slightly bloomed out as a point of light that can be tracked).
Alley notes that one of his most challenging on-set visual effects requirements was for a scene in the film’s prologue, where a Dwarf is holding a special rock called the Arkenstone that is being beaten down by forges. “We had these enormous hammers that the art department had built,” he says. “The idea was that the actor would be able to hold this thing up here – but they realized it wouldn’t be possible. So we had to scramble and survey those hammers in position and the actor and all the other data before they moved on to the next setup, and the hammers would be gone forever from the 3D space. We did a quick survey of them right there rather than scanning, although we did track them down and got a LIDAR of them sitting on the back of a truck waiting to go to the prop graveyard.”
Practical effects
Tami Lane prepares an extra to be an Orc.
In fact, for each day of filming on An Unexpected Journey, Weta Workshop delivered “no less than 36 sets of facial prosthetics,” says creative director Richard Taylor, who oversaw work on designs for Dwarves, Orcs and so many other characters. One advancement was the widespread adoption of silicone facial appliances, previously used to replicate the qualities of human skin for Gimli in The Lord of the Rings, and now the chief material for many of the characters on this new film such as the 13 Dwarves.
“We’ve made extensive use of that on The Hobbit,” says Taylor, “working with an acrylic encapsulated silicone material that we’re impregnating with a flocking like material to replicate the corpuscles within the skin. So when the light penetrates and sub-surface scatters through the skin, a prosthetic that’s glued on will replicate the reflectivity and translucency of the skin.”
Weta Workshop’s Richard Taylor discusses the film.
The studio also devised new ways of constructing Hobbit feet appliances which had previously required foam latex to be glued onto actors every morning of the shoot – something that took many hours of intense labor. “This time we’ve gone for an articulated under-shoe that can be commercially bought, although we’ve adapted it with toe extensions, and then that fits inside a silicone slip-on sleeve that forms the foot,” says Taylor.Another advancement was the addition of robotic milling, laser cutting, digital modeling and 3D printing technology to manufacture intricate pieces of engraved costume armour and weaponry – from swords to axes, mattocks to knuckledusters.
Sizing up
On The Lord of the Rings, the filmmakers relied on ingenious forced perspective techniques to sell the scale differences between characters such as the Hobbits and Gandalf. But for An Unexpected Journey, the introduction of stereo filmmaking required a whole new approach: slave motion control.
Scale motion control was used to make Gandalf play in scenes with the Dwarves.
The actors would rehearse the scene first, and then be given eyelines via small LED lights. They also wered wired up with production sound between sets. The ultimate result was that DOP Andrew Lesnie could move the camera around without it being locked down to a particular performance or framing. “[Andrew] was able to get something much more dynamic and working together,” notes Saindon. “In his camera we fed him both feeds so that he saw a composite while he was moving the camera so he knew exactly what he was going to get, even in a rough form.”
The right tool for the job
Just some of the tools utilized by Weta Digital on the film.
Tissue – a linear-elastic finite-element system to build muscles, skin, and fat that connects to Maya. Weta Digital used Tissue to add simulation and deformations on top of animation for so many of the creatures and digi-doubles.
Massive – well known for helping to realize the intelligent armies in The Lord of the Rings trilogy, Massive was once again put to use for battle scenes and for volumes of Goblins.
Gazebo – a hardware renderer used for quick turnaround times on animation, lookdev and lighting.
Synapse – used for to simulate fluids and other effects, Synapse enabled the high waterfalls seen throughout Rivendell.
Barbershop – hair grooming software which allows artists to literally comb, brush and blow dry CG hair. It was even modified to allow for stiffer bristles on the hedgehogs cared for by Radagast.
Plummage – a feather creation system used for the Great Eagles.
Lumberjack – an art-directable tree growing tool that takes into account how trees compete for light.
The slave motion control approach also worked hand-in-hand with computer controlled lighting that could match both the quality of light and the variation in scale. Weta Digital artists then easily combined Sir Ian McKellen as Gandalf into the sequence with the other actors. One particularly complicated shot following Gandalf between rooms and interacting with almost every Dwarf was 4,000 frames long.Just some of the tools utilized by Weta Digital on the film.
Tissue – a linear-elastic finite-element system to build muscles, skin, and fat that connects to Maya. Weta Digital used Tissue to add simulation and deformations on top of animation for so many of the creatures and digi-doubles.
Massive – well known for helping to realize the intelligent armies in The Lord of the Rings trilogy, Massive was once again put to use for battle scenes and for volumes of Goblins.
Gazebo – a hardware renderer used for quick turnaround times on animation, lookdev and lighting.
Synapse – used for to simulate fluids and other effects, Synapse enabled the high waterfalls seen throughout Rivendell.
Barbershop – hair grooming software which allows artists to literally comb, brush and blow dry CG hair. It was even modified to allow for stiffer bristles on the hedgehogs cared for by Radagast.
Plummage – a feather creation system used for the Great Eagles.
Lumberjack – an art-directable tree growing tool that takes into account how trees compete for light.
“It’s really a homage to back on the first LOTR shot when Gandalf comes through the door and hands off a hat,” says Saindon. “We shot it almost the very first week of shooting, two years ago now. Putting it altogether and roto to put Ian in the middle and in the proper 3D space it did take close to a year to put the shot together.”
When Wargs attack
In the film there are several encounters with Wargs, Middle-earth’s wolf-like beasts. For shots of the Wargs being ridden by Orcs, the filmmakers acquired plates of actors riding on green ‘barrels’ on a motion control rig – using the motion data for the final animation of their digital Warg. Digi double riders were also created where necessary. Warg hair was updated from the previous trilogy using Weta Digital’s proprietary tool called Barbershop.“We no longer model our hair as individual strands and have modelers go and individually move them around,” says Eric Saindon. “We have a set of tools that work a lot like on set. We essentially have a blow drier and combs and brushes that we actually groom our hair within a 3D system to basically groom the Warg in 3D space.”
For Warg animation, Weta Digital actually turned to motion capture reference of a real dog. “We thought about what type of dog would be best, maybe a big German Shepherd or something with a lot of mass to it,” recalls David Clayton. “It actually turned out to be most effective to capture Jake, who is the stage hand’s dog – quite small but he was more athletic and could jump over things. So we were able to capture this dog doing a variety of behaviors. In the scene you could slow it down and get rid of the wagging tail, and all of a sudden you’ve got a library of motions that just plug into your vision as real.”
Capturing the trolls
Troll actors in the capture volume.
The on-set performance.
Final shot.
1. Previs – before any filming began, a team of artists previsualized the encounter with the Trolls to work out major beats and help shoot the scene on motion capture and greenscreen stages.
2. Motion capture – actors Mark Hadlow, William Kircher and Peter Hambleton performed the trolls Bert, Tom and William in motion capture suits on a mocap volume, with a live feed of low-resolution computer generated trolls displayed on monitors as the scene was acted out.
3. Live action – the Dwarf actors and Martin Freeman filmed their part of the scene on a forest set, relying on balls on sticks for eyelines of the Trolls that would later be added to the scene digitally (sometimes along with digital doubles of the Dwarves and Bilbo). A live feed of the Trolls from the mocap shoot was fed into Andrew Lesnie’s camera.
4. Creature creation – the Trolls are modeled and skinned, while animators take the motion captured and re-target the motion to represent the four meter creatures which are obviously much larger than their human performers. Weta Digital’s Tissue system allowed for muscle, skin and fat to be simulated over the animation. At 48fps, animators also paid much closer attention to the blinks and lip sync.
5. Final touches – Compositors combine the live action actors, CG Trolls, the forest set, a digital camp fire, interactive lighting and extra effects to complete the final sequence.
We meet Gollum for the first time (again)
The original Gollum truly set the standard for a living, breathing character appearing on screen – the result of a close association between performance capture by Andy Serkis (who was also the film’s second unit director) and visual effects artistry from Weta Digital. In An Unexpected Journey, Serkis reprises his role as Gollum, with the digital artists ensuring that the original appeal of the tortured creature remained while they upgraded the way he looked and moved to once again define the state of the art in 2012.
Andy Serkis once again performed the role of Gollum.
The filmmakers leaned on tech developed since the first trilogy to allow Serkis to perform on location in a performance capture suit, much as the actor had done on Rise of the Planet of the Apes. The suit also incorporated a facial capture camera. Serkis told fxguide that playing a slightly younger Gollum was certainly a challenge.
“I had to basically unlearn all that had happened in Lord of the Rings,” says Serkis, “as that is all to come, apart from the beginning of the third Rings movie which is his descent into madness and lust and craving things as Deagol. Really that first sequence in Return of the King is his journey and then he spends half a millennium in the Misty Mountains. So what was interesting – where we slightly depart from the book is the notion of his bipolar psychology – his schizophrenic nature. One of the things Peter wanted to hold onto from Lord of the Rings was that Bilbo is having to deal with two personalities rather than just Gollum. So the Smeagol/Gollum conflict very much plays.”
Bilbo Baggins meets Gollum.
Animation-wise, Weta re-modeled Gollum to provide the same look and feel, but paid particularly close attention to his facial puppet controls following the Facial Action Coding System (FACS) used heavily on recent shows. “On someone like Gollum you’ve got to just go that bit further,” adds David Clayton. “You’ve got to study the wrinkles around his face. It’s really just studying Andy from an infinitesimal level – his skin slides over his cheekbones in a certain way when he pulls a certain expression, and the FACS solve’s not going to give you that out of the box, you need to layer in much more control.”A new challenge, of course, came in the higher frame rate and stereo nature of the film – which required more detail. “He probably had 10 times the amount of polygons this time around and at least three times as many facial shapes,” says Clayton. “There’s one shot that comes to mind where Gollum is getting very irate as he’s trying to think of an answer to one of the riddles. Andy went to town with it and had all these crazy facial expressions like he was really straining. It was kind of overwhelming to see it first, but when you break it down to individual muscles and movements and you really study them we were able to replicate that in Gollum and the final result is a really entertaining shot.”
Gollum in CG form.
Final shot.
For Serkis, the increase in fidelity compared to his original performance more than a decade ago was a clear leap forward in Gollum. “Gollum still stands out from the first trilogy,” says Serkis, “but the use of 48 frames per second creates a very interesting move forward, a great leap forward for digital characters and live action. There is something that happens to your brain watching 48 frames per second that makes you unable to question the fact that CG characters and live action characters don’t exist in the same time and space. They just look – it’s impossible to see that they can’t. That is a significant shift.”
And, of course, Gollum was just one of the numerous CG creatures appearing in An Unexpected Journey. Certain characters are discussed in this article, but there are others you may not even notice – the digi-doubles. Just about every race – Dwarf, Hobbit, Wizard, Orc and more – were replicated by Weta Digital either to perform dangerous stunts that actors could not achieve, or for traveling scenes and complex environments. This necessitated, as well, weapons and prop creations, hair sims and cloth simulations (some created in Marvelous Designer).
Return to Rivendell
Bilbo enters Rivendell.
The structures and landscape of Rivendell were, mostly, built as completely 3D environments, but occasionally some two and a half D methods could be used. “Two and a half D is sort of a myth in this movie,” notes Eric Saindon, however. “We do try to do two and a half D, but in a 3D movie is – it never really quite works. If we are going to use matte painting behind Rivendell for the mountains, then the matte painter has a scale reference and the correct proportions so when the matte painting is done it is painted to the right scale, so that in 3D you get the proper depth and everything feels like it’s in the right space. So although we do call it two and a half D, still, it’s very much 3D with matte painting for texturing.”
Final shot and original plate, showing the digital extension work for Rivendell.
Waterfalls were also important to Rivendell’s surrounds. For water sims, Weta relied on its Synapse toolset to create high falls and water flowing throughout the city. “You get all the detail of the waterfall falling, that sense of haze being illuminated,” says White. “They would feed into fluid sims of water, pooling of water and breaking across the rocks. We want to make sure there’s foam across the rocks – all those little details to get that sense of water rushing through.”
Watch out for that boulder: the Stone-giants
The Stone-giants – render.
Final shot.
“They had to be rigid structures when they collide. We we could have them clash and break apart and send out showers of rock, that would help sell scale too because you would see these rocks exploding out falling with gravity quite slowly in the scene and you’d be much more aware of the scale of the creature.”
Weta relied on proprietary rigid-body sim software and a cracking tool for shots of boulders breaking and the Stone-giants clashing. The stormy sequence also relied on rain and fog simulations and dramatic lightning developed using a tool called Tesla that controlled bolts and allowed them to hit with dramatic shapes and detail.
The scary world of Goblin Town
A nasty Goblin.
Here, the Company of Dwarves, Bilbo and Gandalf are manhandled by Goblins – mostly performance captured CG creations by Weta Digital. “When they’re up close you had to make sure the CG characters held up when they’re grabbing and tackling with the Dwarves and the live action plates,” says Chris White. “Then the wider plates are a combination of Massive and animated goblins, rendering sometimes hundreds of them. And then medium shots are a combination of both.”
“We needed them to feel like a horrible pack of rabid rats on steroids but with a horrible pack mentality that without any sort of synergy – they all hate each other,” adds David Clayton. “Terry Notary had done some great work with choreographing a movement and range of motion and ideas for how they should behave. We were taking that and applying it to big packs of goblins. By the end of it we just had goblins scrambling on goblins, kind of mosh-pit of goblins just harassing these dwarves.”
Watch a scene from the Goblin caverns.
Weta Digital spent months crafting the various props inside the caverns – wheels, buckets, weapons, skeletons – plus the diseased feel of the rocky interior. The look for that came from an interesting source. “There was a rock outside of Peter’s house that he liked that had a good quality,” recalls White. “So we went and scanned that rock – we took textures from it and carried through the idea that’s there a disease throughout this whole entire environment and it shows through the architecture and rocks.”The complicated sequence which contained many live action and digital elements was composited in Nuke. “We had to work with deep compositing to make sure the set merged into the CG and extended up,” says White. “Also getting the right level of haze and fog and all the different layers of hundreds of Goblins and torches and structures – it built up quite a bit of layers and that all needs to be seamless.”
Performance capturing the Goblin King
Barry Humphries provided facial performance capture for the Goblin King.
The Goblin King takes shape.
Final shot.
“Sometimes we needed to take more of Barry’s head nuances and some body shudders that would punctuate dialogue and we would merge them with Terry’s performance of the body,” continues Clayton. “The other thing we needed to do with Terry’s body was just slow it down a little bit because he’s quite fast moving whereas of course the King is this humongous lumbering fat guy.”
The studio looked to previous experience learnt on Gollum, Rise of the Planet of the Apes and numerous other characters to accurately portray the Goblin King’s eyes. “In particular,” says Chris White, “from our lighting department we quickly learned how to light the eyes and to light to a character’s expression and we took that to the lighting of The Goblin King, making sure the highlight in the eye was just right, making sure the expression falls across his face correctly – all those little nuances.”
In particular, animators ensured that the Goblin King’s eyes were constantly re-targeting onto actual things in the scene, just as real eyes do. And since the film was shot at 48 frames per second, animators had twice the usual number of frames to work with, but found this actually gave them greater opportunity for lip sync and eye movement detail.