ILM raises the character animation bar with Pirates of the Caribbean: Dead Man’s Chest, and Bill Desowitz gets an overview from John Knoll and Hal Hickel.
By Bill Desowitz
[ Posted on July 14, 2006 ]
 |
 |
| With the help of the Imocap system, Bill Nighy’s creepy Davy Jones is the next great CG performance after Gollum and King Kong. All images © 2006 Disney Enterprises Inc and Jerry Bruckheimer, Inc. Photo credit: ILM. |
|
As most of you have seen by now, the results of the character animation are very impressive. They’ve already begun talking about the creepy Davy Jones as the next great CG peformance beyond Weta’s Gollum and King Kong. Lord of the Deep and commander of the mysterious Flying Dutchman ghost ship, Jones is a delicious mutation: part human and part squid, with a beard full of wiggly tentacles, and crab-like claws.
Unable to rely on traditional MoCap or hand animation, ILM created an innovative new system called Imocap that allowed onset and on location motion capture to elicit the most believable look and performance possible out of actor Bill Nighy.
“The characters required a lot of careful examination of human performances and then trying to combine that with the animation,” explains animation supervisor Hal Hickel. “We knew that there were going to be actors cast to play Davy Jones and his crew, and that those actors would be on set in the plates that we were going to be put those CG characters into and that somehow we had to extract the motion of the performances without having to reshoot later. We didn’t want to bring the mocap stage onto the set. So the R&D and MoCap groups came up with a solution: special [sensor-studded] suits that would be worn by Bill Nighy and other actors playing his crew. We would take reference cameras onto the sets and untethered cameras out on location with lightweight tripods and position them at angles off of what the main taking camera was seeing. This allowed us to track the movements and provided great data from the hero plates with the actors in them, casting their real shadows and making good eye contact with the live actors, and then we were able to extract their motion and apply it to our CG characters and put those characters right on top of the actors. There’s still a lot of animation artistry in there because there’s a lot of interpretation. This is just about getting the skeletal motion of the character; we still did all of the facial animation by hand [in Zeno].”
 |
|
| A second R&D project at ILM involved creating Davy Jones’ tentacle beard itself. Many tests were done to get the behaviors right. |
|
Zeno added an additional challenge. ILM came into Dead Man’s Chest with only a small portion of its creature pipeline function intact. War of the Worlds and The Island, the two previous projects done with Zeno, were primarily hard surface works. The creature work on those didn’t need cloth, sim, flesh or hair. A large part of the effort was re-enabling the pipeline, particularly the facial animation.
Meanwhile, the second R&D project involved the tentacle beard itself. “Our R&D folks worked with James Tooley, our sim guru [creature development supervisor], and Karin Derlich [creature technical director], who came up with behaviors,” Hickel adds. “And we’d do tests and I’d say, ‘This one is too tentacly and this one feels too much like an elephant’s trunk and this one feels too much like a snake.’ We would look at tons of octopus references. After we got it, then those behaviors were added to the solver through what we call ‘Joint Motors,’ so all the tentacles were divided into little joint segments and each segment was essentially a little motor that was directed to move this way or that way. So those joint motor impulses were sent out at the same time the tentacles were receiving force information: I need to swing this way, I need to swing that way… and so it would all happen together.
“Once we added those behaviors to our sim engine, the last thing we needed was something called ‘Sticktion,’ which is a combination of friction and stickiness. The problem was that without Sticktion, the tentacles would just slide onto each other. We really wanted them to be this heap of viscous tubes that would stick to each other and stick to his chest. And the ones at the bottom of the stack would stick there in a big matte. The biggest ones out in front that hang from his chin and moustache-like tentacles could really swing around.”
The crew of The Flying Dutchmen features a cast of characters with visual references to the ocean and its creatures, including coral, sea sponges, barnacles, mussels, hammerheads and puffer fish.
“Imagine a piece of spaghetti sticking to a leather jacket,” suggests visual effects supervisor John Knoll. “That was the effect I wanted to get. R&D added this subtle stickiness to the engine.”
“The great thing,” Hickel continues, “is that as complicated as it was, once Karin came up with basic settings for all of the controls, the sim artists got up and running very quickly. I’m pretty amazed by that, actually, because this was very stressful for me. Back in December, when we really didn’t have this working yet, there was no plan B. We couldn’t animate it by hand and we looked at other sim possibilities, but they didn’t achieve what we had in mind. There are more than 200 shots and 15 minutes of screen time of Davy and we had only one artist who knows how to do this.”
With as many as 50 animators working together on a total of 18 CG characters, there were plenty of technical and artistic challenges. “What makes these characters so complicated is that they are encrusted with sea life and we had to figure out ways to cover them with barnacles and such,” Hickel observes. “We wrote tools that the modelers used where they had a sea life picker, where they could pick a mussel or a barnacle. As our model supervisor, Jeff Campbell, said, it was a little like flower arranging. And they also used ZBrush for displacement textures for the sea life and for the characters themselves and our usual suite of modeling and paint tools.”
The crew of The Flying Dutchmen include Ogilvey, who has a sea sponge head; Palafico, whose head is a red fan coral and very translucent; Koleniko, in which one side of his face is a puffer fish and can puff up with spines; and Knoll’s favorite: a crab-like creature whose head rotates in and out of the shell.
 |
|
| The Kraken’s tentacles modeled in Maya. The creature was keyframe animated with some flesh sim enhancements in Zeno, courtesy of the new creature pipeline. |
|
Utilizing ILM’s new fluid dynamics engine, developed in cooperation with the Stanford University research program, Dead Man’s Chest, like Poseidon, contains improved CG water, in which nifty algorithms are put through multiple processors. And thanks to Zeno, which has been described as “Maya on steroids,” you can introduce particle controls, Soft Body, Rigid Body controls and other techniques.
“We started out in parallel with Poseidon, but they got into a bit of a crisis and we loaned them my entire water crew,” Knoll admits. “They wrapped in April and I got them back to finish my shots. They really pushed the envelope. The development they did at the end of Poseidon really paid off here. We did a lot of difficult water shots right up to the last day. The crew really knew what nobs to turn to get it to look good. We used CG water around the bases of the tentacles when they’re sloshing back and forth underwater. The Flying Dutchman travels underwater and reaches the surface like a submarine, so those shots were done with CG water as well, and the Dutchman is 380 feet long. We got realistic droplet size and realistic dispersion of particles.”
The scenes on Cannibal Island, where Captain Jack Sparrow (Johnny Depp) narrowly escapes, contain a large number of shots where you see different variations of the same view under different lighting conditions, so Knoll and matte supervisor Susumu Yukuhiro needed to think about a 3D solution. “We saw ads for a product called Vue. It’s designed for organic landscapes and getting realistic renders. We started playing around with it and it became our primary tool for big, exotic landscapes.”
Overall, Knoll believes Dead Man’s Chest takes character animation another step forward at ILM, especially considering Nighy’s performance. “There are not as many shots numerically as on Sith, but it’s [a greater accomplishment] in terms of the amount of shots in the time that we had. Sith had 2,400 shots in about two years and this had 1,000 shots in about five months, but the average shot complexity was higher than on Star Wars.”
Looting CG Treasure From Dead Man’s Chest — Part 2
Bill Desowitz concludes our two-part coverage of Pirates of the Caribbean: Dead Man’s Chest with a report on ILM’s innovative Imocap system.
By Bill Desowitz
[ Posted on July 17, 2006 ]
Tasked by director Gore Verbinski to come up with more complex and authentic-looking CG characters in Dead Man’s Chest, since Davy Jones and the crew of The Flying Dutchman would be interacting closely with the live actors, Industrial Light & Magic put its R&D team to work on a new incarnation of its proprietary motion capture system, dubbed Imocap. The results of Jones are so impressive, in fact, that people have already begun talking about the sea-encrusted villain with his creepy tentacle beard as the next great CG performance breakthrough.
ILM’s new Imocap was used to create the CG characters of Davy Jones and his crew in Dead Man’s Chest. All images © 2006 Disney Enterprises Inc and Jerry Bruckheimer Inc. Photo credit: ILM.
|
 |
| |
| Early concept art for Davy Jones. | |
|
Imocap became a new protocol for measuring the actors and obtaining data during the actual shoot for the creation of skeletal motion in the computer. The software contained added functionality and new ways of tracking data. Special sensor-studded suits for the actors playing CG characters were created, which were more comfortable than typical MoCap outfits, as the actors were required to wear them in a variety of simple and treacherous conditions. “…On set, I wore a gray suit, which had reference points comprised of white bubbles and strips of black-and-white material, so that when they come to interpret your physical performance, they’re better placed to do so,” adds Bill Nighy, who plays Davy Jones.
According to Sullivan, “the suits needed to be ‘dignified.’ They had to be comfortable and not look ‘stupid.’ There were a few iterations of the material itself, which started out as a cotton blend but ended up being a stretchy, semi formfitting material. And we arrived at a neutral gray to help with our lighting calculations... and we used some markers and bands to help with the capture process itself. Those needed to be comfortable as well. Cameras were based on location and shooting conditions.”
“For shots where we used reference cameras, Kevin Wooley, our Imocap lead, housed some cameras in watertight enclosures and wired them to a computer for storing the images,” explains animation supervisor Hal Hickel. “This was great for the onset stuff. For beaches and jungles, we used untethered cameras with lightweight tripods. They were a little more trouble on the backend because they weren’t synchronized to each other, but both solutions worked well, and will continue to be used on the third Pirates movie [At World’s End].”
Thus, by integrating the MoCap process with the actual shoot — providing the animators with hero plates with the actors in them, casting their real shadows and making good eye contact with the live actors — they were able to create, for instance, a more expressive, nuanced performance out of the maniacal Davy Jones, with the help, of course, of Nighy.
Davy Jones and the crew of The Flying Dutchman interact closely with live actors throughout the film.
“We had new ways for the computer to analyze the images,” Sullivan continues. “The software piggy backed on MARS, the matchmoving [and tracking] solver. It understood what the actors could and couldn’t do. Our process is more holistic than traditional MoCap. We try to capture the whole body at once from different kinds of information, and that allows the flexibility to use many kinds of cameras and to work with partial information sometimes.
“The product of Imocap comes out as an animated skeleton, just like regular MoCap, and the animators do with that whatever they want, with artists in the middle running the post process. Sometimes they’ll need to cheat the body to get a better composition of the image. But the advantage is that the animators are overriding things and animating for performance reasons rather than just getting the basic physics and timing down. That all comes from the actor.”
Although ILM is currently developing its own facial performance capture system, Hickel determined this wasn’t the time to introduce yet another R&D component. “We have a lot of confidence in our facial animation, so we decided to do it by hand. The creature pipeline was being moved over to Zeno and most of the faces were different enough from the actors anyway.”
 |
|
| During the actual shoot on beaches and jungles, ILM used untethered cameras with lightweight tripods to measure the actors and obtain data that was then used to create the characters’ skeletal motions in the computer. |
|
Concludes visual effects supervisor John Knoll: “For us, it’s taken character animation another step forward with Davy Jones and how nice Bill Nighy’s performance comes through.”
Bill Desowitz is editor of VFXWorld.
Note: Readers may contact any VFXWorld contributor by sending an e-mail to editor@vfxworld.com.
