The pioneering sequence was created using a NUKE-based pipeline by the VFX team at Digital Domain in Los Angeles, and marks another step into new territory for a company that is renowned for innovation. Digital Domain has amassed an impressive list of motion picture VFX credits since its founding in 1993, including Titanic, The Day After Tomorrow and The Curious Case of Benjamin Button, not to mention six Academy awards and scores of gongs in the world of advertising. Digital Domain was also the birthplace of NUKE.
During 2008 the company broke new ground when handling 300 effects shots for The Mummy: Tomb Of The Dragon Emperor, directed by Rob Cohen. It made $400 million at the worldwide box-office, and was the third in the hugely successful adventure franchise. Digital Domain integration artist Justin Van der Lek played a pivotal role in the creation of the effects, and takes up the story of how the mud-gushing Emperor was achieved.
“As the name implies, an integration artist literally brings together different elements to help make a whole,” says Van der Lek. “We’re the intermediaries between the set and the digital realm. Our work is all about measurements, and our primary task is to do the camera tracking for all VFX shots, which involves gathering all the relevant information from the set – camera movement, lens and lighting data, the measurements of the set and objects on the set – and packaging it so that the VFX teams can work efficiently and accurately.”
Van der Lek, who was involved with almost every VFX shot that Digital Domain handled on the movie, says that all was going according to plan apart from the shots in which Jet Li spews forth mud.
“The normal way of doing things, compositing CG animated elements together with the live action, was not giving us the subtlety of movement nor the great-looking results we like to achieve,” he says. “So during one of our regular staff meetings Geoffrey Baumann, Digital Domain’s on-set survey and integration lead, put the call out for a different approach. I had a couple of ideas about using NUKE's UV and 3D projection capabilities, and wanted to see if I could bring them together and make it work.”
Van der Lek began by working on a close-up of Li’s face, which required mud to trickle from his hairline, eyes and nose. Using NUKE he stabilised the actor’s head movements within the live action plate, and then imported the 3D tracking data from the same shot, thus gaining access to the character’s facial expressions and movements. These elements, the environmental camera and the channels files, were then projected to UV in NUKE. This yielded 2D, flattened-out visuals of the head, making it easy to check the accuracy of the tracking, and to make the adjustments to the facial expressions, by scrubbing back and forth along the shot’s timeline in Nuke.
Technical director Steve Avoujageli was instructed to animate the mud over the 3D model it had made of the actor’s head, but without any of the facial expressions. The resulting CG renders, imported into NUKE as .OBJ files, were piped through the same projection setup and UV output.
During the composite of the two sets of UVs, Van der Lek employed NUKE's grid warper to adjust the line-up between the CG mud and different parts of the face where any gaps appeared, such as around the hairline and eyes. Using the grid warper he was also able to subtly adjust or exaggerate the actor’s facial expressions by pushing and pulling key points of the images. The results were then projected over the fully-animated 3D model that had also been imported into NUKE. As all of the motion vectors were also in NUKE, Van der Lek applied motion blurs to create a highly polished result.
“The cool thing about doing this as a 2D process in NUKE is that it’s very interactive and super fast,” extols Van der Lek. “I could quickly scrub backwards and forwards through the material, and easily change things – a little tuck to the eye, or a lift to the corner of the mouth. Doing the same updating and iterations in CG would have been a much slower and more painstaking ordeal.”
The technique, which rapidly acquired the moniker “The Justin Technique”, was taken to new levels on the final shot in the sequence. Li staggers across set and towards camera with sludge bursting from his face, mouth and between the gaps in his armor, before becoming completely engulfed in the stuff. The difficulty of working on this shot was compounded as the director also wanted to tweak the actor’s performance to make the action even more dramatic.
Working closely with Avoujageli, and compositor Francis Puthanangadi, Van der Lek employed the same tracking and UV projection techniques, applying them this time not just to the face, but also to the separate shoulder-pad, hand and chest-armour elements of the actor’s upper body. As the VFX team had data and photographic records from the set, NUKE was also used to project high-resolution imagery onto 3D geometry of separate elements in the set – thereby recreating the set in digital form.
“The only real elements in the shot were the lower waist and legs of Jet Li – the rest was CG-fabricated and manipulated in NUKE,” says Van der Lek. “Effectively we had a new Jet Li, as well as a new set environment, that we could completely control in NUKE. People work with UVs, and project on to 3D geometry all the time, but this may be the first time these techniques have been combined on a human form. I anticipate we’ll use it again in the VFX work we do at Digital Domain.”
To adjust Li’s dramatic performance, Van der Lek selected a set of key frames as the actor’s expressions went between extremes. He then employed NUKE's spline warps to create subtle morphs and blends between the different key frames, and to exaggerate the extremes of these expressions – essentially creating a brand new ‘digital’ performance.
For the hands and shoulder pads and chest, Van der Lek again tracked and stabilised each element through the shot, and then looked through the footage to select a sharp still-frame that would project well into the UV space. However, projecting a still onto the animated CG model would show up the static lighting, give an unreal look and undermine the effect. Rather than asking the CG team to add the highlights, Van der Lek used NUKE's paint and colour correction modules to diminish the highlights and reflections on the still image. He then keyed out the highlights from the original live action plate, and projected these back over the shot, to deliver a highly believable result.
“The key thing is that because we had the elements ‘live’ in 3D in NUKE, we had a huge amount of control,” he says. “So, for example, when it came to doing a colour grade of a particular element in the composite, there was no need to ask the 3D team to render out an RGB pass, we could just do that straight from NUKE. We could easily extend the set to accommodate for performance changes that the director wanted. We could also add flickers to flames in the background and add subtle focus pulls. We had all the tools at our fingertips in NUKE – all very cool stuff.”
Van der Lek is impressed by NUKE's speed of performance too. “With NUKE I had a great platform to try new ideas. Your colleagues want to see progress and within a couple of hours we could discover if a new technique was going to work or not. Also, because of its capabilities for handling 3D geometry, you can undertake processes in NUKE that were traditionally done in CG, making the VFX process more intuitive and quicker.
“NUKE has a really broad toolset for handling images. It goes way beyond a typical compositing application and gives you power and flexibility over many different elements within a scene. With NUKE we could quickly show the director that things were working and that they were going in the direction he wanted them to.”
Digital Domain’s work on ‘The Mummy: Tomb of The Dragon Emperor’, was shortlisted for a ‘Visual Effects’ Oscar.
© Universal. Image courtesy of Digital Domain.