It’s a Pirates life for NUKE and MARI
The Aardman studio is synonymous with high-quality stop-motion film. The studio made its name with successful feature films including Flushed Away, Chicken Run and Arthur Christmas as well as the cult-classic Wallace and Gromit series.
The studios’ latest film, Pirates! In an Adventure with Scientists is the most advanced yet, and is widely considered to be the world's first full stop-motion visual effects movie. The extensive 1550 visual effects shots in the film bring together exciting chases across oceans, crowds of rowdy pirates and dazzling landscapes from foggy London to the sunny Caribbean.
One of the unique features of Pirates! was the scale of the world that the Director, Pete Lord was looking to depict. A big part of the VFX challenge here was extending sets with matte paintings and CG models, and increasing the on-screen population with 'digital extras'. Big sets and big crowds can be very expensive and time intensive for stop-motion. Furthermore effects like explosions, flames, oceans and splashes would have been either too crazy or simply impossible to attempt in pure stop-motion.
Andrew Morley was the VFX Supervisor on the project. He’s been working in the visual effects industry since 1996 – starting as a runner at VTR in Soho. Through his career he’s worked on a wide variety of projects. His highlights include working on Star Wars: Episode II - Attack of the Clones at ILM and setting up a VFX department for Lucasfilm in Singapore. There’s one other project he’s particularly proud of. In his own words: “Working on the best stop frame film ever made – Pirates!”
Ben Lock was the VFX and Pre-Viz Producer on Pirates! He comments: “The in-house VFX team on the project worked tirelessly to push the boundaries and expectations of what can be achieved with traditional puppet animation. Every shot was specially styled to give it a unique ‘shot on the studio’ look, one frame at a time.
Building a ship shape pipeline
Aardman has a top class CG department at the main office in central Bristol where the studio does a lot of their commercials work. Pirates! was in fact shot at another Bristol location which was set up specifically to deal with the demands of the Pirates! project. This was also where the film specific VFX department was based.
The pipeline on Pirates! was designed and built from scratch, making sure it was perfectly tailored to meet the needs of Aardman's first in-house VFX feature film. The task for the set up was a big one. The team brought in all new equipment, software, databases, computer networks, screening facilities, editing suites and of course the best talent from across the globe. Morley comments: “The pipeline was built with no legacy issues at all – it was new and fresh! The decision not to use Shake at all allowed us to focus on integrating NUKE entirely into a robust and controlled pipeline. This was something that we knew we would need to allow the 1500 plus shots to go through the VFX department on Pirates!”
He continues: “At this new facility for Pirates! it was not a switch from After Effects and Fusion, but more of a choice to go with a product that we knew would deliver everything that we needed and more. Aardman were pulling out all the stops on this film, and we needed compositing software that could keep delivering, no matter how hard we pushed it. NUKE could do this.”
When Aardman began putting together the team, Ben Toogood was brought in from the Aardman commercials department as one of the senior artists. Toogood began his career working on Harry Potter and the Prisoner of Azkaban and Wallace and Gromit: Curse of the Were Rabbit at MPC before joining Aardman in 2007.
As the shoot for Pirates! ramped up Toogood moved into the role of CG Supervisor, working with the CG teams on shot production, while retaining some involvement with the project wide tool development. During the pre production phase he spent a lot of time working with the pipeline lead Tom Downes to expand the tool set and figure out some of the processes.
He comments: “We'd done plenty of CG character work and stop-motion post production before, but not at the scale required for this project. I think one of the key considerations was the duration of the project and our wish to be setting something up for the long run. From a pipeline point of view, we needed something easily extensible, as we had to streamline production to handle the large shot count but didn't have a massive development team. That seemed to be and area where NUKE really excelled. The rest of our pipeline was being built around a python/QT framework which was also a great fit with NUKE.”
The comping captain
The nature of the stop frame project meant that there was quite a lot of repetition on the shots with elements such as the character facial cleanup. In conjunction with the CG and effects work there was a fairly massive 2D clean-up job. This would involve rig removal, combining plates and painting out any unwanted glitches that would allow the animators to work more freely.
The use of NUKE was extensive on Pirates! “In essence all levels of complexity of shots went though NUKE,” explains Morley. “At the very basic level, we would use NUKE to do stereo realigns and dead camera pixel removal. The next level up often included removing stop frame character artifacts. Every character would have a join line when the face shapes had been changed by the animators and that all had to go.”
The next level of shot complexity was where the team had green screen type shots with a simple 2D background to layer on top. Additionally there were a number of extremely complex CG backgrounds, with a foreground live action plate that went though NUKE. Finally, most complicated of all, there were fully digital shots that featured no live action at all.
Toogood adds: “The cut-lines were a big issue we had to overcome on the project. These seams were present on the faces of nearly all the puppets in the film. They came from the join between the eye/forehead piece and the interchangeable 3D printed mouth pieces. This had to be painted out or patched over in something like 80% of the frames in the film. This was obviously a very time consuming task, but the comp team at Aardman did an amazing job and had it down to a fine art by the end of the production.”
Where shots were taking two or three days at the start, with the help of the bespoke gizmos in NUKE, problems would be sorted in hours.
Toogood continues: “The ability for the leads to work out how to do something and then package it up as a gizmo for repetitive tasks was just great in NUKE. Those simple collaborative tools really helped with hiding some of the complexity away.”
The visual effects department also had to deal with dead pixels where the Canon 1D MKIII bodies being used for the shoot would loose individual sensors over time due to heat and the heavy load placed on them. “This would manifest itself as very small super bright red, green or blue pixels scattered across the image. We put together a set of tools to simplify clearing those out, but you had to keep an eye out for them creeping in,” Toogood explains.
NUKEX overboard!
The visual effects department also made extensive use of the additional feature set in NUKEX to really push the boundaries of the film. Morley comments: “On some of the more complex shots, we used pretty much every feature we could get our hands on in NUKEX! We even touched on 3D particles, which is a very thorough addition to NUKEX.”
Toogood adds: “The the camera distortion toolset feature was released just in time for us to use on Pirates! We had been looking at alternatives, but when version 6 of NUKEX turned up we found it met our needs perfectly and we ended up using it throughout the project for lens analysis and distort/un-distort.
Many of the compositors also found the 3D tracking tools very handy for assisting with the clean-up tasks. Toogood continues: “They could get a solve really quickly and then generate useful point clouds from those.”
Taking the stereo plunge
Although the film was not originally conceived as a 3D stereoscopic film, when the decision was made to shoot in stereo, the VFX team felt in very safe hands with NUKE. Morley explains: “NUKE’s native stereoscopic support proved really essential.”
Toogood comments: “Each frame was shot on a stereo camera slider and lots of people had problems with highlights, stereo-depth and even stereo-mono transitions that needed special attention. NUKE gave us the flexibility to create fast, artist friendly solutions that integrated seamlessly into the production workflow.”
Ahoy MARI!
James Furlong was the Texture and Lookdev Lead on Pirates! “It was clear from my first day at Aardman that there was an air of apprehension about capturing that 'made in the shed' look,” he comments. “When the project started out the team pretty much just consisted of me but I knew that I was going to be working with artists who were creatively gifted in other disciplines with a variety of experience. There was an evaluation copy of MARI floating around as I was thinking about the best tools for the project. I was sure that it would be a natural fit for Aardman having used it at Weta on Avatar and Tintin.”
The team were supplied with concept artwork and colour charts by the Art Department, and given extensive access to the sets to take reference photographs. Furlong adds: “This was all really important to ensure we could do justice to the painterly style and subtle nuances of the materials.”
As the project progressed and the team grew Furlong introduced them to MARI as the paint tool of choice. He comments: “MARI is such an intuitive and natural tool that the artists mastered it in no time and developed their own working methods. Painting directly onto our assets with the array of brushes in MARI meant that we were able to accurately match the silicon, foam and plasticine textures needed for the digital double characters. It also meant that we were able to quickly work up 'extras' with colour and texture variations. We knew we were on the right track when we won the director's confidence by matching the rocky coastline required for an expansive set extension to the Blood Island set.”
The film had a number of very complicated assets, like the Pirate Captain Ship and the Queen Victoria 1. These had a lot of small, fiddly parts that were of particular concern to the team. Furlong explains: “Using MARI we were able to organise our maps seamlessly, working up the level of painted detail, creating wet masks to help drive the huge water simulations, with the added benefit of being able make adjustments on the fly. All of this could have been a real headache in Photoshop.”
He continues: “Often our bigger assets needed to be reworked to get them to final, like our hero characters - the Whale and Sea Monster. MARI's shader system came in to its own when we needed to build on layers of detail to take it from the concept stage to a final asset that looks like it could have just left Aardman's physical model making department.”
Thoughts on The Foundry
Morley only has good things to say about The Foundry and how the software helped on the Pirates! project. “Working on the Pirates! film was a unique experience for everyone involved, and this included The Foundry folk too. I feel old saying this, but The Foundry offer genuine and real person to person support - something you simply do not seem to get from other software vendors. The Foundry team are simply passionate about what they do, and they want to help out.”