How ILM use GPU in pipeline


How ILM using power of GPU in own pipeline


Industrial Light & Magic (ILM) is a Lucasfilm Ltd. company serving the digital needs of the entertainment industry. Since 1975, ILM has been providing extraordinary post-production visual effects services. Motion pictures, commercials, trailers, music videos and special venue projects have utilized ILM's unequaled artistry in techniques such as model making, matte painting, computer-generated imagery, digital animation and a variety of related processes required in the production of visual effects. Groundbreaking software required for digital image production has been developed and shared with the industry at large and adopted by other companies and software manufacturers. ILM regularly consults with some of the most highly regarded filmmakers on proof of concept tests, and has been honored with fifteen Oscars for Best Visual Effects, and has been awarded twenty-three Technical Achievement Awards from the Academy of Motion Picture Arts and Sciences.

ILM's most recent project was creating 485 visual effects shots for Paramount Pictures' "The Last Airbender," the latest film from director M. Night Shyamalan which was released in theaters in July 2010. The film features dramatic sequences in which characters are empowered to harness the elements—fire, air, water and earth—and manipulate them as weapons in battle. Giant fireballs, walls of earth, tendrils of water and powerful air blasts are a few of the digital simulations that were created by ILM.

Industrial Light and Magic and NVIDIA Quadro

Fire and fluid simulations, such as water, are typically very complex to create, and require the trained expertise of a technical visual effects artist or supervisor who's familiar with algorithms and all of the fine details of how fluid solvers work. Additionally, traditional CPU-based render farms make the visual effects iteration process extremely time consuming, limiting creativity in the time-constrained environment reality of film production.

The visual effects masters at ILM wanted the flexibility to create visual effects simulations for "The Last Airbender" both more easily and in less time, so they could have more iterations to choose from and offer to the film's director within the allotted production timeline.



In order to achieve the desired faster workflows and iterations of the various sequences they were creating for "The Last Airbender," ILM created a proprietary tool called Plume. Plume is both a fluid simulation system and a GPU-based renderer that was built using the NVIDIA CUDA™ parallel computing architecture. ILM was able to realize dramatic speed increases with Plume by running simulation renders on a 12-machine GPU-based render farm powered by NVIDIA Quadro FX 5800 professional graphics cards.

ILM's NVIDIA CUDA-based Plume software enabled their simulations to be final frame rendered on their NVIDIA Quadro GPUs. While Plume was initially designed just for rendering fluid simulations, it proved to be flexible enough to aid in the creation of computer generated (CG) air-bending dynamics, ocean mist, smoke and controlling cloud-bending movements. One of the more challenging shots in "The Last Airbender" shows the film's main character, Aang (played by actor Noah Ringer) fighting off hurled streams of fire with combative gusts of air that push the fire and sand particles aside. This shot showcases two Plume simulations of carefully directed air streams and fire, interacting with each other. 

ILM opted to write its GPU-accelerated fluid solver using NVIDIA CUDA rather than Open GL because it simplified the development process. "It's extremely easy to code using CUDA. Developing Plume required us to essentially build a virtual machine to run custom expressions. This is not a trivial task, and writing that code in CUDA allowed us to work quickly, flexibly and required fewer lines of code than it would have taken to write in Open GL," said Olivier Maury, research and development engineer for ILM. 

ILM's primary toolset on "The Last Airbender" included mental ray and Pixar's Renderman for rendering, Autodesk Maya for animation, The Foundry's Nuke for compositing, and proprietary tools Zeno and Saber for CG and effects creation.


ILM has not only been able to achieve their goals of creating more complex visual effects faster and easier, as they achieved for "The Last Airbender," they've also uncovered future opportunities to harness the power of NVIDIA CUDA and Quadro professional GPUs in their production workflows. 

"Because Plume is accelerated for the NVIDIA GPU, it is highly interactive and provides a tool that relies more on the artist's eye than their technical knowledge. You don't have to understand the underlying algorithms or all of the fine details of how fluid solvers work to drive the detail of a simulation," explained Craig Hammack, associate visual effects supervisor, ILM.

"As with everything in high-end visual effects, iteration is essential," added Maury. "It's crucial for the artist to get a handle on how their shot is working, and it's valuable for communicating an interpretation of the directors' vision. By working within an NVIDIA GPU-based framework, we were able to see up to eight iterations a day of complex fire, dust and air simulations. That is hugely significant and represents a 10-15X speed gain over CPU-based simulations which would require overnight renders—at a rate of about one iteration per day."

Hammack further commented, "Realistic Fire is one of the most challenging elements to create believably in CG. For me, the GPU-based fluid solver was the most significant technical advancement to our production pipeline for use on 'The Last Airbender.' There was trepidation at the outset around creating such a high volume of photo-real CG fire alone, never mind controlled fire that is used as a character tool. Plume was a huge advancement in that it allowed artists to get iterations out fast enough to be thinking about shots more creatively rather than technically."

"Having access to CUDA and NVIDIA GPUs has entirely changed the way we approach complex visual effects challenges. We anticipate doing even more development with CUDA to take advantage of time savings using GPU cores in the future," concluded Maury.

ILM plans to incorporate additional NVIDIA CUDA-based tools into future project pipelines and continues to explore new ways to implement Quadro GPU-accelerated rendering into its visual effects workflows.

Quote from Nvidia web: