The ATDs were invited over to the DNEG studios for a VFX Pipeline overview done by the internal trainer at DNEG Jahirul Amin. A run-down of the different departments and roles were mentioned and the masterclass mainly covered the DNEG VFX feature-film pipeline which is sectioned:
He broke down the different stages into the different departments that exist in DNEG which was insightful as I didn’t realise how it involves an expansive set of workforce to create stunning visual effects work.
PRE-PRODUCTION
PitchVis
Large visual effects films tend to spread sequence across different VFX facilities. The pitching stages take place which companies work to approach specific problems and create a sufficient bid material that will award the company the specific sequence to work on. These are all dependent on the availability and the studio’s experience that will calculate the funding and costs of the shot. Internal tests and tools are begun during these stages too.
Art
The art department work on creating 2D/3D concepts that will inform the artistic direction that will influence the asset building stages. If the reference plate is available draw overs of the environments and characters ideas can also be used to weave the storytelling forward. Even 3D concept sculpts in ZBrush takes place.
Previs
The pre-visualisation stage brings the storyboard in 3D space where the narrative takes place. Camera positioned in different angles and lighting can help the director get a glimpse of how to frame certain actions and parts of the sequence that can help before shooting the elements for the VFX shot.
Techvis
To orchestrate highly advanced visual effects shots, a techvis of the shot will map out in detail the essential information that will be useful before shooting on set. These are using physically accurate measurements using real-world dimensions that would calculate the spatial distance of the camera and the objects in shot while also determining the relationship between the camera rigs, actors, extras and visual effects elements.
Virtual Production
Real-time technologies like Unity and Unreal are also used for Virtual Production which is building the scene in 3D space and using VR/AR technologies where the director can move around and experience the camera moving in 3D space. This can be helpful to get more interesting camera angles and perspectives.
PRODUCTION
Element Shoots
Sometimes shooting real-world elements like crowd, dust, fire and smoke against a plain background (black, white, grey, blue, green, or sky) which can be comped to create the final look of the shot without needing to cache out computationally expensive FX simulations.
Data Capture
Gathering proper lighting data will help enable the CG integration to match the 2D plate match. The data wrangler and on-set shoot team will gather data from the following:
- Macbeth chart (colour grading)
- Grey ball (light intensity)
- Chrome ball (light source positions)
- LiDAR scan (location spatial data measurements)
- Witness cameras (extra video coverage for reference)
- HDRI (determine where light is placed in a scene and how bright the lights can be)
- Photogrammetry (3D scans created from multiple views 2D image for digi-double and props)
- Slate data (lens, camera roll, time of the day)
Mocap
Performance of the actor is tracked and key frame data translated into a motion capture rig. The potential realistic motion will be used as a reference and will be cleaned up and refined by animation.
ASSETS
Data Processing
The data that has been gathered by the shoot department that were onset are stored onto the database. All the camera information for corresponding sequences are available for artists to take away the guess work.
Modelling
Different types of assets from the character, prop, hard surface/mechanical and environment that will be used and integrated on to the shot is built on 3D packages like Maya, Houdini and ZBrush. Ensuring that the model is usable downstream. For character modelling it must be realistic and anatomically correct which follows the topology that can be deformable when rigged and animated. Modellers also unwrap the UV layout that will be used by the texture artist. . There are usually 2-4 different versions of the model from it’s lowest LOD all the way to high LOD which can help to process the models depending on the tasks.
Texturing
Textures for assets are created using Mari and Substance Painter which are then projected on to the asset. High frequency density displacement maps are also created that define pores and wrinkle details of a character. Multiple channel maps extracted from TextureXYZ can be used to better match the textures on to the mesh and add even more realism and definition.
Groom
Guide curves are placed around the character to add groom data that are saved as different sets that will also create the deformable guides used to simulate the hair by Creature FX (CFX).
Lookdev
Lookdev artists are responsible for building preset shaders and put together the assets from texturing, model and groom that will be rendered out to create the desired look that matches the client’s vision. The lighting from the HDRI (high-dynamic range image) captured on set can be applied to test how the materials interact with the real-world reference.
Rigging
Skeletal systems are replicated for the purpose of building animator friendly controls to make animating assets easier.
SHOTS
Camera Tracking
Synchronising the position, orientation and lens of the camera used when element is captured creates a virtual camera. Software like 3D Equalizer and Nuke will figure out the depth from the triangulated points. Once lens distortion is removed and camera track solve generates point cloud data, the position of the camera relative to the spatial source that will match the LIDAR data to produce accurate results.
Body tracking
Lining up the body mesh to match every movement of every frame to track the performance of the character.
Roto
The roto artist is responsible for creating separate layers for the foreground and background elements using curves that are key framed so that the edges are clean and precisely keyed out.
Prep
This process involves using cloning and paint tools that clean up unwanted elements such as wires, tracking markers and shadows.
Layout
Staging the set that will take the reconstructed virtual camera information from the 2D plate. Placement of the CG assets according to the multiple camera elements to ensure that it is unified and in sync with the timing.
Animation
Bring to life the performance of the character movement using controls developed by riggers that are key framed in response to the action that takes place.
Creature FX
Once the performance from animation is translated onto the character, simulation of muscle, hair/fur and cloth is driven to enhance the performance and realism. Simulating crowd behaviours is also within the Creature FX scope.
FX
Dynamic FX simulations (e.g. destruction and water simulations) that replicate the real-world physical characteristics is achieved through building systems that calculate the results by tweaking parameters such as mass, density, elasticity, gravity and wind and running a solver (sets of equations) in Houdini. This is then cached out ready to be sent through to lighting.
Shotsculpt
Taking the character animation and CFX caches layered on top, the shotsculpt artist will fix and correct the intersections and try to preserve the volume and form of the character to help enhance the performance even further.
Environment/DMP
Digital Matte Painting (DMP) and Environment artists work hand in hand to layer over photography and painted elements that can be mapped on top of the original footage to enhance or even extend the environments. DMP mainly use Photoshop and Nuke to create 2.5 projection layered approach while Environment artists use 3D software such as Maya, Nuke and Clarisse to create instances of geometry and elements that can be added on to the shot.
Lighting
The on-set lighting data gathered from the shoot is referenced and recreated in 3D environment. Separate passes are set up and ready to be used by the compositor to tweak the values and intensity of the lit elements.
Compositing
Comp artists have full control and dial down or intensify the shot. Assembling together all the separate layers that can be enhanced and softened to create the client’s desired outcome.
ericka's blog 