There are other projects that I have also been involved in were rigging related tasks. These projects are currently not available to the public due to the delayed release dates that’s been pushed back from the recent pandemic.
The tasks were mainly character digi-double body and costume rigging as well as mechanical and prop rigs. I was also assisting the supervisors and leads from different shows to make QC renders and checks. Due to the nature the work I can’t fully disclose the specific task involvements I’ve been a part of as well as add them onto my blog and showreel.
I was tasked again with another shotsculpt task for the latest installment of the Fast & Furious franchise film, Hobbs & Shaw. This task in particular set up work for the FX team with which the cached digi-double mesh is being deformed by the slow motion punch sequence. I had to ensure that the digi-double matches with the silhouette of the hero characters on the 2D plate while maintaining the volume of the 3D mesh. This is to help simulate the water colliding the the characters.
I was assigned two shotsculpt tasks from the same sequence which was both done within a span of 4 working days. At the beginning I was very perfectionist and particular with how the little details. My lead addressed to me to really focus on maintaining the volume rather sculpting in the costume creases which actually guided me back on track to finish my task on time.
We had a chance to really get a better insight of the production pipeline with Alan Woods (CG Supervisor in Framestore). It was a great opportunity to discuss the process from bidding all the way to the final delivery with which I had very little knowledge of. Breaking it down allowed me to fill in the gaps and reach a better understanding of how the data flows within the pipeline.
PRE-PRODUCTION
A task that ATDs are tasked to do at the beginning of a project is to copy and deploy the setup from older shows. An artist audit is also made at this time while doing software regression tests (to make sure that tools function still properly) and publishing default templates.
BIDDING
Each studios would be given an opportunity to present tailored bid packages. Show clients offer to break shots apart from easy to hard shots as well as negotiate costing in relation with the number of hours in production schedule that are being agreed upon. In order to be awarded work (most of the time sequences of shots) artists within the bid team need to convince the client with how they interpret the client’s vision and solve challenges to bring the shot to life.
INGEST DATA
Material such as shot plates, lens grids, HDRIs, Medusa scans, mocap data, and other digital assets that enter the facility are ideally logged into a database (like Shotgun) and are made available to artists and administration. Useful information such as the film back of the camera, frame rates enable artist to no longer make guess work when recreating virtual cameras, characters and environments. Lower resolution QuickTime scans are used by the artists for tracking and are worked on top of without slowing down the processing power of the computer.
TURNOVER
VFX work are officially handed off from the film’s editorial team to the facility to begin to work on. Ensuring that it is accurate to the bid and departments go through and call out the whole scope of the sequence.
DAILIES (REVIEWING WORK)
This happens when work has been submitted to dailies which is an opportunity to sit with the team to discuss and improve upon work. Dailies can either vary from departmental or show. Work that gets produced by an artist are rendered out as play blasts and QC (quality control) renders. These will highlight the create action points that needs to figure out what need focus. Making sure that a standard is met and adhere to the notes given by supervisors or leads to the artist that will need to be addressed for the next dailies session up until work is approved and good to go.
CLIENT REVIEWS
Real-time playback video conference sessions using Cinesync enable clients to review shots and send their notes over to the facility.
DNEG VFX Breakdown of Men In Black: International.
It’s still so surreal to me that the first VFX project that I got to work on a big film franchise. It was the first time I got to interact and work with the Creature team in DNEG. I was still new and confused with so much that went on during the production which is inevitable during the beginning stages. But as I’ve progressed along the weeks, I started to gain more clarity and understanding of the shot workflow.
My role for this project centred around ‘Shotsculpt’. Shot sculpt involves working on top of the animated and even simulated character(s) that will need to be cleaned up. The clean up would entail fixing intersections on the costume and the body of the character. As well as creating creases and building up the silhouette features to create the desired look.
Before and after: From motion capture data to fully rendered animated character
My first task involved working on fixing the costume of the character ‘Vungus the Ugly.’ The character has been modelled, rigged, animated and even simulated and gets passed downstream on to the pipeline ready to be shotsculpted. This process takes in an alembic cache of the character with all the animation and costume data, in which will require to work on top of the shotsculpt rig. This entails adding key frames on the transforms and deformations within the shotsculpt rig to get the desired outcome of the sculpt.
The task was initially given for me to do within 5 bid days. And so I had a multiple mentors that offered to help me throughout the time I had to work on the shot. My mentors all helped me get familiar with setting up the shot, the techniques and introduced me to bespoke tools that enabled to help speed up the process. I’ve also learnt a lot about the application of different deformers such as lattice and delta mush. The training that you really gain from actually doing the work is invaluable because you understand and there’s a way for you to really look back and apply the knowledge that you’ve gained as a result.
I ended up getting my first shot approved after 14 days! I was quite annoyed at myself at first but I had to remember that this was my first task. You will make mistakes and learn important lessons at the very beginning. But it will be worth the struggle because you would’ve gained more insights and resilience. That it’s okay to ask for help because people are willing to help you.
I was also tasked another shot from another sequence which involved correcting costume intersections from 6 crowd agents. In this task I was given 2 bid days and I was able to finish the shot in under one day. This just goes to show how it gets better and easier the more that you practice and perfect the craft.
After having a masterclass session with Kristin Farrensteiner, a Groom TD in DNEG, I was interested to know how the workflow within the grooming pipeline and how the groom is transferred into the CFX pipeline for simulating hair.
The main process includes building the guide curves and groom of the character in Houdini and exporting both data into the proprietary grooming network tool ‘Fur Shop’ which will read the alembic and project the data onto the mesh which can be visualised and will create cluster filters to produce a more realistic representation of hair along with the stray hair with all the noise and clumping attributes added onto the groom.
On the 27th August, I went to a London ACM Siggraph event which was done by the VFX Supervisor Stuart Penn which covered the decision making, development and methods to solve interesting problems that occurred during the production.
One of the interesting discussions were centred around the character work and the machine learning process that they’ve implemented on the Hulk’s face. Initially taken the assets built by ILM, there were about the 100+ facial blendshapes that were needed to created to drive the simulation through the use of solvers and hand-animation. In the end, Framestore opted to blend together the machine learning solve and the hand-animated face. This created a much smoother, enhanced performance that further drove the storytelling forward.
Just as mentioned earlier, Framestore picked up assets from various studios including ILM for the character ‘The Hulk’ and DNEG which was the character ‘Rocket.’ It’s intriguing to find out that companies have the ability to receive assets and share work across multiple sites and companies and how valuable it is to maintain to the brief and work with what’s given to the artists and companies. The Hulk for instance needed more costume changes, so Framestore had to try and accomodate and build these on top of what they’ve been given. This goes to show how important it is to ensure that the assets are in great condition to be passed onto different companies and how team work extends outside of the studios themselves.
So much work involving the creation of CG environments and replacing the the costumes and necks of characters required a lot of process to ensure that the shots look seamless. The warehouse that they shot the scenes had to be built entirely by CG and some of the boxes for the props were composited on the final plate, so that the lights were placed similarly to the original plate. Artists have the ability to cast the right amount of shadow and reflections for for the glassed window structures.
Insight of how the shots were filmed at the similar time as the Infinity Wars was shot meant that there are certain elements that were added and reviewed that needed to implement the Quantum Suits for the time travel sequences. Originally the characters were wearing their original costumes that needed to be rotoscoped and replaced by the CG costumes of the digi-double version of the characters. Some of the neck were also replaced because of the collars from the original sequence, integrating the heads of the cast.
It was incredible how complex sequences were developed and resolved. How the technological advances around machine learning affects the workflow and process necessary for the creation of CG characters. It puts things to perspective and gives me even more reasons why working in this industry fills me with so much excitement and awe!
The ATDs were invited over to the DNEG studios for a VFX Pipeline overview done by the internal trainer at DNEG Jahirul Amin. A run-down of the different departments and roles were mentioned and the masterclass mainly covered the DNEG VFX feature-film pipeline which is sectioned:
He broke down the different stages into the different departments that exist in DNEG which was insightful as I didn’t realise how it involves an expansive set of workforce to create stunning visual effects work.
Game of Thrones breakdown from Pre-Production, Production and Post-Production stages
PRE-PRODUCTION
PitchVis
Large visual effects films tend to spread sequence across different VFX facilities. The pitching stages take place which companies work to approach specific problems and create a sufficient bid material that will award the company the specific sequence to work on. These are all dependent on the availability and the studio’s experience that will calculate the funding and costs of the shot. Internal tests and tools are begun during these stages too.
Art
The art department work on creating 2D/3D concepts that will inform the artistic direction that will influence the asset building stages. If the reference plate is available draw overs of the environments and characters ideas can also be used to weave the storytelling forward. Even 3D concept sculpts in ZBrush takes place.
Previs
The pre-visualisation stage brings the storyboard in 3D space where the narrative takes place. Camera positioned in different angles and lighting can help the director get a glimpse of how to frame certain actions and parts of the sequence that can help before shooting the elements for the VFX shot.
Techvis
To orchestrate highly advanced visual effects shots, a techvis of the shot will map out in detail the essential information that will be useful before shooting on set. These are using physically accurate measurements using real-world dimensions that would calculate the spatial distance of the camera and the objects in shot while also determining the relationship between the camera rigs, actors, extras and visual effects elements.
Virtual Production
Real-time technologies like Unity and Unreal are also used for Virtual Production which is building the scene in 3D space and using VR/AR technologies where the director can move around and experience the camera moving in 3D space. This can be helpful to get more interesting camera angles and perspectives.
PRODUCTION
Element Shoots
Sometimes shooting real-world elements like crowd, dust, fire and smoke against a plain background (black, white, grey, blue, green, or sky) which can be comped to create the final look of the shot without needing to cache out computationally expensive FX simulations.
Data Capture
Gathering proper lighting data will help enable the CG integration to match the 2D plate match. The data wrangler and on-set shoot team will gather data from the following:
Macbeth chart (colour grading)
Grey ball (light intensity)
Chrome ball (light source positions)
LiDAR scan (location spatial data measurements)
Witness cameras (extra video coverage for reference)
HDRI (determine where light is placed in a scene and how bright the lights can be)
Photogrammetry (3D scans created from multiple views 2D image for digi-double and props)
Slate data (lens, camera roll, time of the day)
Mocap
Performance of the actor is tracked and key frame data translated into a motion capture rig. The potential realistic motion will be used as a reference and will be cleaned up and refined by animation.
ASSETS
Data Processing
The data that has been gathered by the shoot department that were onset are stored onto the database. All the camera information for corresponding sequences are available for artists to take away the guess work.
Modelling
Different types of assets from the character, prop, hard surface/mechanical and environment that will be used and integrated on to the shot is built on 3D packages like Maya, Houdini and ZBrush. Ensuring that the model is usable downstream. For character modelling it must be realistic and anatomically correct which follows the topology that can be deformable when rigged and animated. Modellers also unwrap the UV layout that will be used by the texture artist. . There are usually 2-4 different versions of the model from it’s lowest LOD all the way to high LOD which can help to process the models depending on the tasks.
Texturing
Textures for assets are created using Mari and Substance Painter which are then projected on to the asset. High frequency density displacement maps are also created that define pores and wrinkle details of a character. Multiple channel maps extracted from TextureXYZ can be used to better match the textures on to the mesh and add even more realism and definition.
Groom
Guide curves are placed around the character to add groom data that are saved as different sets that will also create the deformable guides used to simulate the hair by Creature FX (CFX).
Lookdev
Lookdev artists are responsible for building preset shaders and put together the assets from texturing, model and groom that will be rendered out to create the desired look that matches the client’s vision. The lighting from the HDRI (high-dynamic range image) captured on set can be applied to test how the materials interact with the real-world reference.
Rigging
Skeletal systems are replicated for the purpose of building animator friendly controls to make animating assets easier.
SHOTS
Camera Tracking
Synchronising the position, orientation and lens of the camera used when element is captured creates a virtual camera. Software like 3D Equalizer and Nuke will figure out the depth from the triangulated points. Once lens distortion is removed and camera track solve generates point cloud data, the position of the camera relative to the spatial source that will match the LIDAR data to produce accurate results.
Body tracking
Lining up the body mesh to match every movement of every frame to track the performance of the character.
Roto
The roto artist is responsible for creating separate layers for the foreground and background elements using curves that are key framed so that the edges are clean and precisely keyed out.
Prep
This process involves using cloning and paint tools that clean up unwanted elements such as wires, tracking markers and shadows.
Layout
Staging the set that will take the reconstructed virtual camera information from the 2D plate. Placement of the CG assets according to the multiple camera elements to ensure that it is unified and in sync with the timing.
Animation
Bring to life the performance of the character movement using controls developed by riggers that are key framed in response to the action that takes place.
Creature FX
Once the performance from animation is translated onto the character, simulation of muscle, hair/fur and cloth is driven to enhance the performance and realism. Simulating crowd behaviours is also within the Creature FX scope.
FX
Dynamic FX simulations (e.g. destruction and water simulations) that replicate the real-world physical characteristics is achieved through building systems that calculate the results by tweaking parameters such as mass, density, elasticity, gravity and wind and running a solver (sets of equations) in Houdini. This is then cached out ready to be sent through to lighting.
Shotsculpt
Taking the character animation and CFX caches layered on top, the shotsculpt artist will fix and correct the intersections and try to preserve the volume and form of the character to help enhance the performance even further.
Environment/DMP
Digital Matte Painting (DMP) and Environment artists work hand in hand to layer over photography and painted elements that can be mapped on top of the original footage to enhance or even extend the environments. DMP mainly use Photoshop and Nuke to create 2.5 projection layered approach while Environment artists use 3D software such as Maya, Nuke and Clarisse to create instances of geometry and elements that can be added on to the shot.
Lighting
The on-set lighting data gathered from the shoot is referenced and recreated in 3D environment. Separate passes are set up and ready to be used by the compositor to tweak the values and intensity of the lit elements.
Compositing
Comp artists have full control and dial down or intensify the shot. Assembling together all the separate layers that can be enhanced and softened to create the client’s desired outcome.
ericka's blog 