There are other projects that I have also been involved in were rigging related tasks. These projects are currently not available to the public due to the delayed release dates that’s been pushed back from the recent pandemic.
The tasks were mainly character digi-double body and costume rigging as well as mechanical and prop rigs. I was also assisting the supervisors and leads from different shows to make QC renders and checks. Due to the nature the work I can’t fully disclose the specific task involvements I’ve been a part of as well as add them onto my blog and showreel.
I was tasked again with another shotsculpt task for the latest installment of the Fast & Furious franchise film, Hobbs & Shaw. This task in particular set up work for the FX team with which the cached digi-double mesh is being deformed by the slow motion punch sequence. I had to ensure that the digi-double matches with the silhouette of the hero characters on the 2D plate while maintaining the volume of the 3D mesh. This is to help simulate the water colliding the the characters.
I was assigned two shotsculpt tasks from the same sequence which was both done within a span of 4 working days. At the beginning I was very perfectionist and particular with how the little details. My lead addressed to me to really focus on maintaining the volume rather sculpting in the costume creases which actually guided me back on track to finish my task on time.
Being in DNEG there are a plethora of resources that you get to explore and dive right into. At the start of my apprenticeship I took the opportunity to expand my knowledge learning about human anatomy.
This is mainly because most of the character work, for digital-double works (replicating and introducing CG characters onto the film sequence) or any type of character development work, revolves around building the anatomical features such as skeleton and muscles to directly impact the character’s mesh to create a more “life-like” deformation when animating extreme range of motions.
Although it would seem like having to replicate the exact same anatomical features would be the desired workflow, it can become really cumbersome to replicate every single joints, bone structures and muscle structures as the character model will end up becoming extremely heavy to try to replicate.
Functional Anatomy
There are so many things to consider when incorporating or dealing with anatomical features especially in the more complex creature work that is currently taking place in DNEG. A presentation on ‘Functional Anatomy’ helped enable creature artists to become even more mindful on the function and influences of prominent bodily features.
Joint Placements
It is very important to consider and take a substantial amount of time to make sure that the joints are placed correctly. This is vital to be able to place the joints correctly in the first place to ensure that the rig is stable and will deform correctly. Once you animate using the version of the rig that was passed down to the animator, and the rig updates it will end up changing the animation and will behave differently.
Skinning Guides
I briefly looked at the the skinning guides for different sections of the body mainly around the upper body and the arms and legs. Evaluating and improving ways to deform and preserve the volume of the mesh.
Body Types
Another extra bit of information around body types were looked into. To get a better understanding of the frame and muscular build of the three main body types:
DNEG VFX Breakdown of Men In Black: International.
It’s still so surreal to me that the first VFX project that I got to work on a big film franchise. It was the first time I got to interact and work with the Creature team in DNEG. I was still new and confused with so much that went on during the production which is inevitable during the beginning stages. But as I’ve progressed along the weeks, I started to gain more clarity and understanding of the shot workflow.
My role for this project centred around ‘Shotsculpt’. Shot sculpt involves working on top of the animated and even simulated character(s) that will need to be cleaned up. The clean up would entail fixing intersections on the costume and the body of the character. As well as creating creases and building up the silhouette features to create the desired look.
Before and after: From motion capture data to fully rendered animated character
My first task involved working on fixing the costume of the character ‘Vungus the Ugly.’ The character has been modelled, rigged, animated and even simulated and gets passed downstream on to the pipeline ready to be shotsculpted. This process takes in an alembic cache of the character with all the animation and costume data, in which will require to work on top of the shotsculpt rig. This entails adding key frames on the transforms and deformations within the shotsculpt rig to get the desired outcome of the sculpt.
The task was initially given for me to do within 5 bid days. And so I had a multiple mentors that offered to help me throughout the time I had to work on the shot. My mentors all helped me get familiar with setting up the shot, the techniques and introduced me to bespoke tools that enabled to help speed up the process. I’ve also learnt a lot about the application of different deformers such as lattice and delta mush. The training that you really gain from actually doing the work is invaluable because you understand and there’s a way for you to really look back and apply the knowledge that you’ve gained as a result.
I ended up getting my first shot approved after 14 days! I was quite annoyed at myself at first but I had to remember that this was my first task. You will make mistakes and learn important lessons at the very beginning. But it will be worth the struggle because you would’ve gained more insights and resilience. That it’s okay to ask for help because people are willing to help you.
I was also tasked another shot from another sequence which involved correcting costume intersections from 6 crowd agents. In this task I was given 2 bid days and I was able to finish the shot in under one day. This just goes to show how it gets better and easier the more that you practice and perfect the craft.
As well as processing through the ATD – CFX apprenticeships, I also want to work on some personal projects that I’ve had in mind and try to create something that reflects on my current stage of learning and development as a CFX apprentice.
One of the projects that I would like to tackle is rigging and creating controls suitable for production. A lot of the attention can be drawn particularly from the way a character’s facial expressions and hand gestures that when it seems odd it disrupts the suspension of disbelief. Hands are important aspects when it comes to expressing emotions and a character’s personality. There are plenty of opportunities to look into building a rig system that an animator can use to help provide support the flow of the animators’ work.
Hand Rigging (Ongoing)
Bone structure of a hand
At the moment I am learning to rig in Maya which is something that I’ve felt uncomfortable and would try to avoid at any cost! But I told myself that in order to grow and become better at your craft you have to do the uncomfortable. I decided to rig hands simply because it’s more focused on a certain body part rather than tackling the whole human body which can seem overwhelming to wrap your head around at the beginning. Therefore in the next 8 weeks (setting myself a deadline so I can commit to it) I will hopefully get to a stage where I am more confident in creating a hand rig that I can be proud of sharing.
PLAN OF ACTION
I plan to work on this project for 8 weeks. From the research and development stage, (modelling?), rigging, skinning and creating range of motion animation tests.
Feathers Groom and Simulation (Started)
Sectioning parts of the bird and of a single feather
For this project I have set off to learn more about feathers and how to approach the workflow from grooming and simulating feathers. It is of the more challenging aspects in the CFX pipeline which I’d like to tackle and work towards improving my knowledge upon simply because there’s very few feather systems that work to fix the interpenetration between feathers.
PLAN OF ACTION
This project is a little bit more ambitious since I have very little experience with feather systems in general and I’m still fairly new with Houdini. I plan to tackle this project straight after the hand rigging. And an ongoing research and development will still take place as a move through my apprenticeship
One of the areas of interest that I’d like to go in-depth and learn more about are the facial rigging techniques that have been used in the industry. So much emphasis goes into producing extensively complex face rigs that allow the facial animation team to develop expressive and compelling characters.
The current techniques used in industry are blendshape driven that follow the FACS (Facial Action Coding System) along with the AUs (Action Units) to facilitate almost all controls that can help define the facial expressions more effectively and precisely.
After having a masterclass session with Kristin Farrensteiner, a Groom TD in DNEG, I was interested to know how the workflow within the grooming pipeline and how the groom is transferred into the CFX pipeline for simulating hair.
The main process includes building the guide curves and groom of the character in Houdini and exporting both data into the proprietary grooming network tool ‘Fur Shop’ which will read the alembic and project the data onto the mesh which can be visualised and will create cluster filters to produce a more realistic representation of hair along with the stray hair with all the noise and clumping attributes added onto the groom.
We had an opportunity to work with Ceylan Jawara, a CG supervisor that works in a visual effects studio called One Of Us. She was able to show us the different workflows that’s centred around creature FX which is directly related to my apprenticeship in DNEG.
We briefly looked into the tools and workflows that are used in Maya to help add a level of realism by simulating certain elements such as hair, muscle, and a bit of cloth workflow on a walking animated fox cache that have been modelled, rigged and animated by artists working in One of Us.
Shotsculpt
One of the elements we looked at is the process of shotsculpt which looked at deforming certain poses using blendshapes and to fix or add volume/muscle structure onto a character. This is done by keying the specific frame of the sculpt and changing the influence of the float values from 0.000 to 1.000 on the level of influence onto the animated cache.
Hair and Cloth
We also looked at the nucleus solver that drive the hair, cloth and other particle simulations in Maya for simulating the brows, whiskers and ears.
What’s also important outside of work is learning and expanding the knowledge that you’ve acquired from industry-led events. Many talks and events happen across the year which you can book for free and meet like minded people within the industry. It’s a perfect opportunity to expand your network and understanding how different studios work and their methods of solving problems within issues that arise.
This talk in particular focused on the whole spectrum of problems such as making sure that the animatronic version of the dinosaur looked convincing enough, and on to adding the lens on the glasses of the frames with no glass attached to capture the reflections and light.
The ATDs were invited over to the DNEG studios for a VFX Pipeline overview done by the internal trainer at DNEG Jahirul Amin. A run-down of the different departments and roles were mentioned and the masterclass mainly covered the DNEG VFX feature-film pipeline which is sectioned:
He broke down the different stages into the different departments that exist in DNEG which was insightful as I didn’t realise how it involves an expansive set of workforce to create stunning visual effects work.
Game of Thrones breakdown from Pre-Production, Production and Post-Production stages
PRE-PRODUCTION
PitchVis
Large visual effects films tend to spread sequence across different VFX facilities. The pitching stages take place which companies work to approach specific problems and create a sufficient bid material that will award the company the specific sequence to work on. These are all dependent on the availability and the studio’s experience that will calculate the funding and costs of the shot. Internal tests and tools are begun during these stages too.
Art
The art department work on creating 2D/3D concepts that will inform the artistic direction that will influence the asset building stages. If the reference plate is available draw overs of the environments and characters ideas can also be used to weave the storytelling forward. Even 3D concept sculpts in ZBrush takes place.
Previs
The pre-visualisation stage brings the storyboard in 3D space where the narrative takes place. Camera positioned in different angles and lighting can help the director get a glimpse of how to frame certain actions and parts of the sequence that can help before shooting the elements for the VFX shot.
Techvis
To orchestrate highly advanced visual effects shots, a techvis of the shot will map out in detail the essential information that will be useful before shooting on set. These are using physically accurate measurements using real-world dimensions that would calculate the spatial distance of the camera and the objects in shot while also determining the relationship between the camera rigs, actors, extras and visual effects elements.
Virtual Production
Real-time technologies like Unity and Unreal are also used for Virtual Production which is building the scene in 3D space and using VR/AR technologies where the director can move around and experience the camera moving in 3D space. This can be helpful to get more interesting camera angles and perspectives.
PRODUCTION
Element Shoots
Sometimes shooting real-world elements like crowd, dust, fire and smoke against a plain background (black, white, grey, blue, green, or sky) which can be comped to create the final look of the shot without needing to cache out computationally expensive FX simulations.
Data Capture
Gathering proper lighting data will help enable the CG integration to match the 2D plate match. The data wrangler and on-set shoot team will gather data from the following:
Macbeth chart (colour grading)
Grey ball (light intensity)
Chrome ball (light source positions)
LiDAR scan (location spatial data measurements)
Witness cameras (extra video coverage for reference)
HDRI (determine where light is placed in a scene and how bright the lights can be)
Photogrammetry (3D scans created from multiple views 2D image for digi-double and props)
Slate data (lens, camera roll, time of the day)
Mocap
Performance of the actor is tracked and key frame data translated into a motion capture rig. The potential realistic motion will be used as a reference and will be cleaned up and refined by animation.
ASSETS
Data Processing
The data that has been gathered by the shoot department that were onset are stored onto the database. All the camera information for corresponding sequences are available for artists to take away the guess work.
Modelling
Different types of assets from the character, prop, hard surface/mechanical and environment that will be used and integrated on to the shot is built on 3D packages like Maya, Houdini and ZBrush. Ensuring that the model is usable downstream. For character modelling it must be realistic and anatomically correct which follows the topology that can be deformable when rigged and animated. Modellers also unwrap the UV layout that will be used by the texture artist. . There are usually 2-4 different versions of the model from it’s lowest LOD all the way to high LOD which can help to process the models depending on the tasks.
Texturing
Textures for assets are created using Mari and Substance Painter which are then projected on to the asset. High frequency density displacement maps are also created that define pores and wrinkle details of a character. Multiple channel maps extracted from TextureXYZ can be used to better match the textures on to the mesh and add even more realism and definition.
Groom
Guide curves are placed around the character to add groom data that are saved as different sets that will also create the deformable guides used to simulate the hair by Creature FX (CFX).
Lookdev
Lookdev artists are responsible for building preset shaders and put together the assets from texturing, model and groom that will be rendered out to create the desired look that matches the client’s vision. The lighting from the HDRI (high-dynamic range image) captured on set can be applied to test how the materials interact with the real-world reference.
Rigging
Skeletal systems are replicated for the purpose of building animator friendly controls to make animating assets easier.
SHOTS
Camera Tracking
Synchronising the position, orientation and lens of the camera used when element is captured creates a virtual camera. Software like 3D Equalizer and Nuke will figure out the depth from the triangulated points. Once lens distortion is removed and camera track solve generates point cloud data, the position of the camera relative to the spatial source that will match the LIDAR data to produce accurate results.
Body tracking
Lining up the body mesh to match every movement of every frame to track the performance of the character.
Roto
The roto artist is responsible for creating separate layers for the foreground and background elements using curves that are key framed so that the edges are clean and precisely keyed out.
Prep
This process involves using cloning and paint tools that clean up unwanted elements such as wires, tracking markers and shadows.
Layout
Staging the set that will take the reconstructed virtual camera information from the 2D plate. Placement of the CG assets according to the multiple camera elements to ensure that it is unified and in sync with the timing.
Animation
Bring to life the performance of the character movement using controls developed by riggers that are key framed in response to the action that takes place.
Creature FX
Once the performance from animation is translated onto the character, simulation of muscle, hair/fur and cloth is driven to enhance the performance and realism. Simulating crowd behaviours is also within the Creature FX scope.
FX
Dynamic FX simulations (e.g. destruction and water simulations) that replicate the real-world physical characteristics is achieved through building systems that calculate the results by tweaking parameters such as mass, density, elasticity, gravity and wind and running a solver (sets of equations) in Houdini. This is then cached out ready to be sent through to lighting.
Shotsculpt
Taking the character animation and CFX caches layered on top, the shotsculpt artist will fix and correct the intersections and try to preserve the volume and form of the character to help enhance the performance even further.
Environment/DMP
Digital Matte Painting (DMP) and Environment artists work hand in hand to layer over photography and painted elements that can be mapped on top of the original footage to enhance or even extend the environments. DMP mainly use Photoshop and Nuke to create 2.5 projection layered approach while Environment artists use 3D software such as Maya, Nuke and Clarisse to create instances of geometry and elements that can be added on to the shot.
Lighting
The on-set lighting data gathered from the shoot is referenced and recreated in 3D environment. Separate passes are set up and ready to be used by the compositor to tweak the values and intensity of the lit elements.
Compositing
Comp artists have full control and dial down or intensify the shot. Assembling together all the separate layers that can be enhanced and softened to create the client’s desired outcome.
ericka's blog 