Your Future In STEM – VFX Apprenticeships with ACCESS:VFX
On the 11th October, I had the absolute pleasure to speak in front of students, parents and other people who were interested to learn more about the visual effects, about the apprenticeship route as a way to get into the industry during the New Scientist Live 2019 – Your Future In Stem panel in the ExCel Centre . It was also a great opportunity to develop and improve on my public speaking skills. Most importantly it provided a great platform to help influence the future generations a career centred around the STEM (Science, Technology, Engineering and Maths) subjects.
Something that I grown to realise over the course of this apprenticeship was how valuable the foundations that I’ve acquired from the STEM subjects has enabled me to discover my passions getting into the visual effects industry. And I think it’s important to give other alternatives for those seeking to get in, the options that are available to them.
As well as processing through the ATD – CFX apprenticeships, I also want to work on some personal projects that I’ve had in mind and try to create something that reflects on my current stage of learning and development as a CFX apprentice.
One of the projects that I would like to tackle is rigging and creating controls suitable for production. A lot of the attention can be drawn particularly from the way a character’s facial expressions and hand gestures that when it seems odd it disrupts the suspension of disbelief. Hands are important aspects when it comes to expressing emotions and a character’s personality. There are plenty of opportunities to look into building a rig system that an animator can use to help provide support the flow of the animators’ work.
Hand Rigging (Ongoing)
Bone structure of a hand
At the moment I am learning to rig in Maya which is something that I’ve felt uncomfortable and would try to avoid at any cost! But I told myself that in order to grow and become better at your craft you have to do the uncomfortable. I decided to rig hands simply because it’s more focused on a certain body part rather than tackling the whole human body which can seem overwhelming to wrap your head around at the beginning. Therefore in the next 8 weeks (setting myself a deadline so I can commit to it) I will hopefully get to a stage where I am more confident in creating a hand rig that I can be proud of sharing.
PLAN OF ACTION
I plan to work on this project for 8 weeks. From the research and development stage, (modelling?), rigging, skinning and creating range of motion animation tests.
Feathers Groom and Simulation (Started)
Sectioning parts of the bird and of a single feather
For this project I have set off to learn more about feathers and how to approach the workflow from grooming and simulating feathers. It is of the more challenging aspects in the CFX pipeline which I’d like to tackle and work towards improving my knowledge upon simply because there’s very few feather systems that work to fix the interpenetration between feathers.
PLAN OF ACTION
This project is a little bit more ambitious since I have very little experience with feather systems in general and I’m still fairly new with Houdini. I plan to tackle this project straight after the hand rigging. And an ongoing research and development will still take place as a move through my apprenticeship
One of the areas of interest that I’d like to go in-depth and learn more about are the facial rigging techniques that have been used in the industry. So much emphasis goes into producing extensively complex face rigs that allow the facial animation team to develop expressive and compelling characters.
The current techniques used in industry are blendshape driven that follow the FACS (Facial Action Coding System) along with the AUs (Action Units) to facilitate almost all controls that can help define the facial expressions more effectively and precisely.
After having a masterclass session with Kristin Farrensteiner, a Groom TD in DNEG, I was interested to know how the workflow within the grooming pipeline and how the groom is transferred into the CFX pipeline for simulating hair.
The main process includes building the guide curves and groom of the character in Houdini and exporting both data into the proprietary grooming network tool ‘Fur Shop’ which will read the alembic and project the data onto the mesh which can be visualised and will create cluster filters to produce a more realistic representation of hair along with the stray hair with all the noise and clumping attributes added onto the groom.
On the 27th August, I went to a London ACM Siggraph event which was done by the VFX Supervisor Stuart Penn which covered the decision making, development and methods to solve interesting problems that occurred during the production.
One of the interesting discussions were centred around the character work and the machine learning process that they’ve implemented on the Hulk’s face. Initially taken the assets built by ILM, there were about the 100+ facial blendshapes that were needed to created to drive the simulation through the use of solvers and hand-animation. In the end, Framestore opted to blend together the machine learning solve and the hand-animated face. This created a much smoother, enhanced performance that further drove the storytelling forward.
Just as mentioned earlier, Framestore picked up assets from various studios including ILM for the character ‘The Hulk’ and DNEG which was the character ‘Rocket.’ It’s intriguing to find out that companies have the ability to receive assets and share work across multiple sites and companies and how valuable it is to maintain to the brief and work with what’s given to the artists and companies. The Hulk for instance needed more costume changes, so Framestore had to try and accomodate and build these on top of what they’ve been given. This goes to show how important it is to ensure that the assets are in great condition to be passed onto different companies and how team work extends outside of the studios themselves.
So much work involving the creation of CG environments and replacing the the costumes and necks of characters required a lot of process to ensure that the shots look seamless. The warehouse that they shot the scenes had to be built entirely by CG and some of the boxes for the props were composited on the final plate, so that the lights were placed similarly to the original plate. Artists have the ability to cast the right amount of shadow and reflections for for the glassed window structures.
Insight of how the shots were filmed at the similar time as the Infinity Wars was shot meant that there are certain elements that were added and reviewed that needed to implement the Quantum Suits for the time travel sequences. Originally the characters were wearing their original costumes that needed to be rotoscoped and replaced by the CG costumes of the digi-double version of the characters. Some of the neck were also replaced because of the collars from the original sequence, integrating the heads of the cast.
It was incredible how complex sequences were developed and resolved. How the technological advances around machine learning affects the workflow and process necessary for the creation of CG characters. It puts things to perspective and gives me even more reasons why working in this industry fills me with so much excitement and awe!
We had an opportunity to work with Ceylan Jawara, a CG supervisor that works in a visual effects studio called One Of Us. She was able to show us the different workflows that’s centred around creature FX which is directly related to my apprenticeship in DNEG.
We briefly looked into the tools and workflows that are used in Maya to help add a level of realism by simulating certain elements such as hair, muscle, and a bit of cloth workflow on a walking animated fox cache that have been modelled, rigged and animated by artists working in One of Us.
Shotsculpt
One of the elements we looked at is the process of shotsculpt which looked at deforming certain poses using blendshapes and to fix or add volume/muscle structure onto a character. This is done by keying the specific frame of the sculpt and changing the influence of the float values from 0.000 to 1.000 on the level of influence onto the animated cache.
Hair and Cloth
We also looked at the nucleus solver that drive the hair, cloth and other particle simulations in Maya for simulating the brows, whiskers and ears.
Events like the Vertex conference, which took place on the 8th March 2019, are great opportunities to widen my knowledge within the industry as well as network with other talent that share the same passion as you.
One of the things I was able to look into was the ocean simulations for Adrift and the cloud rendering pipeline and solutions that Milk has used. It was quite technical and some of the concepts I know very little about but it’s always interesting to see how other studios.
We also got to meet Tom Box, the Co-Founder of Blue Zoo Animations, in their designated booth and other industry professionals that were involved in the conference.
What’s also important outside of work is learning and expanding the knowledge that you’ve acquired from industry-led events. Many talks and events happen across the year which you can book for free and meet like minded people within the industry. It’s a perfect opportunity to expand your network and understanding how different studios work and their methods of solving problems within issues that arise.
This talk in particular focused on the whole spectrum of problems such as making sure that the animatronic version of the dinosaur looked convincing enough, and on to adding the lens on the glasses of the frames with no glass attached to capture the reflections and light.
The ATDs were invited over to the DNEG studios for a VFX Pipeline overview done by the internal trainer at DNEG Jahirul Amin. A run-down of the different departments and roles were mentioned and the masterclass mainly covered the DNEG VFX feature-film pipeline which is sectioned:
He broke down the different stages into the different departments that exist in DNEG which was insightful as I didn’t realise how it involves an expansive set of workforce to create stunning visual effects work.
Game of Thrones breakdown from Pre-Production, Production and Post-Production stages
PRE-PRODUCTION
PitchVis
Large visual effects films tend to spread sequence across different VFX facilities. The pitching stages take place which companies work to approach specific problems and create a sufficient bid material that will award the company the specific sequence to work on. These are all dependent on the availability and the studio’s experience that will calculate the funding and costs of the shot. Internal tests and tools are begun during these stages too.
Art
The art department work on creating 2D/3D concepts that will inform the artistic direction that will influence the asset building stages. If the reference plate is available draw overs of the environments and characters ideas can also be used to weave the storytelling forward. Even 3D concept sculpts in ZBrush takes place.
Previs
The pre-visualisation stage brings the storyboard in 3D space where the narrative takes place. Camera positioned in different angles and lighting can help the director get a glimpse of how to frame certain actions and parts of the sequence that can help before shooting the elements for the VFX shot.
Techvis
To orchestrate highly advanced visual effects shots, a techvis of the shot will map out in detail the essential information that will be useful before shooting on set. These are using physically accurate measurements using real-world dimensions that would calculate the spatial distance of the camera and the objects in shot while also determining the relationship between the camera rigs, actors, extras and visual effects elements.
Virtual Production
Real-time technologies like Unity and Unreal are also used for Virtual Production which is building the scene in 3D space and using VR/AR technologies where the director can move around and experience the camera moving in 3D space. This can be helpful to get more interesting camera angles and perspectives.
PRODUCTION
Element Shoots
Sometimes shooting real-world elements like crowd, dust, fire and smoke against a plain background (black, white, grey, blue, green, or sky) which can be comped to create the final look of the shot without needing to cache out computationally expensive FX simulations.
Data Capture
Gathering proper lighting data will help enable the CG integration to match the 2D plate match. The data wrangler and on-set shoot team will gather data from the following:
Macbeth chart (colour grading)
Grey ball (light intensity)
Chrome ball (light source positions)
LiDAR scan (location spatial data measurements)
Witness cameras (extra video coverage for reference)
HDRI (determine where light is placed in a scene and how bright the lights can be)
Photogrammetry (3D scans created from multiple views 2D image for digi-double and props)
Slate data (lens, camera roll, time of the day)
Mocap
Performance of the actor is tracked and key frame data translated into a motion capture rig. The potential realistic motion will be used as a reference and will be cleaned up and refined by animation.
ASSETS
Data Processing
The data that has been gathered by the shoot department that were onset are stored onto the database. All the camera information for corresponding sequences are available for artists to take away the guess work.
Modelling
Different types of assets from the character, prop, hard surface/mechanical and environment that will be used and integrated on to the shot is built on 3D packages like Maya, Houdini and ZBrush. Ensuring that the model is usable downstream. For character modelling it must be realistic and anatomically correct which follows the topology that can be deformable when rigged and animated. Modellers also unwrap the UV layout that will be used by the texture artist. . There are usually 2-4 different versions of the model from it’s lowest LOD all the way to high LOD which can help to process the models depending on the tasks.
Texturing
Textures for assets are created using Mari and Substance Painter which are then projected on to the asset. High frequency density displacement maps are also created that define pores and wrinkle details of a character. Multiple channel maps extracted from TextureXYZ can be used to better match the textures on to the mesh and add even more realism and definition.
Groom
Guide curves are placed around the character to add groom data that are saved as different sets that will also create the deformable guides used to simulate the hair by Creature FX (CFX).
Lookdev
Lookdev artists are responsible for building preset shaders and put together the assets from texturing, model and groom that will be rendered out to create the desired look that matches the client’s vision. The lighting from the HDRI (high-dynamic range image) captured on set can be applied to test how the materials interact with the real-world reference.
Rigging
Skeletal systems are replicated for the purpose of building animator friendly controls to make animating assets easier.
SHOTS
Camera Tracking
Synchronising the position, orientation and lens of the camera used when element is captured creates a virtual camera. Software like 3D Equalizer and Nuke will figure out the depth from the triangulated points. Once lens distortion is removed and camera track solve generates point cloud data, the position of the camera relative to the spatial source that will match the LIDAR data to produce accurate results.
Body tracking
Lining up the body mesh to match every movement of every frame to track the performance of the character.
Roto
The roto artist is responsible for creating separate layers for the foreground and background elements using curves that are key framed so that the edges are clean and precisely keyed out.
Prep
This process involves using cloning and paint tools that clean up unwanted elements such as wires, tracking markers and shadows.
Layout
Staging the set that will take the reconstructed virtual camera information from the 2D plate. Placement of the CG assets according to the multiple camera elements to ensure that it is unified and in sync with the timing.
Animation
Bring to life the performance of the character movement using controls developed by riggers that are key framed in response to the action that takes place.
Creature FX
Once the performance from animation is translated onto the character, simulation of muscle, hair/fur and cloth is driven to enhance the performance and realism. Simulating crowd behaviours is also within the Creature FX scope.
FX
Dynamic FX simulations (e.g. destruction and water simulations) that replicate the real-world physical characteristics is achieved through building systems that calculate the results by tweaking parameters such as mass, density, elasticity, gravity and wind and running a solver (sets of equations) in Houdini. This is then cached out ready to be sent through to lighting.
Shotsculpt
Taking the character animation and CFX caches layered on top, the shotsculpt artist will fix and correct the intersections and try to preserve the volume and form of the character to help enhance the performance even further.
Environment/DMP
Digital Matte Painting (DMP) and Environment artists work hand in hand to layer over photography and painted elements that can be mapped on top of the original footage to enhance or even extend the environments. DMP mainly use Photoshop and Nuke to create 2.5 projection layered approach while Environment artists use 3D software such as Maya, Nuke and Clarisse to create instances of geometry and elements that can be added on to the shot.
Lighting
The on-set lighting data gathered from the shoot is referenced and recreated in 3D environment. Separate passes are set up and ready to be used by the compositor to tweak the values and intensity of the lit elements.
Compositing
Comp artists have full control and dial down or intensify the shot. Assembling together all the separate layers that can be enhanced and softened to create the client’s desired outcome.
ericka's blog 