Directing motion capture for an animated TV series

Motion Capture (mocap) is the process where actor performances are captured as motion data, then applied to computer generated characters. An 'optical' mocap stage is a large room lined with infrared cameras, which track markers on actors wearing black body suits.

Choice of mocap for an animated TV series

During pre-production of Saladin: The Animated Series the big question was whether to use mocap or to keyframe the animation manually. After much deliberation - it was decided to use mocap. Reasons were - the character designs were humanoid, it was an action genre that would require a lot of complex performance animation. After a lot of consideration and research, we concluded that mocap would be a time and cost effective solution for the volume of animation needed to make 26 x 24 minute episodes.

Bye bye cartoon physics

The decision to use mocap meant we would have to forego those appealing cartoony qualities like squash, stretch, anticipation, exaggeration and snap. Instead the animated characters would move with realistic human motion. While it's true that mocap'd motion can be edited, in practice it's not that easy to manipulate mocap motion curves (think spaghetti), and in a busy TV series pipeline where artists are under the pump to meet daily quotas, not a practical option. It was a big decision stylistically, that set the tone for the look of the show.

Casting for actors
 
Not all actors are interested in playing mocap characters. While they will get a screen credit, their face will never be seen, and their voice never heard. Actors will be acting on an empty stage and will often be acting parts alone - unable to play off other actors.
 
Generally actors will need to be pretty athletic, with good physical performance skills, and if they have other abilities such as martial arts or dance then they may be able to double up and play other characters as well.
 
Actors are essential - they are trained in performance. And the same goes for stunt performers and dancers - so don't skimp on your mocap talent.
 
Directing mocap actors
 
Directing mocap actors is more about choreographing movements in time and space, than extracting authentic emotional performances. With the task of capturing thousands of shots for a TV series - it's a production line.
 
Before performing a sequence - play the animatic to the actors. Brief them on what's happening in the story at that moment, the emotion in the scene and your vision for the scene. Rehearse the shot. Let the actor interpret your instruction - allow them to have input. Help to locate their starting and ending positions on the stage. Rehearse the shot until you and the actor are ready to go for a take.
 
Timing of actions is critical - the animatic should have already been cut and timed to dialog recordings before starting any mocap - or at bare minimum the storyboard completed. Sequences are therefore in effect pre-edited - so if a storyboarded shot has the actor taking three steps, turning left and jumping - in three and a half seconds - then that's what has to be performed. There's little room for improvisation - the actors are essentially performing what's already been storyboarded and voiced by vocal actors.
 
When the mocap technician is ready, she will call 'standby' then 'recording'. Like a film director, the mocap director calls 'action' and 'cut'. This protocol might vary from one studio to the next.
 
The mocap technician will review each take first. If the take was technically good - without corrupt data caused by occlusion (explained later) or other system limitations, then it will be reviewed by the director and actor to assess whether it's a keeper or not. If not, subsequent takes will be made until a good performance with clean data is achieved. Sometimes a continuous sequence of several shots with different camera angles may be captured in one take, and later in the animation department, the shot cut up into the different camera angles.
 
Mocap stages will have some basic props and performance aids such as platforms and boxes to simulate stepping or climbing motions; mats for actors to fall or tumble on, and other props such as weapons (typically simple non-reflective wooden or plastic proxies).

Limitations

The active area of the stage is a performance limitation - actions in the storyboard that need more space than the stage can provide will need to have workarounds figured out - such as cycling a running action or capturing big actions in parts.
 
Occlusion (in optical mocap systems) is when a marker is obscured from the cameras. Markers need to be visible by at least two cameras to triangulate their location in 3D space. Whether it's self-occlusion or occlusion by other players or props, the result is bad data and retakes. Generally, the more cameras and the fewer players on the stage, the less chance of occlusion. Unless you're on a Hollywood budget, you may be working in a small mocap stage with 12 cameras or less, in which case you can probably put only one or two actors on the stage at a time.
 
Staging

On a mocap stage there is no physical camera to compose the shot, and the actor/s may have no awareness of camera placement or shot composition. So the director has to imagine the performance as it will finally appear on the screen.

However, a well equipped mocap stage will have a virtual camera - a small hand held screen that acts as a viewfinder for the director to frame the shot. The scene is rendered in the screen in real time - and may also include virtual props and set. The director can use the virtual camera to get real time feedback and make composition, staging and lensing decisions. The virtual camera also has markers so it's position and movement can be recorded.

Mocap directors must understand screen direction and continuity if they are capturing scenes for continuity cutting. When all the shots are conformed in the animation timeline later, they must be continuous, otherwise it's back to the mocap stage for a retake, which means time and money (and an anxious producer).
 
Clean up

Selected mocap takes are sent for 'clean up' before they can be added to the animation pipeline. Autodesk Motion Builder is the standard app for cleaning mocap data. If you're lucky your data will be pretty clean, if not there will be jerks and bumps caused by occlusion, reflections or other system glitches that need to be smoothed out.

Mocap vs keyframe - conclusion
 
So did mocap make it easier or faster to make 26 half hours of animation? Well... no. Mocap added a whole other layer of complexity where things could go wrong. The time taken to capture 8,000 shots and the cost of the mocap stage, technicians, stage hands, actors, clean up artists, mocap directors could arguably have been better spent on more animators, resulting in a more traditional animated motion and simpler pipeline. End of the day the realistic motion didn't look out of place on the stylised human characters and the co-production partners were pleased with the outcome.
 
I'd love to hear about your experience with or thoughts about mocap - leave a comment.

Steve Bristow is an animation producer/director based in Melbourne Australia.