Creating Facial Expressions Synced to Dialogue
Creating speech may appear to be a difficult endeavor, but on inspection it's not
as formidable as one might think. English speech can be generalized into seven basic
mouth expressions as shown below.
Starting from the neutral expression:
- The mouth is held open wide in a big smile to form the A, I expression.
- The teeth are held together with an open mouth to form the C, D, E, G, K,
N, R, S, T expression.
- The lower lip is held against the upper teeth and the center of the upper lip
curves up to form the F, V expression.
To form the remaining dialogue expressions:
- The mouth is held partly open, with the tongue placed between the teeth, to form
the L, Th expression.
- The lips are held together when forming the M, B, P expression.
- The lips are in a puckered position when forming the O, Y, W expression.
- The mouth is in a smiling position while the teeth are held against each other
to form the Z expression.
Step 1 - Recording The Dialogue And Creating The
Sound Diagram
Planning a dialogue begins with recording the sound and then placing the words
underneath the audio tracks for use in determining the mouth position to be used
and the keyframe it is to be assigned to.
|
|
|
Illustration 1 - Creating the sound diagram
|
- Make a digital recording of someone talking and open the AIFF file in a program
such as Adobe Premier or Sound Edit.
- Take a screen shot of the sound plot (Command>Shift>3).
- Crop the screen shot in a program such as Adobe PhotoShop.
- Type the words underneath the sound plot. Space out the vowels and consonants
that are drawn out as shown in Illustration 1 above.
Step 2 - Creating A Model Of The Head
The next stage is the creation a model of the head to be used in forming the various
expressions that correspond to the seven basic mouth positions shown above. Refer
to the Face Sculpt example for a detailed description
of how to model the human face.
|
|
|
Illustration 2 - Opening the Modeler Morph Dialog
|
- Open Modeler and model a face with a mouth and a mouth cavity.
- Convert the model into a Morph object and select the model.
- Click the add button in the Morph dialogue box.
- In the Morph dialogue box, type the first set of consonants that the face will
pronounce. There are seven mouth positions that will cover every word in the English
language as shown in Illustration 2 above.
Step 3 - Creating The Facial Expressions
The next step involves reshaping the model of the head to create the seven expressions
defined above.
- After specifying the first morph target in the Morphing Object Parameters dialogue
box, shape the model's face into that position. You do this by selecting groups of
vertices and moving them as shown in Illustration 3 below.
|
|
|
Illustration 3 - Moving vertices to create a facial expression
|
- You might decide to use a hand held mirror in order to see what the different
facial expressions are.
- Continue setting all the morph targets by clicking the add button in the Morphing
Object Parameters dialogue box. It should look like the one in Illustration 3.
Step 4 - Setting the Lip Sync Animation Timing
With the information gathered from the timing diagram, and with the seven dialogue
expressions defined in the Modeler module, it's time to proceed to the Presenter
module and set up the animation.
- Open the model in Presenter and make sure the current frame is set at zero as
shown in Illustration 4. Then double-click on the Head object and drag the morph
plug-in into the Head object's Attributes box as shown in Illustration 5.
|
|
|
|
Illustration 4 - Setting time at frame = 0
|
Illustration 5 - Assigning the KineMagics Morph plug-in
|
- Open the morph module's dialogue box and drag the first facial expression's target
parameter to the right as shown in Illustration 5 below. This should match the first
vowel or consonant of the sound graph.
|
|
|
Illustration 6 - Setting facial expressions to match a dialogue.
|
- Option drag the morph module to the second frame of the animation (little red
square) as shown in Illustration 4 above.
- Open the second morph frame and set the next morph target according to the sound
chart.
- Continue setting animation frames according to the sound chart. Some animation
frames will be spaced out more than others due to the fact that some sounds are more
drawn out.
- Do a quick and rough preview animation in Presenter's Control Room with sound
turned on as shown in Illustration 7 below.
|
|
|
Illustration 7 - The Presenter Control Room set up for previewing
the sound and animation.
|
- Check the mouth movements with the sound to make sure they match each other.
Take notes describing parts of the animation that need to be adjusted - either extended
or shortened.
- Adjust the length of your animation.
- Continue rendering quick preview animations to see how the sound matches the
facial animation.
- Once you are satisfied with the animation, then do a final high quality animation.