Was this video useful to you?
Rate this video from 1 to 5.
Learn how to import sound and apply lip-sync techniques. Using sample material, this lesson teaches you how to import a voice clip for lip-syncing. The Xsheet and cell swapping are discussed, and a demonstration of how to map lips to the sound file.
Welcome to the tutorial How to Import Sound and Add Lip-Sync. In this tutorial, I'm going to show you how to import a sound file into your Harmony project and then how to map the character's various mouth positions to make it seem like your character is saying the words in that sound sample. To begin, you should always prepare your sound file outside of Harmony in a third party software. If your sound file needs editing, it's not exactly the way that you want and we're talking more specifically here about dialogue as opposed to background music, but if it is background music or sound effects, those things should also be edited before you bring them into the software. There are several different ways that you can import a sound file into your scene. The first way is by going to the Top Menu and selecting File > Import > Sound. Another way is to the top menu here in the Timeline and by selecting Import > Sounds. Or you could go to the X-sheet View and use the top menu there where there is also a File > Import > Sounds option. Or what you can do is click on any of the cells of a given column and select Import > Sounds. Then the Select Sound File dialogue box opens. The three formats that you can import are WAVE, AIFF or MP3. I'm going to select this superhero MP3 sound file and then click on the Open button. You'll notice here in the Timeline that the superhero sound layer has been created and we know it's different from the other layers because it has a little speaker here because of the note that denotes the type of layer that it is, and that we see the wave form here in the right side of the Timeline. If we click on it, we can also see that it exists here in the X-sheet View. If you right-click on any of the cells of this column, you can select what you view in the cells. Right now, we see the sound name, but we can also see the wave form if we like. If we do that, we see a vertical display of the horizontal wave form that we see here in the Timeline View.
The sound file will play until the end of the file. In this case, I'm ending the scene at 75, so it'll stop at 75, but in fact the sounds goes for much longer. You can see here, around frame #110 or #111, the sound continues. Or if you double-click on the sound file, you can open the Sound Element Editor and here you can choose your start and your stop frame, so you can stop it a bit sooner if you want. In order to listen to our sound, we actually have to enable this button right here, because right now if I just play it, you'll see that you can't hear anything. In fact, one other thing that we have to do, let me just slide my red playhead back, is disable the peg for the camera. That's why the superhero character disappeared from our Camera View because if you recall from the last tutorial, we created a camera zoom-in. So if we deselect the camera peg, we'll still be able to see what's in the Camera View, but all of that movement, that zooming in, will be disabled. Like I was saying, in order to hear the sound, we have to click on the Sound button and now if you click on the Play button, you can hear what is being said. If you click on the Sound Scrubbing button but unclick the Sound button, if I press Play, you won't hear any sound, but if I grab the red playhead and I scrub across the wave form, you can hear the various syllables or phonemes of the sound, so per frame what sound is being made. That's called sound scrubbing. You can also have both selected so you can both listen to the sound and you can scrub as well. That's what these two buttons here do.
Now let's take a look at the hero mouth positions. What I'm going to do is just scroll the playhead back to frame #1 and actually turn off the Sound and Sound Scrubbing buttons. Then I'm going to select the Transform Tool and then select the mouth here in the Camera View. As you can see here in the X-sheet View, every drawing, as I scroll across, has been ascribed a different letter. This is mouth position A, this is B, this is C, etc. all the way down. Actually, what I'm going to do is go to the Library View and you can also see these, what we call drawing substitutions or different drawings for the same body part, here in this little window, as well as the name of the drawing here in the little field below. Of course, you can scrub across here to change the drawing on the selected frame. Let's find the layer in the Timeline View for the hero lips by clicking again in the Timeline to put the focus around the Timeline View and then click on the Center on Selection button. It takes us first to the torso and then you can always click it again or use the keyboard shortcut [O] to bring us to the lips. As you can see here, there are eight drawings. If we scroll across, we'll see the hero mouth position move and you'll also see it move across here in the drawing substitution window. Let's also go to where this drawing module is in the Network View. For that, we have to enter the hero group, we have to enter the head group and then the facial features group and it's right here. Let's zoom in a little bit to take a look at it, so the hero lips.
The way that I created extra lip or mouth positions for the hero was by selecting the lips drawing column in the X-sheet View. I then entered the new drawing names, so the mouth position names, such as A, B, C, D, etc. in the first nine cells of the lips column. I then went to the Drawing View and using the Drawing Tools plus the Onion Skin feature, I drew the other mouth positions. You might be wondering how I knew how to draw these various mouth positions for the three-quarter profile view of our character. I didn't reinvent the wheel. There's already an industry standard for the various mouth positions that represent, what we call, the phonemes of, in this case, the English language. I believe the phonemes vary between languages, but I obviously chose to draw the eight phonemes that exist for the English language. Even though we ascribed letters as names to these various positions, and let me select that again so you can see the letters appear here in the drawing substitution window of the Library View, this mouth position does not represent the character saying the letter "B" or making a sound like "buh" or "bee". If you go to your Help Menu here at the top, you could actually go to the Online Help and it brings you to this page eventually. It'll actually open up a web browser with a list of different help documentation that you can access for Harmony and I chose the Getting Started Guide. Here, in the How to Import Sound and Add Lip-Sync, you can see that there's a chart for the various mouth positions that exist for a character. You can use this to create your own mouth positions for your character, for example, "A" is a closed mouth, "B" is slightly open, "C" is a little bit more open and you can see more of the tongue, etc. If we scroll down a bit here, you can see an approximation of what each mouth shape is supposed to be producing. "A" is actually supposed to represent the m, b, p and h letter sounds. People that have been doing this for a long time have this chart memorized by heart, so they could just flip through the sound and know that if the character says "what let's go", then they might put in an F for the "ooh" sound, a D for the "uh" sound and then a B for the "tuh" sound to get "what". They could do it manually in the X-sheet by entering in these letters where they hear the sound using the sound scrubbing. That's the manual way to do it. However, this is an auto-lip sync detection feature in Harmony and let's take a look at that.
Right now in the hero lips layer, you can see all of the eight drawings and I believe there's an extra one, there's "I" as the extra drawing position, which is this one right here. But this was just to show you what I had drawn in all these different cells. As soon as we do the auto-lip sync, these will all disappear and be replaced by what the software believe should be the correct position for the sound file that we imported. Let's scroll up and go back to our sound file. Let's select the layer and then click somewhere on one of its cells on the right side of the Timeline and select Lip-Sync > Auto Lip-Sync Detection. You'll notice that there's a little progress bar there at the top and that's just Harmony processing the sound and then assigning the lettered phonemes to the sound. Then what we're going to do is right-click again somewhere on the right side of the Timeline on the sound layer and then select Lip-Sync > Map Lip-Sync. In the Lip-Sync Mapping dialogue box, you can see that our source layer has been accurately selected and that's the sound layer, the superhero sound layer. Then for our destination layer, instead of Helmet, from the drop-down, let's select Hero_Lips and here you can see a similar mouth chart to the one that we saw through the Harmony Online Help. Once again, you can see that the letter represents a specific image and these letters here are actually the letters of the various mouth position drawings on our Hero_Lips layer. For example, if we hadn't labelled them A, B, C, D, etc. and we had named them 1, 2, 3, 4, 5, then what we'd have to do to put those numbers in as the drawing names for our Hero_Lips layer, so that Harmony knows that the phoneme A will be mapped to drawing #1 on the Hero_Lips layer, etc. I'm just going to click on the OK button and now if we scrub across, you can see that lip positions have been assigned to the sound. Let's enable the sound and then click on the Play button to see what that looks like.
If for reason you think that the job that the Auto Lip-Sync Detection did was not accurate or you think that you could change the mouth position make it look a little bit more obvious as to what the character is saying, you can always go to the X-sheet View and change the position that was automatically given to this specific cell, this specific frame. In other words, you can use the Lip-Sync Detection as a basic once-over, a general pass over the sound to block in the basic mouth positions and then you can go back through and refine to make sure every word is being enunciated properly. That's it for the tutorial How to Import Sound and Add Lip-Sync. Stay tuned for the next tutorial, How to Add Effects to your Scene.