|
In Starcraft 2, you'll notice that each unit portrait accurately lip-syncs with the the dialogue that they say. For instance, when a marine says 'BY THE NUMBERS, BOYS', the marine's lips actually move to form those words.
This is vastly different from the ones in Broodwar and War3. In Broodwar, The portraits were just little FMVs that runs a pre-recorded animation whenever the units talks. In War3, the portraits were in real-time 3D just like in SC2, but the characters' mouths just arbitrarily open and close, poorly mimicking the quotes that they were saying. Only in SC2 do the portraits feature almost perfect lip-synching.
So I was wondering if the lip-synching is manually done by Blizz devs (animation done almost entirely by hand) or do they auto-sync with any recognizable human voice (the game automatically scans the voice input and auto-animates the mouth accordingly)?
|
I believe the animations are manually created. I don't think there is any speech recognition engine that can properly guess at mouth movements based on sound.
|
On April 29 2011 03:10 artanis2 wrote: I believe the animations are manually created. I don't think there is any speech recognition engine that can properly guess at mouth movements based on sound.
I'm pretty sure blizzard said that they have a system like that in place, it certainly hasn't seen much use in custom maps to my knowledge, however.
|
On April 29 2011 03:10 artanis2 wrote: I believe the animations are manually created. I don't think there is any speech recognition engine that can properly guess at mouth movements based on sound.
No way it was done manually -- do you realize how long it would take, especial with all of the languages that are released for the game? There are definitely algorithms imo.
|
The way this stuff works is somewhere in between manual and algorithmic. They make a model of the marine's face as it pronounces each Phoneme. They then just play the animations for each phoneme in the output phrase sequentially, probably with some blending in between.
|
I saw a documentation about it, there are definatly problems, that make the lip sycnin pretty easy. The animations are definatly NOT hand made
|
On April 29 2011 03:13 rbkl wrote:Show nested quote +On April 29 2011 03:10 artanis2 wrote: I believe the animations are manually created. I don't think there is any speech recognition engine that can properly guess at mouth movements based on sound. No way it was done manually -- do you realize how long it would take, especial with all of the languages that are released for the game? There are definitely algorithms imo.
yeah it took 12 years
|
Yeah, that would be absurd to individually animate. I would imagine there's some rudimentary algorithm in place that selects from 5-20 short animations and strings them together according to whatever the spoken sentence is. For B/P/M have its lips closed. For F/V have upper teeth touch lower lip. For W have lips rounded and almost closed. For basically every other consonant have mouth slightly open. Similarly, each vowel sound matches up with a different degree to which your mouth is open and lips are rounded. (Tongue position is effectively invisible). Text-to-speech engines can take a transcript of the sentence and translate it into phonemes, and a basic flowchart along the lines of what I just said can turn the string of phonemes into a string of animations. Different text-to-speech engines for different languages, and BAM, roughly lip-synched portraits!
|
Team Fortress 2 has the "Meet the Heavy/Spy/etc" videos in many different languages. If you watch them, you'll notice that they have unique lip animations for each language. Valve has used this voice/animation system for years- I'm sure its not out of Blizzards reach to do the same.
Edit: Let alone, as someone else said, all the languages SC2 has been released in.
|
I think some people might be considering certain things automatic and certain things not automatic. What Blizzard might very well have is a system where you TYPE what the unit says, and the time it takes for the unit to say it (possibly with other details such as for pauses), and the system will do the lip syncing based off that. I don't think automatic audio alone could work too well since you then have to deal with voice recognition, and when there's background nice, different accents, different languages, and all sorts of stuff, it is problematic. It would just be too much work to get it do do that accurately I'd say.
On April 29 2011 03:24 Nokarot wrote: Team Fortress 2 has the "Meet the Heavy/Spy/etc" videos in many different languages. If you watch them, you'll notice that they have unique lip animations for each language. Valve has used this voice/animation system for years- I'm sure its not out of Blizzards reach to do the same.
Edit: Let alone, as someone else said, all the languages SC2 has been released in. What is your point, though? are you saying that that's reasoning for a voice-based system or not? Places like Blizzard and Valve have multi-million-dollar budgets just like the movie production companies which make entire movies with lip-synched animation — it makes it very likely in my opinion that it's not using a voice-based system, and even if it was, it would still probably be touched up.
|
On one hand, it probably was a tough job to animate the faces of most of the Terran units and some of the campaign units.
On the other hand, Blizzard's job was probably eased by the fact that Protoss don't have mouths and Zergs make unintelligible noises, so they didn't need to worry about at least 2/3 of the unit portraits.
Plus, the Marauder and Raven portraits technically don't have mouths, so that probably made their job even easier.
|
|
On April 29 2011 03:13 rbkl wrote:Show nested quote +On April 29 2011 03:10 artanis2 wrote: I believe the animations are manually created. I don't think there is any speech recognition engine that can properly guess at mouth movements based on sound. No way it was done manually -- do you realize how long it would take, especial with all of the languages that are released for the game? There are definitely algorithms imo.
Not very long once you realize that they are just made up by chaining the same short movements together. They make 20 of those and then the unit can be lipsynced to anything.
|
Advancements have been made in animation that allows this to be done easier than it was in the past. One method is actually recording someones mouth that has been rigged with tracking points. Usually a high contrast sticker. Then take the numeric information and attach it to the respective points on your 3d model. The model itself is also complex in its rigging to allow for proper bending and such.
I would wager this is how they did it. A lip syncing software would be overly complex even if such a thing exists.
|
iirc blizzard used a lip synching software that does it automatically. i think they said it was included in the galaxy editor aswell. This is from blizzcon.
|
It can not be all hand-made.. There are much older games out there where lip-syncing is done much better than in sc2.. Legacy of Kain comes to mind.. And in that game, the animation is probably hand made; if in sc2 it would be tha same it should be perfect to the point of you knowing what the characters were saying with the sound turned Off.. And it isn't the case.. It is based on some algorithm, and a rudimentary one at that..
|
|
|
|