Our team of four designed a web app that streamlines the process through which 2d animators create lip sync animations. Lip sync animation is a technique that animators use to create realistic speaking animation. Instead of using a generic open mouth to represent vocalization, animators can use specific mouth shapes to represent individual phonemes. This technique is known as lip sync animation.

Typically, animators must animate every mouth movement, one frame at a time. Animators must also ensure that the mouth movements match the audio. A single minute of lip sync animation can potentially take half an hour to animate. This process is not only monotonous but also extremely time intensive. As a result, most large-scale production companies do not use lip sync animation. Lip sync animation is simply too time intensive to meet deadlines and keep costs low.

We want to streamline lip sync animation by automating the process. The goal is to produce lip sync animation with a set of images and an audio file alone. The set of images should depict all the most common phonemes. The audio file should contain the dialogue. The web app will convert the audio file to text, extract the syllables, match the syllables to the images, and then output a video file.

Share this project: