MouthBrush

Draw with your voice!

Comment 1

Vocasso: The Voice-Controlled AI Drawing Game

Inspiration

Classic drawing games are fun, but we wanted to try something new. We wondered what it would be like to draw using your voice. We wanted to combine sound and art to make a game where you have to control how loud and high you speak to paint a picture.

What it does

This is a drawing game controlled by your voice. Here is how you play:

  1. Pick a word to draw.
  2. Google AI makes a simple reference picture for you to copy.
  3. You move your mouse to point, but your microphone actually controls the paint. How loud you are changes the brush size. How high your voice is changes the color. You can even pick different brush styles by making specific sounds.
  4. When you finish, the AI looks at your drawing, compares it to the reference, and gives you a score out of 10.

How we built it

We built the server using Python to handle the game and talk to Google AI. For the drawing board, we used standard web code like JavaScript and HTML. To make the voice controls work, we used tools built into the web browser to listen to the microphone. We created a setup phase that measures how loud and high your specific voice can go. This allows the game to translate your voice into colors and brush sizes accurately, no matter what you sound like.

Challenges we ran into

Browsers try to block long, steady noises because they think it is annoying background hum. This caused our voice controls to suddenly drop out. We had to write extra code to tell the browser to give us the raw, unfiltered sound. Some of our special brushes, like the glowing brush, were accidentally mixing their glow effect into the regular brushes. We had to carefully separate how each brush is drawn so the effects would not bleed together. Making a menu that you navigate entirely by changing your voice was difficult to program without the selection jumping around randomly.

Accomplishments that we are proud of

We are very proud of the voice setup system. Since everyone sounds different, building a setup phase that learns your voice makes the game work for anyone. We are also happy with how well the AI understands the messy drawings and gives accurate scores.

What we learned

We learned a lot about the hidden rules web browsers have for microphones and drawing boards. We also learned how to give better instructions to the AI so it acts like a real art judge instead of just describing the picture.

What is next for the project

We want to add funny comments from the AI alongside the score. We want to let players download their final drawing and score to share with friends. We want to add a hard mode where the reference picture disappears after five seconds.

Share this project:

Updates