Inspiration

This product was not our idea. It was our users' idea. Actors use ActinGym today to get feedback on their audition tapes before submitting them. They kept asking for help preparing, too:

  • "Please help me get ready before I walk into the room."
  • "It's hard for me to get into the scene."
  • "I need a way to rehearse without depending on my partner, my family, or paying a reader."

An audition can arrive with a deadline that same evening. An actor needs to learn the lines and step into the scene, often while rehearsing alone in a bedroom.

Finding someone to read the other character’s lines takes time. Friends and family may be busy, and hiring a reader adds a cost to each audition. Even with a reader, the actor still has to imagine the space around them.

VR gave us a way to address both needs. A headset can place an actor inside the scene with an AI partner who knows the script and can rehearse as long as they need. This competition gave us the push to bring that idea into ActinGym.

What it does

ActinGym Rehearsal Room is the first VR rehearsal room built for actors. You put on the headset, step into the scene, and rehearse with a scene for as long as you need it.

  • The Studio: You arrive in a rehearsal room. A corkboard holds your scenes. On the front wall there is a stage door.
  • Choosing the scene: Pick a script (or add your own), choose who you're playing, select a scene partner, and an environment: a theatre stage, a principal's office, and so on.
  • Generate a place: Ask for the place your scene needs. In this build, it opens our candle-lit restaurant; any place you can describe is what we are building next.
  • Step into the scene: The stage door opens, and you can enter the scene.
  • Scene Partner Interactions: As you practice your lines out loud. Your partner hears you, answers with their own lines, and reacts with their face and body language.
  • Script Customization: Depending on where you are in the memorization journey, you can practice with the full script in front of you, the lines floating next to your scene partner's face, or with no script at all.
  • Raise your palm up: If you forget a line, raise your palm up, as actors do in every rehearsal room, and your partner quietly gives you the next few words.
  • Scene reviews: After each run, you see what really happened: which words you dropped, how quickly you came in on your cue, and whether your eyes were on your partner or on the page. You can replay the take from your own seat.

This flow helps you learn your lines without asking for favors, allowing you to run the scene thirty times at 2 a.m. and your partner never gets tired.

How we built it

  • WebXR, built with Three.js and Vite.
  • Hand tracking: for every action: pinch to press, an open palm, and the raised hand that calls for a line.
  • Speech: Audio from the headset microphone is split into phrases and sent to Wit.ai, Meta's speech service. Our AI agent line tracker follows the actor through the script, word by word.
  • The partner's voice: Our AI agents use xAI text-to-speech through a server function, and its timing drives the partner's mouth.
  • The partners are rigged characters we built in Blender, with their own gestures and expressions.
  • The places: The Studio and the restaurant are modeled and lit in Blender. The other places are generated by our AI agents to fit in the headset and fill the users' requests as accurately as possible,
  • The notes: come from a recorder that logs every line, pause, head turn, and hand movement, and reports only what it measured.

Challenges we ran into

  • We built our own listening path from the raw microphone for Text-To-Speech capture.
  • Making the partner feel like a person: They must wait while you think, say their lines on time, and never talk over you.
  • Putting a person inside a picture. A 360° picture has no floor and no distance. We had to turn and place each one so the eyeline and the space feel true.
  • A whole app with no controllers: Every screen had to work with a pinch, so we turned the menus into objects in the room: a cork board, a gate, a page.
  • Honest notes: Actors do not want a machine to judge their talent. We report facts only.
  • We worked to keep the app responsive during rehearsal so scenes don’t freeze. Making characters show believable emotion is still a challenge.

Accomplishments that we're proud of

  • The full loop works with hands only: Actors can choose a scene, rehearse, request a line, and review their take using only their hands.
  • The stage door: The moment it opens and the bedroom becomes the scene is the moment people understand the product.
  • The raised hand: One of the oldest habits of the rehearsal room became a natural gesture in VR.

What we learned

  • Being inside the place does half of the actor's homework: People get into the scene in seconds instead of minutes.
  • Actors want facts, not scores
  • Privacy is a feature: Alone in a headset, actors try things they would never try in front of a friend.
  • VR and actors are a natural match. Their whole craft is believing in a place that is not there.

What's next for ActinGym Rehearsal Room

  • Scene Generation: Generate scenes for any place you can describe.
  • Partner Generation: Generate the character you are playing against.
  • Close the loop with ActinGym. Rehearse in the headset, read your notes, film your tape, and get ActinGym's analysis of the take you are about to send. Train, improve, submit, all in one place.
  • Your ActinGym account in the headset: your scripts, your takes, and your progress.
  • More scenes, and scenes with more than two characters.
  • A full release on the Meta Horizon Store.

Better rehearsals mean better auditions, and better auditions mean more callbacks. That is what our users asked us for.

Built With

+ 3 more
Share this project:

Updates

Submission history