Pictionary 64

Paint 3D sculptures in thin air with your phone while everyone else shakes their phone to guess, and an AI rival races you to the answer.

Comment

Inspiration

Pictionary is great, but it's always flat: a marker, a whiteboard, and everyone squinting at the same angle. We wanted to ask what happens if the drawing lives in the room around you instead. Every phone already has a gyroscope, so we turned the drawer's phone into a brush and built a party game around it. The reveal is the fun part. The sculpture first shows up edge-on and slowly spins into view while everyone races to guess it.

What it is

Sculpt & Spin is a 3D Pictionary party game for 4-12 players. There's no app to install. A laptop or TV runs the host screen, and players join from their phone browser by scanning a QR code.

  • The sculptor holds their phone like a camera and sweeps it to paint glowing 3D strokes, with a depth slider for how near or far the ink lands. 25 solid 3D shapes (cube, sphere, cone, ring, star, heart, knot, spring and more) can be stamped in: turn one with the X/Y/Z buttons or a wrist flick, hold to grow it, let go to place it. With 🧲 snap on, ink follows a shape's surface, so details can be painted right onto it. Swiping spins the sculpture on a turntable or tips it to reach the top and bottom, 👁 Peek shows it from the audience's side, and a shake undoes the last stroke or shape.
  • 24 ink colours, including greens, browns, skin tones and greys, plus a picker for any colour, so trees, animals and people look right.
  • About 1,000 words in 10 packs (Classic, Spooky, Penn State, Tech, Food, Animals, Sports, Around the House, Nature & Space, Fantasy). Words played recently in any room are remembered, so the choices stay fresh game after game.
  • Guessers shake their phone to unlock a guess. A wrong guess freezes you for 4 seconds. Guessers can also drag to orbit and pinch to zoom on their own screen.
  • Sabotage power-ups (fog, flip, dizzy, freeze, shield, secret hint) are thrown at rivals with a flick of the phone.
  • Gemini joins every game as an AI player. It takes three looks per turn, and afterwards reviews each sculpture like an art critic. ElevenLabs voices an announcer on the big screen.
  • At the end, the host renders every sculpture as a spinning, captioned take-home GIF.

The Mystery Stack (General Track)

  • Phone motion is the core interaction. Pointing and sweeping the phone is the 3D brush, a wrist flick turns a shape as it grows, and a shake undoes. Guessers shake to unlock a guess and flick to throw a power-up. You can't play without moving your phone.
  • Rivalry. The first correct guess takes the points, faster guesses score more, rivals sabotage each other with power-ups, and there's a live scoreboard and a podium. Even the AI is a rival you can sabotage.
  • Group play, 4+. Built for 4–12 players, each on their own phone. Everyone sculpts once per round.

Project status

The full party loop works end to end: lobby, turns, scoring, power-ups, the AI rival, the announcer and take-home GIFs. Both the drawing and the viewing are 3D: the sculptor can stamp solid 3D shapes and paint onto their surfaces, spin and tip the sculpture to reach every side, and peek at it from the audience's side, while guessers orbit and zoom on their own phones. What is still missing from our full vision is free movement. The brush follows the direction the phone points, with a depth slider setting the distance, instead of tracking where the phone is in the room. Next we want true 6-DoF sculpting, for example with WebXR tracking on Android as an optional "pro brush".

How we built it

  • Server: Node.js, Express and Socket.IO run one authoritative game state machine (lobby → choose → draw → reveal → podium). The server auto-generates a self-signed HTTPS certificate for the laptop's LAN IPs, because phones only expose motion sensors over HTTPS.
  • Phones as controllers: DeviceOrientation becomes a quaternion for the brush direction, and an on-screen slider sets the depth. DeviceMotion picks up shakes (to guess, or to undo while sculpting) and wrist flicks (to throw power-ups, or to turn a shape as it grows).
  • 3D rendering: Three.js draws glowing tube strokes on the TV, a spinning reveal camera, and a "magic window" view for the sculptor.
  • Turntable: strokes are stored in the sculpture's own coordinates. When the sculptor spins or tips the sculpture, new ink is mapped back through the inverse rotation, so the TV and guessers always see it in the right place, and the sculptor's marker moves around the sculpture as they spin it.
  • 3D shapes and snapping: 25 shared Three.js geometries (boxes, lathes, extrusions, a torus knot, a helix) are stamped with a rotation and a size, validated on the server and synced like strokes. Snapping raycasts from the brush onto the shapes, so new ink lands on their surface and follows it.
  • Real-time strokes: The drawer streams points in batches about every 50 ms. The server validates every point and relays it to the room.
  • AI: Gemini receives a snapshot of the sculpture and the word pattern three times per turn (at about 35%, 58% and 80%), skipping a look if nothing changed. It guesses with a one-liner and critiques the finished piece. If the free quota runs out, the game keeps going and shows the AI resting. ElevenLabs voices the announcer, with cached lines.
  • Take-home GIFs: The host browser renders each sculpture with gifenc and uploads it to the server. Phones download it automatically.
  • Testing: A bot script joins fake players that sculpt, guess and sabotage, so we could playtest alone.

Challenges we ran into

  • Phones can't track position. Double-integrating the accelerometer drifts by tens of centimetres in a second. We project the brush along the phone's pointing direction instead. Depth started as a wrist twist; we replaced it with an on-screen slider so depth is easier to see and control.
  • Secure-context sensors. iOS and Android need HTTPS plus a permission prompt for motion data, which led to the auto-generated certificate.
  • Reachable QR codes. Laptops have many network adapters (VMs, Docker, VPNs). We rank adapters so the QR code uses the real Wi-Fi address, and added a Cloudflare tunnel fallback for venues that block device-to-device traffic.
  • Clocks and drops. We send "time remaining" instead of timestamps. A dropped sculptor gets an 8-second grace period to rejoin.
  • Never trusting clients. The server limits who can draw, bounds coordinates, and caps stroke counts and sizes.
  • Fair guessing. Fuzzy matching, synonyms and plural handling keep the game fun without letting spam guesses win. Wrong guesses also freeze you for 4 seconds.
  • Drawing in thin air. With no surface, strokes wobble. We added undo (or just shake the phone), clear and recentering, plus solid shapes with surface snapping so there is something to draw on.
  • Reaching every side of a 3D object. The sculptor stands in the middle looking out, so the back, sides, top and bottom were out of reach. We added a turntable you control by swiping (sideways to spin, up and down to tip) and a peek view, which meant converting every new point back into the sculpture's coordinates.

What we learned

  • Sensor fusion and quaternion math in the browser, and why orientation is far more reliable than position on a phone.
  • Real-time multiplayer design with Socket.IO, covering batching, validation, reconnects and clock-independent timing.
  • Using generative AI as a game participant rather than a feature. Gemini as a cheeky rival and critic made the game funnier.
  • Scoping a hackathon project: get the party loop working first, then chase the full 3D vision.

What's next

  • Fully 3D drawing and viewing
  • Team mode with 2v2 relay sculpting
  • "Everyone sculpts the same word" voting rounds
  • Exporting sculptures as .glb files for 3D printing
  • WebXR 6-DoF sculpting on Android

How we used Gemini

Gemini is a player, not a chatbot. During each turn the TV sends Gemini a JPEG of exactly what the audience sees (mid-rotation and unfinished) plus the word pattern with any revealed letters. Gemini replies with a guess and a quip as JSON; if it's right it wins the turn and scores like a human, and players can throw power-ups at it to blind it. At each reveal Gemini acts as an art critic and scores the sculpture out of 10, and its reviews also appear in the podium gallery. The API key stays on the server, and Gemini only takes a look when the sculpture has changed.

How we used ElevenLabs

ElevenLabs gives the big screen a game-show announcer: the welcome, "you're up", the 3-2-1 countdown, correct guesses, Gemini's guesses and reviews, sabotage, power orbs and the winner are all spoken live. Lines are cached on the server so repeats cost nothing, and the browser's built-in voice takes over if there's no key.

Accomplishments that we're proud of

  • Drift-free 3D drawing in an ordinary phone browser, with no app and no AR framework.
  • The edge-on to front reveal, which turns a drawing game into a guessing race.
  • An AI opponent you can sabotage, which makes the AI part of the rivalry instead of a gimmick.
  • Painting every side of a 3D sculpture from a phone: 25 snap-on 3D shapes, a turntable you swipe to spin and tip, and a peek view.
  • It works on any phone that can open a web page, for up to 12 players.

Credits

Built with open-source three.js, Socket.IO, Express, gifenc, node-qrcode and selfsigned (all MIT), Google Fonts (Bricolage Grotesque, JetBrains Mono), the Google Gemini API and the ElevenLabs API.

Built With

Share this project:

Updates

Submission history