gestro

• 

  #1 Original input (drawn with hands)

• 

  #1 AI generation with bg (selected theme fantasy world)

• 

  #2 Original input (drawn with hands)

• 

  #2 No bg AI generation (selected theme cartoon)

• 

  #3 Original input (drawn with hands)

• 

  #3 AI generation without bg (selected theme Renaissance)

• 

  User's view on his projects (all saved and can be continued for iteration)

Inspiration

I love drawing but my skill never matches what I picture in my head, and AI tools made it worse by taking over the whole picture from a text prompt. I wanted the opposite: a tool where I stay the artist with my hands, and the AI just polishes what I already drew.

What it does

gestro is a website where you paint with your hands in front of your webcam, using up to ten finger cursors across both hands and six different brushes. When you like your sketch, the AI refines it without changing your idea, and you keep painting on top in a loop that never ends until you say it is done.

How I built it

I used Next.js, React, and TypeScript for the site, with a custom brush engine on Canvas2D and Google's MediaPipe for hand tracking, plus a smoothing layer I wrote so cursors glide instead of shaking. The AI side runs through fal.ai, with Firebase for login and Cloudflare R2 for storing pictures cheaply.

Challenges I ran into

Raw fingertip data is shaky, so my first lines looked like heartbeat graphs until I tuned the smoothing enough to feel like real brushstrokes. Getting ten fingers to paint at once without mixing up their brushes, and making the AI feel fast instead of like a loading screen, both took multiple rewrites.

Accomplishments that I am proud of

I built a real brush engine from scratch and got ten hand cursors painting smoothly at the same time, solo, inside the hackathon window. The result feels like a calm art app, not a flashy AI demo, which is exactly the tone I wanted.

What I learned

Feel matters more than features: a brush that is technically correct but slightly off on smoothing feels broken, no matter how pretty the UI is. I also learned that giving the AI less control is more fun, because people want to be the artist and the AI should just help.

What is next for gestro

Next up is a philanthropy track with free access for kindergarten teachers, so kids can improve their fine motor skills and get familiar with AI image generation in a fun, hands-on way. I also want to add simple monetization through small credit packs, so real users paying a little can keep the app online and free for the classrooms that need it.

Built With

• firebase
• mediapipe
• next.js
• typescript