DesignFlow XR

DesignFlow XR — Create smarter, design faster with gestures and AI.

Comment

Inspiration

Design tools are powerful but often cluttered and unintuitive for quick ideation. We wanted to create a hands-free, distraction-free creative space — combining gesture recognition and AI assistance — to make design flow as natural as hand movements and voice commands.

What it does

DesignFlow XR is a web-based design workspace that allows users to:

Manipulate design elements using hand gestures via webcam

Use voice commands for actions like generating elements or changing properties

Generate color palettes, moodboards, and layout suggestions using AI-powered APIs

Experience a minimal, responsive, and animated UI designed for fluid creativity

How we built it

React.js + Tailwind CSS for building the frontend UI

MediaPipe Hands to detect and map real-time hand gestures

Web Speech API for voice command recognition

Unsplash, Coolors, and Google Fonts APIs to fetch moodboards, palettes, and font suggestions

HTML5 Canvas and Konva.js for interactive design canvas

Framer Motion for smooth animations and UI transitions

Challenges we ran into

Gesture mapping was tricky — ensuring accurate interpretation of hand poses in real-time

Combining gesture and voice control without interference or lag

Managing asynchronous API calls and live feedback loops in a seamless UI

Limited time meant we had to prioritize features for the hackathon

Accomplishments that we're proud of

Built a gesture-powered design tool from scratch within 24 hours

Seamlessly integrated multiple APIs for moodboards, fonts, and palettes

Created an intuitive, animated, and fully responsive UI

What we learned

Hands-on with MediaPipe, Konva.js, and real-time gesture tracking

Managing real-time data streams from webcam and microphone

Handling multiple asynchronous processes with performance in mind

The importance of UI/UX clarity in tools meant for creativity

What's next for DesignFlow XR

Add multi-user collaboration via WebSockets

Integrate AI-based layout generation from text prompts

Allow saving/exporting sessions and design assets

Launch as a browser extension or plugin for existing tools like Figma or Canva

Built With

  • ai
  • animations
  • apis
  • canvas
  • colormind
  • css
  • css3
  • fonts
  • framer
  • framework
  • gesture
  • getusermedia()
  • github
  • html5
  • images
  • javascript
  • konva.js
  • mediapipe
  • moodboard
  • motion
  • react.js
  • real-time
  • recommendations
  • tailwind
  • tracking
  • unsplash
  • vite
  • webcam
Share this project:

Updates