idontwantanipadbaby

An AI that recognizes children's hand drawings and give suggestions on how they can improve on their drawing skills based on their voices' inputs.

Comment

Inspiration

The idea for idontwantanipadbaby came from a desire to provide children with a fun, interactive, and educational drawing experience that doesn’t just rely on screens for entertainment. We wanted to create a virtual teacher that goes beyond just instructing; it engages with kids in a meaningful way, helps them express their creativity through drawing, and provides constructive feedback to improve their skills. The goal was to design an AI-powered tool that encourages hands-on activity while fostering artistic development.

What it does

idontwantanipadbaby is an AI-powered virtual drawing tutor that interacts with children to help them learn how to draw. The process involves:

  1. The AI asking the child what they would like to draw.
  2. Taking the child's voice input to understand the drawing concept.
  3. Waiting for the child to draw the chosen concept.
  4. Analyzing the hand-drawn sketch and providing feedback on its accuracy and aesthetic qualities using a LLM.
  5. Offering suggestions on how the child can improve their drawing skills using Voice AI text-to-speech.

How we built it

We utilized a variety of tools and technologies to bring this project to life:

  1. Deepgram API for live voice transcription, converting the child's spoken input into text.
  2. Groq API to process the transcribed text and understand the drawing concept.
  3. JavaScript and React for the front-end interface, providing an intuitive and interactive experience.
  4. Node.js on the back end to integrate various API calls and manage real-time interactions.

Challenges we ran into

Real-time Voice Processing: Getting the live voice transcription to work smoothly was challenging, especially handling background noise and ensuring accurate speech recognition. Drawing Analysis: It was tricky to evaluate freehand drawings in a meaningful way. We had to fine-tune the drawing analysis to balance between providing helpful feedback and encouraging creativity. Integrating Multiple APIs: Combining different services and ensuring smooth communication between them was difficult. Each API had its own requirements, and coordinating these services to work seamlessly took some effort. User Experience for Young Children: Designing a user interface that is simple enough for children to use, yet robust enough to support our AI’s functionalities, required careful consideration.

Accomplishments that we're proud of

End-to-End Functionality: We successfully built a prototype that guides a child through the entire drawing process, from voice input to drawing analysis and feedback. Seamless Voice Interaction: We achieved a smooth voice-to-text and text-to-feedback loop using the Deepgram and Agora APIs, enhancing the interactivity of the project. Impactful User Interface: The user interface was designed with young users in mind, making it visually appealing and easy to navigate.

What we learned

AI Integration in Education: We gained valuable insights into the challenges of using AI to teach creative skills like drawing and how to provide constructive feedback in a way that encourages rather than discourages. API Communication: Working with multiple APIs taught us how to effectively manage API calls, error handling, and asynchronous programming in a real-world project. Voice and Visual Processing: We improved our understanding of voice recognition and image analysis techniques, especially in adapting these technologies for young users.

What's next for idontwantanipadbaby

Enhanced Drawing Analysis: We plan to improve our drawing analysis algorithms to better assess sketches based on style, color use, and overall composition. Expanded Educational Content: Adding more topics, fun facts, and interactive lessons could make the experience richer and more educational. Voice Customization: We aim to personalize the virtual tutor's voice, allowing children to choose different voices and even languages for a more tailored experience. Parental Dashboard: We are considering adding a dashboard where parents can monitor their child’s progress and see how their drawing skills are developing over time. AR Integration with Snap AR: We want to explore augmented reality to make drawing even more interactive, such as projecting a 3D model of the concept for children to follow.

Built With

Share this project:

Updates