MindLabs

• 

  chemitry lab, gesture controlled, experiments

• 

  heart, human anatomy lab

Inspiration

Inspiration - The Inspiration for MindLabs came with my visit to a government school where lack of facilities made me realise that a basic necessity like a laboratory for students to learn is so hard to actually implement in many parts of our country and I thought to myself, what If I could bring a Lab to students without it physically being there!

What it does

MindLabs is a combination of advanced CV architecture and gesture controlled system that enables students to actually perform experiments, look at parts of human body, in a one of a kind interactive environment. Students can actually pick up test tubes, Mix elements, see reactions happen, Talk to the AI in their own local language and it will talk back like a friend and answer their queries. An experiment can be performed million times over without worrying about wastage of chemicals and hazards of real apparatus

How we built it

MindLabs's core driving engine is mediapipe by google, which is a state of art hand and face recognition system, paired with an LLM of choice (used GROQ for protoype purposes) and build on a simple react+vite architecture, so, it's device agnostic and works on laptop, tab, phone.

Challenges we ran into

The biggest challenge was making it feel as realistic as possible. To actually feel picking up, smoother gestures, chemical changes etc.

Accomplishments that we're proud of

I am proud of being able to make it smoother, ensuring the entire laboratory actually Follows Government instructed curriculum, so no student is left behind.

What we learned

Learned more about browser side rendering based projects, Usage of framer motion for smoother functioning, LLM integration with eleven labs for seamless communication

What's next for MindLabs

MindLabs aims to help every student learn science the way it was intended. We aim to democratize education for everyone in this country. This hackathon is one of the first places Mindlab is being features since it's initial development. A lot of changes have been made for the Hack47 submission of MindLabs, And to truly understand the beauty that Mindlabs, is to experience it.

Built With

• cv
• javascript
• llm
• react
• stt
• three.js
• tts