Claroma

<!All of the writing are our original script, we used GEMINI to fix the grammar> Prompt we used “ fix the grammar and make our english better without changing the meaning"

Inspiration

Understanding human anatomy is a complex task. Our team bonded over a shared experience from our biology classes. While we found the subject fascinating, we all struggled with memorizing the different parts of the human body,specifically the bones, muscles, and their functions. We recalled our high school days of learning from paper textbooks. This method was adequate, but it was also limited and forgettable. We had to review the material multiple times, and our knowledge was confined to two dimensions. This experience inspired us to build our project, “Claroma”. It's a Virtual Reality (VR) application that allows users to visualize complex human anatomy in a clearer, more descriptive way. With the power of an LLM (Gemini), users can also ask questions about different anatomical images, creating a powerful new way to learn.

What it does

Users can interact with 3D models of human anatomy, which currently include the human skeleton, muscles, and heart. Using an Oculus controller, a user can easily pinpoint a region of interest and receive detailed information about it in a simple, easy-to-understand format.

How we built it

We started by collecting various 3D models of human anatomy. We used WebXR as the platform to render these models and developed functionality for interaction. We also integrated the Gemini Vision API to accurately identify anatomical regions and provide relevant, descriptive responses. The prompt we use for gemini in generating response was “”

Challenges we ran into

The ideation phase took up a significant amount of our time. Since we were working with 3D models, finding high-quality, free versions was a major challenge, and we went through several options before finding models that fit our needs. Beyond that, implementing our ambitious plan was difficult. We ran into technical problems, such as exceeding API request limits. While our project works well, we wish we had more time to implement a feature for selecting more specific anatomical regions.

Accomplishments that we're proud of

In a very short time, we learned how to implement a Large Language Model (Gemini) to simplify the complex process of learning human anatomy. We are proud to have contributed, even in a small way, to making education more fun and intuitive while decreasing the cognitive load on the learner. We applied best practices in Human-Computer Interaction to minimize the user's effort in the learning process.

What we learned

We learned how to collaborate effectively within a new team. Beyond the collaborative experience, we shared our knowledge and learned a new tech stack, WebVR, in a very short amount of time.

What's next for Claroma

We believe this concept can be developed into a scalable product. The MLH hackathon provided a platform to build and pitch our idea, and we see this as a foundational experience. We want to continue developing Claroma into a useful, scalable product that addresses the needs of students and educators.

Built With

• css3
• gemini
• html5
• javascript
• metaquest
• ocolus
• python
• vr
• webxr