X.R.E.S. (Extended Reality Education System)

• 

  Web application flow

• 

  peoples need of this function and what all solutions we are providing

• 

  peoples need of this function and what all solutions we are providing -2

• 

  peoples need of this function and what all solutions we are providing -3

• 

  UI 3

• 

  UI 1

• 

  UI 2

• 

  Landing page

• 

  Future Idea:a tablet made up of arrays of telescopic tubes which have electromagnets and can be used as a braille smartphones and visualizer

Inspiration

Questions like How online education have been faring for students who are blind,deaf, have cerebral palsy, etc., How does student with stuttering/stammering /lisping engage in classroom ? How does an adhd kid ensure productive and fruitful online education? Were our inspiration for research and finding solutions for people whose daily life may have influential impact from them.

What it does

Our System provides platform with technology using extended reality and speech recognition system for two products ,augmented learning platform and stuttering/stammering teletherapy Platform . Our augmented learning platform provides assistive tools and services which if scaled could revolutionize education landscape ,paving way for a more inclusive and accessible education . Presently we have included augmented 3d models to assist educators and students to create a more fun and better learning environment especially for kids with learning disability this tool would have a great impact.Our stammering teletherapy product aims to provide a platform to identify their weakspots causing stammering and provides personalized assistive practice and test exercises .

How we built it

Augmented Learning system was created using c++ and 3js , we used mediapipe for hand detection and tracking it. For stammering teletherapy app we referred many top tier research papers and implemented few of them leaving a large future scope for work like stuttering detection using mouth and face movements and using augmented reality to correct those wrong movements.

Challenges we ran into

Audio processing was new topic for us and it took a great amount of time to understand it. Mediapipe 3d interaction was difficult for us to crack in our augmented learning system for interacting with models.

Accomplishments that we're proud of

Our stuttering teletherapy backend successfully identifies words,syllables and the severity of few syllables .Future scope of this could be to generate hyper personalized exercises for improvement.

What we learned

First and foremost is team work and collaboration , forming team with strangers and getting to know each other ,learning about each other experiences ,learnings ,it aided in great team work and collaboration

What's next for X.R.E.S. (Extended Reality Education System)

Apart from polishing our current products we have designed a revolutionary hardware solution for blind people . Solution consist of a grid of thin telescopic tubes which works on electromagnetism for telescopic movement , these grids of tubes together form a tablet interface will act as a virtual interactive screen for smartphone using braille and even for 3d textures and images it will simulate them capturing depths and also create 3d visuals.

Built With

• c++
• fastapi
• figma
• git
• github
• machine-learning
• mediapipe
• postman
• python
• react-router-dom
• react.js
• tailwindcss
• three.js