Magic Leap, Unity, IBM Watson Unity Plugin (Speech-to-Text), Free models from Unity Asset Store.
Modern schools are still based upon an old factory paradigm that struggles to meet the individualized needs that each student deserves. Schools won't go away anytime soon. Student variation will never go away. With In. Class, it is the boring, uniform classroom that goes away.
We started with a North Star of helping english learners. Research shows the students learn a language best when shown closed-captions below a video. We wanted to add this to real-life transcription to classrooms. Once we brought that technology to the student, it opened a rich world classroom possibilities. IN.CLASS is accessible by providing closed captioning, by tracking focus, and by facilitating truly differentiated learning.
IN.CLASS is an immersive in-classroom tool for students and teachers that helps with focus and learning. By providing unique tools, such as closed captions and 3D visual aids, it increases students comprehension and eases the teaching burden. While teachers can review the engagement level of the students, real-time transcriptions of the course support english learners. 3D tools let teachers push 3d objects to helps everyone better learn the material. Differentiation lets a teacher push, say, a 3D heart to each student's desk, but each students may receive different models of varying complexity, depending upon their own learning plan.
What it does
IN.CLASS is a platform that helps both the teacher, and the student.
IN.CLASS brings augmented reality to the classroom, and gives teachers a control panel to monitor student progress, follow lesson plans, and bring Mixed-Reality-aware tools to their fingertips.
Students see their desks transformed from a boring slab, into a dynamic workbench. The student can receive pushed learning-objects from the teacher. Items can range from simple quiz questions, to personalized learning aids, such as 3D visual aids and experiential simulations.
Feedback tools, such as automatic attentiveness tracking, give teachers ways to more easily tailor the the class in real-time.
- Real-Time closed-captioning of the teacher
- Teacher initiated on-desk quizzes
- Teacher initiated 3D objects to the desktop
- Attention tracking of students, relaying to teacher
- Built-In Differentiated Experiences (The teacher could push a 3D heart to the class, and AP students would receive a more detailed model, but standard student could receive a less details model appropriate to their own learning plan).
- Each student has a different set of needs, and can receive a unique experience, depending upon their situation.
- English Learners can receive real-time transcription
Deaf students could read transcription, or potentially pipe-in real-time signing services
A large class demands much from a teacher, our Teaching Control Panel helps
Instantly track attention and/or confusion (via eye tracking) and tailor the lesson, accordingly.
Quickly and dynamically push 3D visual aids to students
How we built it
We're using the Magic Leap to create a mixed-reality environment for the student. Unity is our engine, and IBM Watson provides our real-time transcription services.
Challenges we ran into
None of us had worked with the Magic Leap before, so a lot of time was with system basics. We explored the capabilities of the system and spent way to long getting the dev tools installed.
Accomplishments that we're proud of
The transcription concept works surprisingly well.
What we learned
Brainstorming with a diverse group can lead to great new ideas.
What's next for In. Class
IN.CLASS is a fun proof of concept that shows a possible future for the classroom, but the underlying idea of the teacher-student connected platform is a powerful concept that could implemented with existing technologies and budgets. Our team hopes to incubate the concept into a practical product.
What would a one-year product look like? Imagine starting with Magic Leap classroom as the vision, but targeting classes with Smart Boards and iPads, which are in place today. A teacher you could still pushing quizzes, questions, and 3d objects to each iPad. Objects and questions could still be differentiated based upon the student. Teachers could project a room-sized object and students would lift their iPad to see an augmented view of the room, seeing that room-sized object.
Unrelated to learning objects, realtime transcripts, pushed to an mobile phone or to simpler over-the-eye displays (like Google Glass, or Epson Smart Glasses), could be implemented and tested in-class within the year.