Example of insights generated by app
In life outside of school, we rarely succeed based on our ability to output a single correct answer, we persist in a gray area and there is more than one “correct” answer or outcome to arrive at. However, our education system holds the standardized multiple choice assessment exams on a pedestal. This is not only detrimental to the students but it has also been documented to be discouraging to different types of learners that display their “intelligence” and “competence” in non-classical ways.
Kolideo seeks to be the first-ever AR learning platform that is based on and can run real-time analytics on the learners themselves - not on their “answers” but on their process to their answers. We were inspired by the fact that through mobile AR, our phones can observe our behavior and make us better. We want to use this power for education.
What it does
We focused on building Kolideo for the Merge Cube with mobile AR because we want to make sure that it is accessible cost-point and hardware wise to students from all socioeconomic backgrounds. Our platform includes educational puzzles in the frontend and a machine learning based backend to run analytics on the student as they progress through the puzzle. Our app can let a teacher know items of interest like how long a student took, how many tries it took, how they went about solving the puzzle itself, which parts of the puzzle they excelled at etc. Ultimately, these observations can lead to insights for the learner and the teacher.
How we built it
We used the Merge Cube SDK, Unity, Photon, Vuforia, AR Core and the Samsung S8. We spent a lot of time ideating on the best demo for a puzzle. We settled on a geometry puzzle that teaches students about spatial learning and that requires trial and error based problem solving. This was done intentionally so that we could generate insights for the teacher/learner based on how the student progresses through the puzzle. We built all the assets ourselves.
Challenges we ran into
Originally, we wanted to create a version for the Meta (so we could demo "low-end AR" and "high-end VR" but we ran into issues since they do not have Vuforia support. Therefore, we decided to focus on the Merge for the purposes of the hackathon.
Accomplishments that we're proud of
We're proud of our experimentation into running real-time analytics on AR experiences. We believe that mobile AR is a powerful tool for observing human behavior and we are excited about applying this tool to education.
What we learned
This was all of our first times developing for mobile AR so we learned about the pros/cons compared to the HoloLens which we usually develop with. Ultimately, we believe these tradeoffs were necessary because our ultimate goal with Kolideo is accessibility.
What's next for AnonymousRaccoon
We would like to add more educational puzzles, fully buildout our analytics backend and add functionality into the app that suggests next learning steps. We would also like to build a function that links students to other students who have complimentary learning techniques to facilitate peer-peer group learning. We would like to partner with experts in the education space to refine and fully build out our metric. We want this “metric” to be different from the one size fits all of the traditional skills assessment since this metric is based on how the student processes information or thinks as opposed to only being about what they “know”. We believe that observational AR is uniquely poised for figuring out this metric.