StudySmARt

A unique way of getting students to study

Comment

Inspiration

The three of us are Computer Science students. As such, we tend to be lazier but more driven than usual; if we could find a cooler way to do something, we would take it. One of the most tedious things that we have to do on a daily basis is, of course, studying. We decided to try and make the chore of studying a bit more interesting and fun for a student.

What it does

Upon opening the application, the user is greeted with a camera view. However, once the camera hovers over one of our pre-made QR codes, the user is transported into a mini-AR, surrounded by 3D cards with information about the subject that they are currently learning about, in every direction. As is conventional with AR, the student moves her phone around to see each card, as if they were right in front of her. These cards are interactive; upon click, some cards flip into questions related to the subject that the card gave the student information about. Other cards also turn into a real-time 3D animation of the topic, so that the students can visualize more abstract concepts easier. The idea is the teacher, or textbook creators, can make their own cards/animations for their curriculum modules, and upload them to a server managed through AWS. Then, they can just give their students the QR code associated with the module, and the student can learn at her own pace in a way that's much more engaging and fun to use than reading a boring textbook.

How we built it

The application was written completely in Swift with Apple's newly-released ARKit, incorporating SceneKit, as well as pre-existing framework Vision, which was used to find and read the QR codes from the ARSession. The .dae 3D models and animations were made using SketchUp and Blender, and hosted on an AWS server. Icon images and some other graphics were made using the Vectr program.

Challenges we ran into

The original plan was to pull the .dae files from the AWS server only when needed in the device, but due to time and resource constraints, we were forced to simply host the files locally for demo purposes. However, the files are on a server, and can be accessed/implemented at a later time.

We also planned on ditching the idea of markers - in the spirit of augmented reality, figures and information would pop up in real time based on text recognition of a textbook page. However, after creating an entire model that used the Vision framework and third party machine learning OCR model Tesseract, we learned that the new ARKit and older models such as Tesseract did not mix well. As we really wanted to use AR to create the diverse user experience, we had to ditch the OCR idea, and basically start from scratch to incorporate a fully ARKit-functional application.

Accomplishments that we're proud of

The thing works! It looks decent, too. None of us had any real experience with AR or 3D modeling before, and using these new technologies was very frustrating/fun/satisfying to work with to make a full product. Additionally, we finally found an actual use for the 4 years of physics instruction that we've received.

What we learned

ARKit is really powerful, but does not play nicely. In general, it was really fun playing with such a new technology - it really makes you feel like you are on the cutting edge of something new and exciting. We also learned that 3D is a hell of a lot harder than 2D.

What's next for StudySmARt

Make the graphics much better. Apple has a powerful rendering structure that we haven't really utilized yet, and the cards do still blink and are sometimes hard to read. Additionally, add new features - maybe graphics such as graphs and charts?

Built With

Share this project:

Updates