Living Textbooks

Inspiration

As we were trying to come up with ideas for this hackathon, we got on to the topic of our recent exams. We were discussing how sometimes it can be difficult to visualize and understand concepts by just looking at 2D diagrams. Hence, we were motivated to build something that could provide a better visual aid for concepts in our textbooks. Our chosen medium was 3D augmented reality. We felt that the extra dimension would inherently provide more intuitive sense to a 3D based human brain. How we built it We used Google’s ARCore API to be able to detect flat surfaces, estimate lighting, and place 3D models on the flat surfaces. We then combined that with the Image Recognition API to show certain 3D models, graphs, and animations upon detecting certain images. We generated an image database using the SDK tools, and tied every image with a 3D model or animation. We programmed the animations inside ARcore.

Challenges we ran into

One of the main challenges we faced was our lack of experience. So we spent a lot of time watching tutorials and reading articles on how to implement the features that made our project what it is. This lead to starting and abandoning features which lead to wasted time. Accomplishments that we’re proud of We all feel really proud to have AR working, recognizing textbook images and providing an interactive AR experience, especially given how little prior knowledge we had on the subject. Moreover, we take great pride in our improvement in programming, as it was the first hackathon for the majority of the group.

What we learned

Prior to creating this application none of our group had any experience with Augmented Reality and the majority of our group had little experience with android app development and web-dev. We feel like this hackathon has given us all a better understanding of web dev and other programming concepts like views and activities in. We also felt that we have all gotten substantially better at working as a team, delegating tasks to each individual and managing our time efficiently. What’s next for Living textbook With LivingTextbook we feel like there are limitless applications, and we are really looking forward to developing tools that allow publishers and creators to implement them. The main tool for this will be our website that will provide an intuitive interface for the average user to create their own 3D models and animations. One place we see this being very useful is in children’s fiction books. We would love to give publishers the chance to create friendly entertaining animation to bring a children’s book to life. This will engage the child and make learning and reading more fun. Another exciting place we see this being used is IKEA type assembly instructions. Animations would give the user a better understanding of how parts fit together. Another example, we could see construction and architecture companies using this technology to add 3D models and animations to better visualize and spot flaws in their designs. Additionally, we think it will be really cool to have the models completely parameterised and interactable, this will let students see how different variables change how a system operates. Finally, we think it will be really useful if we can have models be automatically generated from pictures.

Built with

• Java
• ARcore
• Sceneform
• Android Studio

Built With

• android-studio
• arcore
• java
• scaleform