A recent research published showed that a large majority of the world's population is not tech savvy enough to use do simple things like search online. This can prove to be a major hindrance to learning as it inhibits a persons ability to search for information to clarify doubts and expand their current knowledge base. We sought to somehow plug this gap by coming up with a suitable solution

What it does

Our software uses an image of what you are reading to detect key concepts and based on which concept is most relevant to you its give suitable video suggestions. It also provides a call feature to connect you to people exploring similar areas to help you share and learn more.

How we built it

We used technologies like AR.js, Three.js, AgoraRTC SDK, javascript , HTML and CSS to build a fluid, attractive and efficient front end to interact with user. Our backend comprises of a flask server written in python which runs an OCR on the image taken on the user's phone and then does text analysis followed by some web-scrapping to serve relevant information to the user.

Challenges we ran into

This project threw some interesting challenges at us. At the very beginning getting ourselves familiarized with the implementation of AR in the browser was a tedious yet eventually fruitful task. Subsequently isolating and retrieving frames from the video stream of the user camera was a challenging task as well. Processing of the image as well as the subsequent text generated before actually retrieving the relevant videos required a lot of research and experimentation to land a feasible prototype.

Accomplishments that we're proud of

We are proud of the fact that we were able to meet or targets and be able to create a working prototype in the limited time-frame. A large majority of the work we put in required us learning things from scratch. The fact that we not only successfully learnt but also successfully implement our idea into a working product is truly a great experience and accomplishment for us.

What we learned

First and foremost we learnt learnt how to do a thorough background research to back the feasibility of our idea. Following that as we delved into the tech stack we got hands experience into implementing and debugging code aligned to augmented reality, networking, protocols, optical character recognition,etc.

What's next for CubEd

We hope to keep growing by first optimizing our algorithms for faster and more relevant responses, as well as incorporate algorithms to connect individuals base on their level of understanding of relevant concepts to make this a truly beneficial product for all.

Share this project: