Current students struggle with gaining the most value in the shortest time, We learned about the best methods of consuming information - visual and summaries, but current teaching techniques lack these methods.
What it does
Scan your writing using your phone which then converts it into a text file. Then extract the keywords from that file, look up most relevant google photos from that and brings it back to the phone to display the summary with the photo and the text below. Options of more specific summaries for the individual keywords will be provided as well. It can also read aloud the summary for the visually impaired in order to have better traction and user interaction.
How I built it
We used unity to develop an augmented reality environment , we integrated machine learning model at the backend of that environment using rest API. We used the google cloud vision API to scan images for text and then using that text as our input to produce our text to speech , summarization and slideshow capabilities
Challenges I ran into
1.)Difficulties in interaction/ creating communication channels between Unity applications and python machine learning models due to lack of existence of documentation and resources for the same 2.) integration of python models into REST APIs 3.) Difficulties in uploading our API into an online cloud based web hosting service ,due to its large size
Accomplishments that I'm proud of
1.)Successfully integrating two different platforms together which end up linking two very powerful and vast fields(AR and machine learning) 2.)Creating a working app in 2 days which happens to be an unique product on the market
What I learned
1.) we learnt how to form a data pipeline between Unity and python which supports AR and machine learning 2.) how to form realtime applications that run ML at thier backend 3.) how to effectively manage our time and make a product in 2 days as a team activity 4.) how to make our own APIs and how to work with them ,
What's next for TextWiz
We currently only support english , we wish to support more languages in the future and we wish to be able to translate between those language to increase connectivity on a global level. Our current machine learning model while being good still has room for improvement and we plan to increase the accuracy of our models in the future.