MathRead

Android App
Chrome Extension

Inspiration

Vitaliy Vertogradov - Ph.D. in Mathematics, conducts research in the field of black holes and gravitational collapse. He is completely blind, therefore a special program reads the plain text to him, and difficulties arise with formulas.

Vitaly was looking for volunteers who can read books with formulas. But we want to offer him a new revolutionary solution - an application that could voice the formulas for him.

What it does

It is a Chrome Extension and an Android App that voices formulas on the web page and pdf documents that are not voiced by the regular text-to-speech algorithms

How we built it

Our algorithm finds all of the equation images in the documents or/and web pages and voices them app by parsing through image with Google Vision API, convert image into a text and voice the text with Google Text to Speech API. We created a Google Chrome extension using JavaScript and HTML. For the mobile app, we used Android studio and Figma for the frontend.

Challenges we ran into

Integrating the python algorithm into the javascript file of the extension was the greatest challenge we faced. There were no open source solutions we could find.

What's next for MathRead

Building an API that will convert images into LaTex objects and API that will convert LaTex objects into a text to improve the recognition of complex mathematical symbols.