Rube Codeberg

Flow chart

Inspiration

Online translators are often difficult to use; we simplified the translation process for English to French.

What it does

Given text written on a surface, a webcam from a Computer A takes a picture. With the picture, Computer A uses Google Cloud Vision API (OCR) to read the text into English text. From there, the computer sends the text to AWS Polly for text-to-speech, reading the text aloud in English from a bluetooth-connected speaker. A nearby Android phone listens to the audio with its microphone and parses the audio with the Google Cloud Speech API. That text then is translated into French via the Google Cloud Translate REST API, and this string is then converted to morse code. With the morse code string, the phone flashes its light according to the corresponding pattern: dashes are represented by 500ms of light and a 100ms pause; dots are represented by 200ms of light and a 100ms pause; spaces are represented by a 1100ms pause. Letters are separated by a single space and words are separated by two spaces (a 2200ms pause).

The laptop then utilizes OpenCV to detect the the phone's light flashes using a blob detector, blur, and image segmentation based on hue/saturation/value ranges. If a frame had values above a certain threshhold, we add a 1 to an array and we append 0 to the array for every frame without the light. From there, the laptop converts the morse code back to plain French text; the string of French text is then sent via the Gmail API to an email that was connected to a fax machine number, with the text as the body of the email. This converts the text into a fax cover sheet, which is then faxed over to the fax number via the Twilio API and TwiML that receives the fax and saves it to pdf. This pdf is then opened up in a new tab automagically, revealing the translated text under the fax's "Cover Message."