Speech-to-See

Speech-to-See is a framework for visualizing speech in real time.

Comment 2

Inspiration

Initially, we were inspired to help designed prototype customer requirements. The app would help both designers and clients by allowing the client to describe the website they want and visualizing it in real-time. We then realized the potential of this project and decided to expand it to any application where speech visualization could be useful.

Examples of such applications include:

  • helping students with attentive disorders by visualizing certain concepts
  • speed up web design and prototyping
  • help employees with their day-to-day meetings and discussions.

What it does

It records your voice and adds visuals to a page in real-time.

How we built it

  • React application for the front-end
  • React component for speech recording and transcription
  • Google's Natural Language API for Natural Language Processing
  • Shutterstock's api for image search
  • Node.js server for the backend

Challenges we ran into

  • Unfamiliar technologies
  • Recording audio proved to be very finicky and difficult.
  • Google's and Shutterstock's API's had some gotchas which we didn't expect
  • Shutterstock authentication wasn't super straightforward

Accomplishments that we're proud of

  • Learned new technologies: React, node.js
  • Learned new API's: Google Natural Language and Shutterstock
  • Made a working prototype despite problems.

What we learned

  • New technologies: Node.js, react, API's

What's next for Speech-to-see

Currently, only basic functionalities are implemented, adding/removing text and images, and positioning them. We plan to add more functionality and make it more polished.

Built With

+ 1 more
Share this project:

Updates