Inspiration

Have you ever found yourself in meeting, discussing great ideas, accomplishing milestones, and walking out to say “I just forgot what we were talking about.” Well now you don’t have to worry with our innovative web & desktop application that allows users to recall meeting content with ease.

What it does

The loads the recorded files generated in Zoom in our desktop (Electron) app and allows the user to process the transcription. Behind the scenes, the application uses FFmpeg to convert the individual files to one merged one for Google Cloud Speech to Text and stores it in Firebase Cloud Storage. After the transcription is done, it's stored in Firebase Firestore and accessible in our web app built with React.

How I built it

For the front end, we implemented with React and Material UI. The back end is using NodeJs, Express and Firebase Cloud Functions. This means the entire project is entirely "serverless" and therefore very scalable. For the database we choose the Firestore Firebase. We used Electron for cross platform desktop applications with Javascript to allow for easier local file uploads. We also used Firebase Auth & UI for our authentication.

Challenges I ran into

The Google Cloud Speech to text had issues to transcribe the speech for each speaker individually using the file loaded from Zoom via a cloud webhook to GCP. We needed a solution to have a file with each speaker track instead. This was only available on local recordings, so we had to switch to a desktop application for easier access to local files. Once we got one track for each speaker and again we faced issues on merging it down to a multitrack audio file that Google could process. Our final issue we ran into was connecting firestore to our web application.

Accomplishments that I'm proud of

We successfully set up and use the firestore database for storing transcriptions. Our serverless app takes real world recordings from Zoom and uses a plethora of technologies to convert it into a readable transcribed document.

What I learned

We approached the project a little differently recruiting novice developers who were looking to gain more experience building a full stack app. That resulted in one more experienced developer (Max) writing all the code while explaining it as he wrote it. It was an interesting experience as he was experimenting with new technologies as well (serverless & other firebase products) that was ultimately rewarding for everyone on the team as we ended up making a full functional app.

What's next for Retrospective

Set up billing, security, and rolling it out into a full project.

Built With

Share this project:
×

Updates