Project Journey: Interpreter App Submission for Google Hackathon

Inspiration

Working in a healthcare company, I was motivated by a colleague's struggle to find affordable interpreter training services, with only one expensive vendor available..

Challenges Faced

• Integrating getUserMedia API for browser compatibility and backend audio formatting.
• Implementing conditional rendering for frontend responsiveness.
• Designing a generic prompt for VertexAI validation.
• Learning the importance of architectural planning before coding.

Learning Experience

• Emphasized the significance of pre-coding planning.
• Expanded knowledge of Google services like Gemini models, Speech to Text, and Cloud Storage.
• Transitioned from separate backend and frontend repos to a unified containerized setup for streamlined development.

Project Overview

• Users engage in interpreter assignments within videos, pausing for prompts.
• Video/audio conversion to WAV via Google Speech to Text API.
• Validation using VertexAI with feedback on interpretation accuracy.
• Iterative improvement through multiple interpretation prompts.

Built With

• ffmpeg
• flask
• javascript
• mui
• node.js
• python
• react
• speechapi
• vertexai