Inspiration
In 2022, my (phuthai) mother passed away as I was graduating RPI (now doing masters). Kelly and I were inspired to use AI to help re-live interactions with past (or current) loved ones that are no longer near. Keeping with "Retro v Modern", my mother and many people in our age group have parents who were living it up "retro" 80s. We put a happy spin and and theme on this interaction, and hopefully bring joy/entertainment to anyone who wants a retro version/conversation with a person.
NOTE: IF YOU VISIT THE URL, YOU MUST BE ON RPI WIFI. I BLOCKED SUBNETS 4 SECURITY
What it does
Upload an image of anyone, and watch them go 80s!
Pre-Prompt the interaction, and carry on unforgettable conversations in a variety of voices!
Render an interaction, and watch as the image comes to life!
How we built it
The infrastructure is built in VULTR cloud, deployed via Terraform
We used Text->Text and Text+Image->Image models from Gemini API
We audio/voices from ElevenLabs
Build on a simple Python/Flask, HTML, CSS, JS tech stack
Used a custom Wave2Lips LipSyncing model to bring images to life!
Challenges we ran into
Because we ran our own local machine learning model to create lip-syncing images, the Python required heavy CUDA/CPU usage. We had to use the more expensive VULTR VMs to continue running remotely
Because of computational requirements, I neglected to perfect the Kubernetes deployment. The Cluster is up, but the app is not functional on Kubernetes
Web Cam doesn't work when run on the server, only locally
There are 2 versions of Google Gemini's APIs, that aren't compatible. Using AI to learn this API causes issues since even Gemini mixes up the APIs. (google-genai vs google-generativeai)
Repo is private, and we'll offer on request or make it Public after the Hackathon ends. We did not do a good job cleaning up the API keys from the repo + repo history
We both hit CoPilot prompting limits
Accomplishments that we're proud of
- Most things worked! The coolest parts of the project worked
What we learned
- A lot more about APIs and reading docs, trusting it over AI recommendations
What's next for Echo Of You
- Find a permanent home outside of the unique temporary IP address it currently lives on
Built With
- css
- elevenlabs
- flask
- gemini
- html
- javascript
- kubernetes
- loadbalancing
- persistantstorage
- python
- terraform
- virtualmachines
- vpc
- vultr
- wave2lips

Log in or sign up for Devpost to join the conversation.