FlashBack

Recall in real time!

Comment

Inspiration

Inspired by our own struggles with career fairs, we decided to come up with a solution for it.

What it does

FlashBack is an AI assistant that travels with you to refresh your memory. The way it works, is by recording footage from your point of view, sending it to the cloud. At a future time, when you need to recall events of the past, a web interface is available for you to ask questions about who you met, or important information about people you talked to.

How we built it

A raspberry pi computer, with a camera and a mic is the heart of Flashback. The pi records a video from your point-of-view, and sends it to a flask-based server. This server serves as a middleman, processes the video, and sends it to google cloud for storage.

The flask-based server also hosts a web interface for prompt inputs. When the prompt is entered, an API hosted on the google cloud is called, which in turn calls the Gemini API to fetch insights about the videos for you.

Challenges we ran into

  • Hardware failure: The camera gave some issues. Thanks to Jared's help, we were able to procure another camera.
  • Audio + video: The raspberry pi records audio and video separately. We needed to combine them in one file, which we achieved in the flask application, and this file was then sent to the google cloud for storage.
  • Continuous Deployment: As we developed more features, some updates gave fatal issues which we could not deploy. We had to figure out changes to the deployment environment to be able to host the updated application.

Accomplishments that we're proud of

  • A complete end-to-end product.
  • Integrating 3 different components in such a short span of time.
  • Being able to deliver value.

What we learned

  • Raspberry pi
  • Issues with continuous development + docker
  • How to use the Gemini API

What's next for Flashback

  • Longer video durations
  • Live streams
  • Asynchronous uploading with prompts.
  • Agent with more functions : e.g. book a calendar slot based on recalled conversation with someone else.
  • Using voice queries instead of text
Share this project:

Updates