TimeScribe

TimeScribe is a digital journal that lets you effortlessly log your thoughts. Using real-time speech transcription and seamless memory storage, TimeScribe can summarize your entire journaling history.

Comment

Inspiration

  • wanting to easily log our dreams to keep track of them before we forget them forever
  • helpful for patients dealing with Alzheimer's or any dementia
  • also helpful just for journaling thoughts and ideas that could be accessed later

What it does

  • used Groq's blazingly fast latency to easily log users' thoughts by typing them into the text bar or by using the recording feature
  • Because of Groq's speed, our product can quickly summarize and output the logged entries in between the given date range chosen by the user

How we built it

  • Used Supabase to store user IDs, agent IDs, and the raw entries for each user
  • Used Groq for STT, TTS, and to summarize the entries
  • Used Letta to add the summarized entries to the context, which is threaded in the background

Challenges we ran into

  1. At first, we wanted just to use Letta Cloud to store the memory, but it took too long to retrieve the information, so we decided to store the raw data in Supabase instead
  2. We had issues trying to connect Groq and Letta, so we decided to manually send the context from Letta to Groq
  3. When trying to summarize, the context had a lot of metadata, so we had to manually filter it to only show the text of what the user logged
  4. We also ran into problems when trying to send TTS audio back to the frontend, as we did not fully understand how to format and read the HTTP requests.

Accomplishments that we're proud of

  1. We were able to incorporate multimodality, including STT
  2. We also incorporated the summarization across customizable date ranges
  3. Our product was able to log and summarize the text relatively quickly
  4. A user can successfully access their summarized entries even when logging in on another device

What we learned

  • How to work with Supabase, a database, to store user information and their data
  • How to use Groq's quick latency to our advantage
  • How to use Groq's whisper model to incorporate STT to transcribe the message to English text, and playai-tts to get the summarized text back to TTS
  • Learned to integrate multiple frameworks and technologies (Groq, Letta, Supabase)
  • Explored threading to run Letta in the background to decrease computational time

What's next for TimeScribe

  • Proper authentication systems to maintain the privacy of our users

Built With

Share this project:

Updates