Pod Piper

"Pod Piper: Your pod, your way!"

Comment

Inspiration

A combination of 2 things:

  1. Our ever growing list of articles that we save, and definitely, totally promise to read later.
  2. The lack of a truly personalized and configurable podcast that seamlessly integrates with all of our interests

What it does

Pod-piper curates the semantically relevant content from the log of articles that you may have bookmarked, and converts to an easily consumable format that you can consume at any point on the fly!

How we built it

We embedded the articles (scraped from the urls from Pocket) as well as the input query using OpenAI's davinci model. We stored the embedding for articles using LanceDB's vector database. We used semantic search using embeddings for the query and the articles. We then prompted ChatGPT3.5 to summarize the relevant retrieved articles and then piped the summary into ElevenLabs to generate a customized podcast. We used pydub to concatenate multiple audio files to add sound effects within the podcast. We also used New Relic’s OpenAI observability platform to track the openAI requests. The app is currently hosted using a Flask server and a React frontend.

Challenges we ran into

  • Latency of OpenAI
  • Retrieval of the most relevant articles given a natural language input
  • Edge cases in data cleaning when scraping data from pocket to make it tenable for podcast format
  • Prompt engineering - getting GPT and Eleven Labs to output a legible podcast script

Accomplishments that we're proud of

  • Getting a fully working end-to-end workflow using multiple generative models the way to a published, dynamic, evocative podcast episode.
  • We are specifically proud of the refinements to prompt engineering that we made to the OpenAI and Eleven Labs.
  • Newrelic integration

What we learned

  • Versatile use of LanceDB and its features - Semantic search using embeddings
  • NewRelic instrumentation to optimize cost and debug latency issues
  • Multi-modalities
  • Limits of prompt engineering

What's next for Pod-Piper

  • Closer integration with Pocket or Chrome extension to write URLs to LanceDB’s database
  • Make use of LanceDB’s multi-modal data store for generating Vodcasts.
  • Automated voice and podcast length customization by each user
  • Multiple user inputs and queries
  • Ship the app!

Built With

Share this project:

Updates