palate

Architectural Design
settings mock
audio player ui mock

VIDEO LINK

Inspiration

We all have more content that we want to consume then we have free time available. We are using AI to solve that problem.

What it does

It takes one or more pieces of written or audio content and converts them into a personal "podcast" based on the content you feed it. You can customize it to be a certain length, language, reduce complexity to make it more understandable, decide what phrases/words you don't want to hear and even customize the voice of the person behind your new "podcast"! We named it Palate because our product takes content and creates net new content based on taste.

How we built it

We built it using these tools:

Whisper API and Amazon Transcribe to take audio and convert to text
Store text in the Amazon Document DB
Leverage GPT 3.5 to ingest the new text and feed it to LanceDB for summary storage to vectorize our data using the ada-002 model to help with ban-listing phrases that users don't want to hear.
Feed that back to GPT 3.5 to stitch the new content together.
Synthesize using speech models and store in an Amazon S3 bucket that gets served to the user.

On the client side we're using SwiftUI to create an iOS app where users can edit their palate, make new episodes based on sources that they give the app, and listen to their episodes.

This uses a lot of different tools that could get expensive as the app scales, so we are using New Relic to monitor our cost and usage of the different models.

Challenges we ran into

We've had to combine a lot of different services for a product that is useful to the end user, which was a bit challenging and took some time.

Accomplishments that we're proud of

No one of the team knew each other before Saturday, so we're very proud that we were able to quickly work together to understand what we want to build and hack something together.

What we learned

We learned a lot! Including:

It was our first time using all of the technical tools we used to build the product, so just buy stitching all of the services together we had to learn a lot
What users want by testing our proposed MVP by surveying people who have come for the hackathon.

What's next for palate

We're collecting users for the waitlist and want to put together a MVP that we can release! A roadmap to improving the product includes creating a recommendation engine based on the user's interests (using the LanceDB), offering more languages/dialects, and more.