ready player n

Inspiration

Thinking on the submission categories we were all drawn to environment, not surprising given the beautiful state we live in. We were all aware of hiking videos and tik toks and wondered if we could leverage Twelve Labs’s Marengo engine to parse and find hikes with ‘Hidden Gems’ ex: waterfalls.

What it does

Take in a user request, query the indexed hike videos, respond with a thumbnail of the video snippet, a description of the video snippet and ‘Hidden Gem’, a link to a hiking website with additional details.

How we built it

We started by experimenting with indexing hiking videos and searching for specific ‘Hidden Gems’ ex: Waterfalls. Once we proved the core concept was viable we started building in Gradio. Gradio was a serviceable prototype framework but we ran into issues displaying video. We migrated the core code to Streamlit. With core functionality working, displaying a thumbnail from the video, a time stamped link to the video, and our additional hike website metadata we started working to hook in Groq. We use Groq to seamlessly refine the user input into a high quality prompt search query to pass to Twelve Labs.

Challenges we ran into

Scope! As always. The original idea had many more bells and whistles. Geolocation tags sourced from the video text and linked to google maps displayed in app. Current population and usage stats pulled from government websites, etc etc. There are endless ways to expand on the core functionality. We decided to scrap all of these as we ran into environment setup issues, data collection challenges and time constraints. We additionally ran into issues around retrieving thumbnails from Twelve Labs which served the file, which required planning to download the thumbnail and store temporarily. We next learned that an embedded Youtube link in a Gradio chatbot component was a non starter. This led to partially rebuilding into Streamlit.

Accomplishments that we're proud of

Collaboration! Perseverance! Patience! Grit!

Ensuring that all voices in a group of 5 new collaborators was a challenge that required compromise. We threw lots of initial ideas at the wall, tested many different types and categories of videos on Twelve Labs before taking a vote to narrow scope to hiking. Then we discussed strengths and weaknesses to find how we could best use our time and divide the project into tasks.

Perseverance, there were significant development environment issues and lessons learned that landed at the most efficient use of time being over the shoulder collaboration.

Patience, shockingly nerves never got so frayed as to consider giving up. Attitudes remained up and the project was completed.

Grit, all of the above challenges led to more than one discussion on whether the project was worth completing.

What we learned

The time spent discussing and ensuring everyone was on the same page was well worth the time. Free experimentation allowed every idea to be rapidly prototyped, videos indexed and queried to rule out what was practical and within the time allowed. Because we engaged all ideas, it was easy to all agree on a go forward plan. Next time the time to synchronizing development environments could have saved significant time and allowed for more refinement and development instead of troubleshooting.