Inspiration
Our inspiration was the fact that learning how to cook can be hard: even if you follow the recipe, measure everything to the teaspoon, and take everything out at the exact right time, it's hard to have confidence about whether what you're doing is correct. We wanted to create something that could give you real-time advice and feedback and that could help you out when you were confused and needed to ask a question or when you were missing an ingredient and needed a replacement.
What it does
Point your camera at yourself as you cook or do a math problem, and InstructorAI will track your progress and give you live advice and feedback as you work. It is also able to answer any questions you have.
How we built it
We used Overshoot AI to provide video context to a LiveKit voice agent so that the agent has all the info necessary to help you.
Challenges we ran into
Prompting the LiveKit voice agent to do what we want.
Accomplishments that we're proud of
Successfully integrating both products and using the strengths of each to build a product that can be actually helpful and instructional.
What we learned
How to use LiveKit and Overshoot successfully and how to chain the inputs and outputs of both.
What's next for Compass
Further refine our prompts and find ways to provide even more context for the agent.
Log in or sign up for Devpost to join the conversation.