Sidekick AI is a context-aware AI teammate designed for group FaceTime calls. It follows conversations and shared visual context to understand what the group is working on, remember participants' roles and goals, and provide relevant recommendations without interrupting the conversation.
Our first use case is cooperative gaming. Sidekick can remember which player is responsible for a position, recognize situations that need attention, and show short recommendations in a private companion interface. Users can request a detailed voice explanation that plays locally on their device.
We're building Sidekick with Chert and LiveKit for FaceTime connectivity, Gemini for AI reasoning, a TypeScript session engine for group memory and recommendations, and a React companion interface.
The project is still being integrated. The session engine and mock reasoning pipeline are tested, while live FaceTime input, visual interpretation, and the complete end-to-end experience remain in development.
What it does
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for Sidekick-ai
Built With
- chert
- docker
- livekit
- node.js
- python
- react
- tailwind
- typescript
- vite
