Co-Assistant AI

bout the project Inspiration We took inspiration from university students and workers who are looking for more ease of access to handling workplace tasks using AI tools and integrated voice activation and interaction in order to perform scheduling and managing actions with a simple voice command. Similar to Marvel and how Iron Man commands Jarvis for actions.

What it does It includes voice activation and interaction which allows users to handle Slack executions such as sending messages and any features inside of Slack through AI integration and databases.

How we built it We built Co-Assistant using Next.js for a modern, responsive web interface and Supabase for our backend database and authentication. The core "brain" is powered by OpenAI's GPT-4o, which we engineered to use function calling to perform actions like creating notes, saving memories, and fetching Slack data.

For the voice interface, we integrated ElevenLabs for ultra-realistic text-to-speech and used browser-native speech recognition for input, creating a seamless hands-free experience. The Slack integration was built using a custom secure OAuth flow, allowing the AI to query channels and post messages on the user's behalf. All specific user data (notes, memories) is stored in Supabase with Row Level Security (RLS) to ensure privacy.

Challenges we ran into One major challenge was state management for the AI agent. We needed the AI to propose actions (like "send a message") and wait for user confirmation before executing them, which required a complex state loop between the frontend and the LLM.

Another hurdle was handling Slack OAuth securely. We had to ensure tokens were encrypted at rest in our database and only decrypted server-side when the agent needed to perform an action, while also handling edge cases like token expiration.

Finally, ensuring low latency for voice interactions was tricky; we had to optimize our API calls to make the conversation feel natural and not robotic.

Accomplishments that we're proud of We are incredibly proud of the Long-Term Memory system. Unlike standard chatbots that forget everything when you refresh, Co-Assistant saves key facts, preferences, and decisions into a "Vault" that it can recall weeks later.

We're also proud of the zero-UI Slack integration. Users can just say "What's happening in #general?" or "Tell the team I'll be late," and the system handles all the API complexity behind the scenes without the user ever opening the Slack app.

What we learned We learned a lot about Agentic Workflows—moving beyond simple "chatbot" responses to AI that can actually do things. We also gained deep experience with Vector Databases for memory retrieval and the importance of accessible design patterns for neurodivergent users (like our "Quiet Mode" for ADHD focus).

What's next for Co-Assistant AI We plan to expand our integrations to include Notion and Google Calendar, allowing Co-Assistant to manage your entire digital workspace. We also want to develop a dedicated mobile app to make the voice features even more accessible on the go, effectively turning it into a private, secure "Jarvis" for everyone.