Inspiration

I always wrote in journals and made to-do lists but never tracked what got done. I wished I had a personal assistant who didn’t just help plan but got things done. As engineers, we dream of building systems for our own messy lives. That’s how DoneDone started—a system to manage daily routine using agents.

What it does

DoneDone takes your daily voice or text logs and turns them into action using smart agents.

  • Tracks health, mood, tasks, shopping, and social reminders
  • Pulls everything together into a daily email report
  • Works in both personal and work mode

How we built it

We used a modular setup with agents built using Gemini Flash 2.5.

  • Chrome web speech2text API to convert voice to text
  • Custom prompts for each agent
  • Google Agent SDK to run agents on cloud
  • Google Calendar, Sheets, Email, Drive APIs for actions
  • Simple UI to upload notes and show final output
  • Shared memory across agents, with a CoordinatorAgent pulling everything together

Challenges we ran into

  • Making agents general but still personal
  • Scoping personal data for each agent and keeping it private
  • Handling mixed input types (voice, text, structured data)
  • Detecting intent and mapping it to the right agent
  • Aligning context between agents for daily summary
  • Avoiding vague or wrong suggestions from LLM
  • Managing agent communication and keeping the flow clean

Accomplishments that we're proud of

  • A working multi-agent system that runs end-to-end
  • Smart suggestions based on mood and social interactions
  • Smooth handoff between personal and work tasks
  • Final email that feels like someone actually wrapped up your day

What we learned

  • Prompt quality > model selection between llms
  • People want actions, not dashboards
  • Clean memory and data flow between agents matters
  • You can automate a lot more than you think if you structure it right
  • Making agentic flows deterministic through mix of traditional routing is key to higher reliability and accuracy

What's next for DoneDone

  • Real-time voice journaling using socket connection
  • Feedback loop to improve agent accuracy
  • New work agents like email replies and scheduling
  • Integrations with Notion, WhatsApp, Slack
  • A voice-first UI that feels like a real assistant
  • Observability with Traceloop, Grafana, etc.
  • Governance layer on top of agentic kit to manage tokens, memory, latency
  • Open-source Agent Starter Kit for builders like us

Built With

Share this project:

Updates

posted an update

Currently we are working on making the output consistent with the user scenarios. We are testing the app output with different personal and work notes. Stay tuned to use this app to perfectly hire your AI assistant for free.

Log in or sign up for Devpost to join the conversation.