Coco

your little ai assistant that keeps getting better

Comment

Inspiration

Personalized AI coding setups are awesome, but they all tend to hit the same three walls:

  • They're boring (sometimes straight-up brutalist)
  • Context is a pain in the ass to manage
  • They don't get better. Agents that keep improving is one of the coolest things modern AI infra can do, and these setups miss it

What it does

Coco is a little robot buddy that lives quietly in the background of your laptop. It fixes all three of those, wrapped in an interface that I gotta say is sooo clean:

  • Boring, brutalist UIs → a robot buddy that talks and draws on your screen
  • Context is a pain → vectorless RAG over a graph of you
  • Agents never improve → specialist Cocos that learn from feedback

Let's go from the ground up.

Talk to it

Hold ⌥ + ⌘ and just ask. Coco can do anything through your computer:

  • play music
  • open videos and files
  • pull up recipes
  • book things in your calendar

It draws on your screen

Coco can draw directly on your screen and intuit its way through interfaces it's never seen before.

  • Drop it into Canva cold and it can walk you through it or use it to help you build whatever you're making
  • Stuck on a practice problem or a video? It sketches the explanation right on top of it

It knows you, without clogging context

Coco uses industry-standard vectorless RAG (Tree DSA) to sift through a context graph it builds about you.

  • It knows a lot about you, but only pulls in what's relevant, so your context window stays clean
  • The more you use it, the sharper it gets at your personalized tasks

Specialist Cocos that learn

But it gets even better 😂

You know the drill: want an agent to fine-tune resumes? You build up skills, hook up tooling, and then finally get to do the one small task.

With Coco, those skills and configs are saved as versions you can switch between on the fly:

  1. You: "make me a resume for this job"
  2. Coco picks your saved resume agent, pulls in everything relevant it knows about you, and writes it
  3. You: "too wordy, keep it to one page"
  4. Coco updates the agent and learns from it

Share your Cocos

As a cherry on top: built a specialist you love? Share it with a link and anyone can use it.

Challenges we ran into

  • macOS permissions: ok I get macOS is trynna stop me from getting a virus, but goddamn these make it hard to autonomously build/deploy/test 😭
  • Coco went blind: for some reason it would just start saying it couldn't see the screen, despite all the visual infra built in
  • Coco was slow: 20 seconds to respond to a "hi"
  • Ugly drawings: the on-screen overlay kept getting built from math-based SVGs, which looked so ahhhh

Accomplishments that we're proud of

  • it works 🙏

What we learned

  • Bruh, pretty much everything here was a first for us. Getting something to work dynamically and autonomously across such a diverse set of tasks was such a pain

What's next for Coco

  • more real-world testing
  • more UX polish

Built With

Share this project:

Updates

Submission history