Symbiosis

Inspiration

Companies spend $1.8 trillion annually on repetitive work, and it's hard to select which tasks to automate without understanding the implications that it might have. Managers guess. Employees don't notice their own patterns. So we asked ourselves, what if AI could just watch how people work and tell you?

What it does

Symbiosis records employee screen activity, analyzes it with AI, and outputs exactly which tasks could be automated, with ROI estimates attached. First, the desktop app captures screenshots at 1 FPS with on-device OCR. Then our AI pipeline clusters recurring patterns using cosine similarity and scores each by automability: A = 0.4M + 0.3F + 0.3D. M, mechanical score (weighted 40%): how repetitive and rule-based the task is. Copy-pasting data = high. Creative writing = low. F, frequency score (weighted 30%): how often this task appears relative to others in the session. If it's the most common task, F = 1.0. D, the determinism score (weighted 30%): how predictable the task category is. Data entry = 1.0, research = 0.5, meetings = 0.3. Finally, the admin's dashboard shows aggregated opportunities across the whole team, ranked by total hours and dollars saved.

How we built it

We used Tauri 2 + Rust desktop app for capture and on-device OCR, SvelteKit + Supabase cloud backend for the analysis pipeline and admin dashboard, and a unified AI abstraction layer that supports GPT-4o, Claude Sonnet, and Gemini Flash interchangeably.

Challenges we ran into

The biggest challenge we ran into was understanding how we were going to analyze the video recordings, as video file are heavy and hard to deal with. We figured a workaround, using sparse frames (1 fps) and combining it with OCR information, we were able to fit sessions into analyzable data.

Accomplishments that we're proud of

A fully working end-to-end product, desktop app, cloud pipeline, admin dashboard, and privacy mode, built in a hackathon. We didn't think we would have ever gotten close to finish a project like this with this low amount of time. But we did it, its not perfect, but we are extremely proud to have a functioning product.

What we learned

A problem well understood is a problem 50% solved. We went in thinking the hard part would be the AI and how to build the product, when in reality it is all about understanding what value you want to give, why are you doing this.

What's next for Symbiosis

We still want to perfect this idea. The problem is one that truly exists, and the solution that we have ideated makes sense. We are going to continue fine-tuning Symbiosis. We have already thought of new features we could add and ways to create more value to the product. We are excited.