Inspiration We spend a massive portion of our day performing repetitive desktop tasks—jumping between browser tabs for research, formatting documents, and manually entering data into spreadsheets. We realized that while conversational AI is great, we needed an AI that actually does the work for us. We wanted to build a true local assistant that bridges the gap between human intent and desktop execution, giving users their time back to focus on high-level, creative work.
What it does OSmosis is an autonomous, multipurpose desktop agent. You give it a high-level natural language prompt (e.g., "Research the history of the strands framework and create a Google Doc summarizing the findings"), and OSmosis handles the rest. It leverages LLM-driven planning to figure out the necessary steps. It uses Playwright to autonomously navigate the web and scrape detailed information, and then utilizes Google Workspace integrations to seamlessly synthesize its findings into newly created Google Docs or formatted Google Sheets—all with zero human intervention.
How we built it We built OSmosis in Python, utilizing the Strands Agents SDK as the backbone for our agentic architecture. This allowed us to move away from rigid, hardcoded state machines and rely on an LLM to dynamically orchestrate its tools.
We integrated Playwright for robust, automated web research and scraping. For the data output layer, we tied in the Google Workspace APIs (Docs and Sheets). To ensure the agent is lightning-fast and capable of complex reasoning, we powered its "brain" using the Groq API (running models like Qwen) to rapidly process prompts and execute tool calls.
Challenges we ran into Building an autonomous agent comes with unique hurdles. One of the biggest challenges was handling the non-deterministic nature of LLMs when chaining multiple tool calls together; if a web search failed, the agent needed to know how to pivot and try again rather than crashing. Additionally, ensuring our Playwright scraper reliably extracted information across vastly different website DOM structures required a lot of fine-tuning. Finally, managing the secure OAuth flow and API credentials for Google Workspace integrations was a complex hurdle to cross.
Accomplishments that we're proud of We are incredibly proud of successfully taking OSmosis from a concept to a fully functional desktop agent during the hackathon. Watching the agent take a vague prompt, autonomously search the web, synthesize the data, and generate a fully formatted Google Doc right before our eyes was a massive "wow" moment. We successfully bridged the gap between a chat interface and tangible, local desktop execution.
What we learned We gained a deep understanding of modern agentic architecture. We learned how to effectively bind complex, real-world Python functions (like web scraping and API calls) to an LLM, trusting the model to handle the orchestration. We also learned how crucial it is to write highly specific system prompts and tool descriptions to keep an autonomous agent on track.
What's next for Osmosis | AI Desktop Agent This is just the beginning. Next, we plan to expand OSmosis's toolset to include local file system management, calendar scheduling, and email integration. We also aim to implement a persistent, long-term memory system so that OSmosis can learn the user's specific workflow preferences, writing style, and habits over time, making it a truly personalized AI desktop companion.
Built With
- python
- strand
- toml
Log in or sign up for Devpost to join the conversation.