Docket

A Stub Demo of the Agent working

Inspiration

Chores are annoying both offline and online, wether its sorting through your email to find a link to the meeting your late for, or downloading a collection of short stories off the internet. To make browsing much easier for everyone, me and my friend has decided to create an autonomous agent that can complete digital tasks for you with your identity.

What it does

Docket has a chatbot you can discuss you tasks with, for example sorting through your email, or downloading some movies for a plane ride. After deciding on a good plan, Docket AI creates a new task in the Task List, which an autonomous agent starts to work on via playwright chromium browser automation. Playwright allows the agent to view elements on screen, and interact with them easily as a human would, this allowing the Docket to complete tasks efficiently without taking screenshots every few seconds.

How we built it

We built Docket with React, Typescript, Node.js, and Websockets creating a chat interface that allows users to use chat with Docket, powered by Deepseek, to preform ordered task queues. The agent analyzes each webpage's sematic DOM, and controls a Chromium browser through Playwright.

Challenges we ran into

Many website elements when shown in text form appear the same to each other, making Docket commonly go for the wrong element. We managed to fix this by giving Docket an option to take a visual screenshot instead of a sematic one. Many modern websites also have strong anti bot features like captchas and rate limiting, which can stop the Docket Agent in its tracks. We attempted to solve this but incorporating a manual mode the agent can switch to, which allows it to ask the user for help. Though Playwright browsers are extremely laggy especially when hosted inside a website, so even manual actions are hard to control.

What we learned

Our understanding of how AI reacts differently to inputs based on the prompts you give it. While Deepseek might not be able to click on a enter button on an input bar, we can teach it to use keybinds like enter to send a message instead.