Inspiration
The inspiration came from me previously building a monitoring tool for my Windows 10, which could track different processes and generate a pie chart which shows the usage percentages for each application throughout a single session
What it does
This application captures different events from user interaction and different event listeners such as mouse clicks, scrolls, drag, keyboard presses, releases, process start and termination and generates valuable insights, tips, summaries, warnings, recommendations based on these events
How we built it
The application was totally built in mind with local usage. The entirety of the application is divided into major sections. Event collectors, data ingestor, Event fetcher, Prompt builder, AI inference and the frontend application. Everything is written in Python, from the backend (FastAPI) to the frontend (PySide6 + Qt). GPT-OSS-20B quantized model was used for generating inferences, insights, tips and summaries
Challenges we ran into
- Lack of resources - Running on a local system means, scaling of resources is pretty non-existent. Not having a workhorse does slow down the entire processing flow.
Accomplishments that we're proud of
The design was really put down perfectly into place. Rather than just building a scrappy codebase, everything was organized into modules, trying to simulate a local microservices architecture.
What we learned
The one major learning was model handling, and still didn't get to do it that well! Lot of refinement and fine tune needs to be done to the model to actually generate really really great insights that can be of industry use.
What's next for Local Assist, Event Tracker
- Fine-tune the prompts to generate better insights
- Modify the system to construct new information and form insights based on the given event data
- System resource monitoring (CPU temp, system vitals etc.)
- Try to add infographics, perhaps!
- Detection and notification of suspicious background input and processes (could be a really valuable MSP)
- Improve UX
- Other stuff that I haven't thought about :)
Log in or sign up for Devpost to join the conversation.