ctrl-flow

Inspiration

Our project had two main inspirations. The first being IDEs in the pre-AI coding era like Vim and VSCode, where being an efficient coder looked like using shortcuts like it was second nature. We also were inspired by the 3D design tool Blender, specifically the ‘Shift+R’ repeat-past-action command and the easy-to-use pie menu. The goal was to bring the efficiency provided by powerful shortcuts into the AI era and everyone's workflow, without the steep learning curve typically associated with them.

What it does

Ctrl-flow is a lightweight layer on top of a desktop or browser that categorizes and transforms ordinary, user interactions on their computer into generalized, data-driven commands without the user having to explicitly define them. In more express terms, the tool works by segmenting a user’s inputs into specific “Actions,” stripping them of any miscellaneous activities like dismissing notifications or fixing typos. These actions are then transformed into an intermediate .json representation, where specific information like highlighted text or starting application/URLs are separated from the general control flow of the action. The former are stored as variables with default values, while the latter is transformed into an executable script, which can be assigned a shortcut. The next time the user decides to do a similar action (or if they set the app to ‘Eager’ mode, which enables automatic command calling), they could simply highlight some text/change a URL and press the assigned hotkey to automatically run the same action again, with different data passed in as variables. The repeat command is one of 6 official ctrl-flow hotkeys, but we also allow the user to save and define as many custom hotkeys as their hearts desire (or keyboards can handle).

How we built it

We came into this hackathon with a loose preference for the ‘Cook Smarter’ Track and a boatload of ideas (some of which were better than others). The three of us spent our first hour or so running experiments to see what kinds of simple productivity tools we thought were useful for students like us. After this first hour, while Mohit and Aryan continued experimenting and refining their tools, Advaith took the results of our best ideas and tried to find the thread running through all of them, which culminated in our master project specification .md file. After some iteration on the spec, we got to work implementing and testing the bulk of ctrl-flow, while immortalizing some of our favorite initial ideas into our built-in shortcuts. Finally, we designed a logo, cleaned up the UI, and made sure things worked as smooth as possible.

Challenges we ran into

Our process was remarkably smooth! According to Aryan, his biggest challenge was spending time working on the project and not hanging with the animals out in front of the HUB. Leading into the "what we're proud of section," we think we were able to define a scope that was reasonable enough to be completed while novel enough to be exciting.

Accomplishments that we're proud of

This is the first hackathon any of us have ever done. We’re proud that we were able to create a serviceable product that we think has the legs to be genuinely useful for a lot of people, while having an incredible time along the way.

What we learned

Aside from the niche, technical aspects of the project like learning how to use CEF to build an app that works on the desktop as well as in the browser and figuring out real-time data processing, our biggest lesson was to “kill your darlings,” and that sometimes it’s necessary to explore a few good ideas before you can conceive a truly great one.

What's next for ctrl-flow

“Big moves are coming” – Aryan Rodrigues. We’ve already reached out to some of the other teams at DubHacks and got a couple of inquiries to work together to develop the app into a full product. On the technical side, our hackathon project involved the use of LLM's like Gemini for semantic processing, but we recognize that a properly trained QA, classification, or other type of model would be sufficient for a lot of the steps. In the best case scenario, after further development we are able to make running the app entirely locally feasible for most devices. Ctrl-flow is an idea that we’re really excited about, and we genuinely believe that we can be the ones to write the next chapter for computer shortcuts, and build the UX on top of the exciting technologies brought about by the future of AI.

Note

Quick thank you to the Dubhacks RSO and sponsors! The event was an amazing first experience and we're excited to come back again next year - Advaith, Aryan, & Mohit

Built With

Submitted to

DubHacks '26

Created by

I worked on the Chrome extension: the pie menu with one-key commands, Repeat/Keep macros, Find PDF, Focus Flip, auto project spaces, meeting prep, and the AI standup with voice. I also set up the AI proxy on Cloudflare, connected Tiger Data and ElevenLabs, integrated everyone’s code, and led testing and privacy safeguards.

Mohit Ramesh Kumar
I worked on the desktop app, macro recording.

Aryan Rodrigues
I worked on defining the project specification, the structure of the project, and defining the high-level logic of the system. I also took the lead on the design of the project's branding and our pitch for judging

vankadv