Tech Bro

Doing everyday tasks is such a massive waste of time. Who wants to constantly respond to emails and figure out different issues? Not me. Use Tech Bro to automate the boring day to day tasks.

Comment

Inspiration

What inspired this project was a late-night brainstorming session. We were pressed for ideas and were getting kinda lazy scouring the internet. Then we started asking AI to come up with ideas for us, but that wasn't really how we wanted to do it, we wanted to search ourselves but at peak efficiency. We wanted a way for us to tell an AI how to use our computer for us to do some easy tasks.

What it does

It uses a translucent overlay on the user's screen to act as a to-do list. The code will then go through the to-do list explaining how to do each task while completing it for the user. It uses AI to take the to-do item to convert it into a task to perform on the computer by analyzing what is on the screen and where to move the mouse.

How we built it

We built it by using this workflow:

1) Take screenshot 2) Send screenshot and to-do list to the pre-prompt 3) Add lines to the screenshot. 4) Run 11labs on the output of the pre-prompt 5) Send the line screenshot with the output of the pre-prompt to the main prompt 6) Run the output of the main prompt. Do everything again until the task is complete

Challenges we ran into

We had a time when legit everything was not working. The entire workflow, top to bottom was broken. Some issues were that the cursor was always missing and moving to the wrong place, the screenshot and the actual computer size were different, dependency conflicts, the images were not being properly saved, and there wasn't proper grid overlay. Everything that could have broken had broken at some point.

Accomplishments that we're proud of

We are proud of getting the anthropic API to work and being able to properly parse the screen and move the cursor. Most of the functionality is working there are just some slight bugs and tweaks that need to be made. Overall we are still proud of everything.

What we learned

We learned a lot about not being overly ambitious without our project. Our target goal with very high and we were not able to get everything completed in the required time. We also learned a lot about the next.js framework and different API calling methods.

What's next for Tech Bro

If this project gets completed it can be a computer app that users and download and assist them in their daily lives. It can be very helpful for senior citizens who don't know a lot about computers and need a friendly assistant.

Built With

Share this project:

Updates