A team member saw their mom working and repeating the same steps over and over on her computer. Open an excel sheet, copy 10 lines, tab into another sheet, paste, scroll, repeat. These monotonous tasks are a time waster which is terrible especially since the technology exists to automate these tasks. The problem is that most people aren't aware, have forgotten, or find it to complicated which is why we aim to reintroduce this technology in a simple manner that anyone can use. An application that could automate such repetitive tasks would not only save time but also could have other uses. For example, individuals that have arthritis may find it challenging to complete tasks on the computer that require repetitive movement on an everyday basis. We offer an application that can take these repetitive movements and simplify it to 1 click.
What it does
Our program, dubbed "Ekko", serves to:
- Reduce menial/repetitive tasks
- Increase accessibility to mouse and keyboard actions for those with diseases/disabilities
- Aids those who aren’t technologically literate Users will be able to record any series of clicks, cursor movements and key presses as they would like, and save this sequence. They would then be able to play it back at will whenever they desire.
How we built it
To create this application we used various Python libraries. In the backend we had to implement the Pynput library which allowed tracking of clicks, cursor movement and key presses. We have individual threads set up for monitoring both keyboard and mouse actions at the same time. In the front end, we used QtDesigner and PyQt5 to design our GUI. Using Python Libraries/Software: Pynput, PyQt5, QtDesigner, Multithreading Used Pynput to track and record cursor movement and keypresses and clicks Used PyQt5 and QtDesigner to design the GUI
Challenges we ran into
Initially, without the use of the Multithreading library our program would crash after one use. It took a lot of time to figure out what the exact problem was and how to allow for recording of cursor and keyboard inputs while the Python program was running. Getting the GUI to interface with the recording/playing scripts was also annoying. A bit of a funny obstacle we encountered was that while 1 team member was following a tutorial he didn't realize they were alternating between Spanish and English while writing code which led to a few confusing errors.
Accomplishments that we're proud of
We had created a timeline for our project with deadlines for specific tasks. We were able to stick to this schedule and complete our project in a timely manner, even though we were busy. We also successfully implemented a program with a working GUI. Finally, it was the first hackathon experience for 2 of our members so it was nice to have a finished product which often doesn't happen on an individual's first hackathon.
What we learned
We learned many new tech skills such as how to create GUI's, using threads, how to track keyboard and mouse input. We also learned important soft skills such as time management. These skills will be useful in whatever future projects we decide to pursue.
What's next for Ekko: Automate Your Computer
Implementing voice recognition to make automating tasks more accessible. The movements of the cursor can also be more accurate during repetition.