Duo

What it does

Duo passively learns data entry automations as queries on a knowledge graph representation of the UI by just observing the user interact with their computer. After it learns the task, it automates it by clicking and typing on the UI.

How we built it

We used TigerGraph in Python along with Windows Accessibility service for the UI layer. We started with ideation where we thought from the perspective of what we could do that is the most impactful for this challenge. Then we thought from the perspective of how we could best use TigerGraph. From there, we delegated responsibilities, created a plan and got to work. We used grpc to communicate with the computer and weren’t able to get all of our items down.

Challenges we ran into

We struggled to get some things running on an M1 Mac. Something that we could have improved is having a smaller scope of the project. We had huge aspirations to build a robust system and worked towards that starting from the system architecture and in the way that we were using TigerGraph. Towards the end we realized we needed to cap off what our goals were and scope down the plan. Even still, we found it hard to get everything done by the deadline and should have been a bit less ambitious in our project goals.

Accomplishments that we’re proud of

We are super proud of the fact that we came up with an idea that could impact many people across the world in a highly positive way. We worked really hard to get done what we were able to do and were proud of the collaboration that we managed to achieve working as a team. We are all happy that we were able to learn a new technology that we will likely use in the future. We are proud of the fact that we were able to set the goal of working together and getting this hackathon done by the end date and that’s what we did.

What we learned

We learned many things. We learned about each of us closer - our working styles and were able to feed each other and learn from each of our strengths. We also learned a lot more about graph programming and TigerGraph as a whole. Overall, we learned that anything that is worth doing will be hard to do and that’s a feature not a bug.

What’s next for Duo

We enjoyed working with TigerGraph and will become a tool that we use in the future. We hope to continue to working on the project moving past TigerGraph and smooth out things we weren’t able to by the submission period. Those things include getting actual input instead of simulated input up which increases the complexity tenfold but also allows for a more ubiquitous experience.

Built With

• c#
• python
• tigergraph