HelloHelp

Inspiration

Yesterday, at the airport before our flight here, we were stopped by an elderly lady who needed to upload a photo ID onto her laptop in order to board the plane. Though the solution seems simple to us, it isn’t quite as easy for many others out there, and without us, she might’ve missed her flight. Thinking about this, we decided to create a product that inspires technological independence for all.

What it does

In summary, we're developing HelloHelp, a desktop application that will assist users in understanding interfaces and completing computer-based tasks. You can message HelloHelp anytime you're stuck, and it will image parse your screen and guide you. As demonstrated, we provide step-by-step, localized annotations—recommending actions to take. If preferred, you can even choose automated completion and let the computer handle everything.

How we built it

The process involves GPT-4 Vision to parse images and Google Vision OCR to automatically complete your request resulting in a product that educates and provides users with the most personalized, convenient, and direct solution—which no other product in the market can achieve.

What's next for HelloHelp

We'd like to fully automate the process across the entire desktop, without limits to the application. Additionally, we'd like to develop a mobile app that allows us to extend our service to mobile applications as well. This would also provide a way for users to track their progress, seeing how many tasks they completed in a specified time block, and also the problems they ran into today paired with the provided steps to solve them, for if they experience the same pain point in the future.