dot.UI

Setup

Inspiration

The proliferation of AI in society following the debut of ChatGPT and the prevalence of machine learning within the computer science field inspired our group to pursue the ML/AI track for MHacks16. We also love the hacky startup culture that surrounds Ann Arbor and wanted to develop features for a cutting edge product.

What it does

dotUI is an modular user interface that connects the blind and visually impaired community to the powerful tool of ChatGPT, a OpenAI vision powered pdf reader, and a vision powered Wikipedia client.

How we built it

We utilized local startup NuHaptics "HAP" api that simulates their marquee multi-line braille display, this API allowed us to rapidly prototype and iterate on our ideas with a hardware digital twin without needing to have the expensive physical setup. Our team brainstormed what the most powerful tools for the tablet could be, and then we started building.

Challenges we ran into

There were many individual coding challenges along the way, but the higher level project challenges were all spawned due to the 24-Hour build requirement. Because we were dealing with many of these problems for the first time, we dove head first into problem solving without knowing if we were taking the most efficient path to the solution, and as the hours wore on it became more challenging to focus on writing clean code instead of something that "just worked".

Accomplishments that we're proud of

For three of our four members MHacks16 was our first hackathon and we made something that worked, a functional product that feasibly could change how many members of the blind community interact with tools such ChatGPT, images, PDF's, and Wikipedia.

What we learned

Building is a process, a skill, and a muscle. As the hours wore on throughout Saturday we all ramped up our production as our confidence with the idea and tech stack increased. We all became more skilled using a number of different libraries including OpenAPI, Wikipedia_API, PyQT. Finally, the more time that we spend building, the less friction there will be in the process and the more efficient we will all become.