Accessibility4all

A family member of a contributor is dyslexic, and we have seen first hand how difficult it is to find accessibility features that work consistently. We wanted to create an open source all in one software to improve accessibility in technology use, and allow many features to be added under a unified program.

We knew we wanted a modular program, so our approach to building was based around creating a project structure that kept various features entirely separate from one another to allow for new features to be added. We added a template and guide for feature development to ensure than any potential contributor can easily understand how to design their code.

For the voice control feature, our original idea was to analyze the element names in a website to determine what to click, however the issue was that elements were named differently than anticipated and it was not a very reliable way to use a website. To circumvent the issue we decided to make our program convert speech to text, take a screenshot of the page the user was looking at, which was then sent to a Grok via API which would then return coordinates of where the program should click.

For the dyslexia font feature, one of the main features we wanted to implement was to force all fonts into a dyslexic friendly one; chrome has no feature to override fonts in websites, so we made the python program add an option to open up chrome with a custom made chrome extension that overrides all fonts.

Additionally, we built a page reader, keeping those who can’t see in mind. With the page reader we had two main goals. First, to allow the user to have a better understanding of where their mouse is located and allow them to more naturally interact with their device. The second goal was to implement AI to make our program stronger than the average page readers other companies offer. For example, our page reader uses a screenshot of the user’s screen and uses Groq AI vision model to return a summary of the main content of the screen rather than reading each and every non-important word. This was built using python

For the Tone and Social Cues feature, we wanted to help autistic users and anyone who struggles to read implied meaning in text. Select any text and press a hotkey, and a panel appears breaking down the tone, sarcasm, indirect requests, and other social subtext. The main challenge was prompt engineering: LLMs tend to confidently invent subtext that is not there, which would be harmful for someone relying on this tool. We constrained the model to always hedge its language, quote the exact words behind every cue, and return an empty result when the text is plainly literal.