SightMate

Inspiration

More than 95% of the websites are failing to pass the accessibility standards. For millions of people with vision impairments, Internet is hard to "surf".

As developers ourselves, we know that the lack of accessibility does not come from a lack of caring. It happens, as those small markdowns and changes are easy to miss. So, we started thinking about how to build a tool that resolves the issue. And most importantly, how to build the tool that does not just report the issue, but points it out on the page and provides a solution that aligns with the context.

What it does

SightMate is a Chrome extension that analyzes websites for improvements that can be made to make it more accessible for the visually impaired. It prints out a list of the website's issues, where the issues are located in the code and the changes to be made.

How we built it

We used React.js to build the website, axe-core library to analyze the code for issues, and Google Gemini API to provide contextual fixes.

Challenges we ran into

Google Gemini API keys have a limited amount of requests that you can do per day for free (capped at 20), which made it difficult for us to test the product and make adjustments. Overall, we have generated 5 API keys from 5 different Google accounts in order to debug our code.

Another challenge that we faced was when we thought of providing URLs for analyzing websites. How could we make the extension so does not close when we readdress the user to another page? Cool thing is that we now know how to do it (by opening an extension in a side menu, which is super easy), but the URL feature is not going to be implemented because of the time constraint :( But now we know!

Accomplishments that we're proud of

We actually just built our first extension! Opens a lot of possibilities for us (it's actually a cool thing, and you can do a whole lot with it, try it out).

And we also built something nice with a use of Gemini API.

And it works!

What we learned

Gemini AI has a lot of different models that can be used in different contexts, and those models give you different responses. Even if you make a script to be very strict, and ask Google Gemini AI to follow the format, there is going to be small changes in between responses. On top of that, models have their own capabilities and limitations, including optimization limitations. So it is really important to understand which models to use and why before you start to implement a project.

What's next for SightMate

We hope to build a feature for users with visual impairments that will refactor the page they are on up to all accessibility standards. That way, people can access the pages and use them without any delay (well, maybe after a minute or so).