It's an idea we conceived a while ago but never had the cause to act on until this opportunity motivated us. Two of our team members are or are closely related to someone who struggles with dyslexia and who would find many things to be more accessible if they had a reliable way of interpreting signs and paper instructions. Which resulted in our discussion of designs that could provide such a service.
What it does
We have made a glove that reads out texts from signs and messages found in the street. This allows people who may be partially sighted or finds it difficult to read, to become much more self-sufficient and find public places more accessible.
How we built it
The hardware is just a raspberry pi 4, a camera, a button, a power supply, and a glove. We met up in school and in a team member's garden the next day to become familiar with the setup. Then developing the software remotely at home using voice calls to collaborate as we developed our respective functions of code to be tested remotely using VNC viewer.
Challenges we ran into
We were originally going to use a raspberry pi zero as they use less power, are a lot cheaper, and take up less space as well as getting less hot but we found it so painfully slow to develop and use, we upgraded to a raspberry pi 4. We also faced challenges in being unable to meet up due to the pandemic, which VCN viewer and our school providing coronavirus testing kits so that we could meet up safely. In terms of software, it was originally quite slow as were using taking pictures saving them somewhere, and then opening them using OpenCV. We solved this by connecting OpenCV to the raw feed from the camera. This made it much faster.
Accomplishments that we're proud of
We are extremely pleased with how well our product works in terms of scanning in text, filtering out incorrect text, and reading it out.
What we learned
We’re proud to have developed using OpenCV, tesseract, and VNC viewer which are all skills we were quite intimidated by before the hackathon and are now more proficient in, which we may not have ever learned without the hackathon.
What's next for Reading glove
Ideally, we would get this working on a smaller chip such as the rp2040, which would require further developing the schematic we have been working on. It would result in it using far less power and being far more portable. You can see this schematic in the images above.