Code:Star

A solution for individuals with cognitive disabilities to convert classroom material into audiobooks, making it easy for users to absorb information.

Comment

Inspiration

Dyslexia affects approximately 10% of the population, making it difficult for individuals to read, write, and comprehend text. These challenges can lead to lower academic achievement and decreased self-esteem.

However, technology can provide solutions for dyslexic students to overcome these barriers. Our mobile app can be a game-changer for dyslexic students, providing them with a tool to convert class material into audio books. This allows them to learn and absorb information more easily and efficiently.

Research has shown that using audiobooks can significantly improve reading comprehension and accuracy for students with dyslexia. According to a study published in the Journal of Educational Psychology, students with dyslexia who listened to audiobooks improved their reading comprehension by 76% and their reading accuracy by 39%. Furthermore, audiobooks can also help dyslexic students with focus and attention, as well as reducing their anxiety around reading.

What it does

Our App allows the user to upload images of their school/college class material and convert it into Audiobooks which they can listen to anywhere thus preventing any cognitive difficulty which is intrinsic to dyslexia. Furthermore, it allows the users to naturally train themselves to combat dyslexia and grasp the material with practiced ease via our reading and writing tests.
All of this is governed by a smart contract deployed on Polygon Testnet. Each User has a wallet with a certain amount initialised. Every action on our app has a certain cost which gets deducted from this wallet, thus monetising the app.

How we built it

The App was made in Flutter. For extracting the text from the images we use Firebase MLKit, more specifically the Text Recogniser. The backend was written in Node.js using Express.js framework. We used CockroachDB as database. For synthesising Audiobooks we are using Google Cloud's Text-to-Speech API and Cloud Storage for storing the audiobooks. We wrote the smart contract using Solidity in Remix IDE, deployed using Hardhat, Viewing Transactions/ Debugging on Alchemy, deployed on Polygon Mumbai testnet, and finally Metamask as the wallet. We used ngrok and postman for testing our APIs.

Challenges we ran into

As someone who didn't have any relevant experience with Web3 beforehand, this project was a big challenge for us. For quite a while, we wanted to use VerbWire to deploy our smart contract but that didn't work out, so to still qualify for the blockchain track we looked for alternatives. Thankfully, we found polygon's quick start guide and Quicknode's doc on deployment quite friendly and were able to make up for lost time. Setting up the text recognition inside the Flutter App was also quite hard, because we had initially planned to use certain APIs for the purpose, but its turns out that they were paywalled, thankfully MLKit had the exact functionality we needed.

Our Target Tracks:

Best Blockchain Hack: Implemented a Smart Contract-controlled wallet that governs the entire working of the App. The smart contract was written in Solidity, Deployed using Hardhat to the Polygon Testnet using the Alchemy API.

MLH - Best use of Google Cloud: We used Google Cloud's text-to-speech API to synthesise Audiobooks and stored them inside a Bucket in Cloud Storage.

MLH – Best Use of CockroachDB: We used CockroachDB as our database for the entire application. Given that our data was structured, using an SQL database made sense, and CockroachDB was a joy to work with.

MLH – Best Accessibility Hack, Sponsored by Fidelity: Our App primarily targets to make the Academic field equal for all. Dyslexic students face difficulties that our App is able to solve. We have incorporated features like variable text size, variable spacing and dyslexia-friendly fonts to make our App accessible to everyone. We also use Speech-to-Text for our Reading tests to facilitate better User Interaction. Education is a birthright of every human being, and this app is a step in the direction of making good on that promise.

What's next for Code:Star

As we progressed the development of this App, we thought of quite a lot of features to implement that we weren't able to in this Hack's time:

  • Store the Audiobooks for offline listening.
  • Enhance the Audioplayer UI according to best material design and Apple human interface principles.
  • Have an Audiobook be minted as an NFT and be used in a Social setting.
  • A community forum where experts can guide dyslexic students in a professional manner.
  • In the future, i would also like to explore the possibility of storing the Audiobooks on IPFS and streaming it from there.

Built With

Share this project:

Updates