Illuminate

• 

  sample website navigation (instagram) telling a user to post

Inspiration

I had a friend whose cousin was visually impaired. Her was interested in content creation specifically on different social networking applications like instagram and facebook among others but she was used to be helped by the typical solutions to provided to challenges faced by visually impaired such as integration with screen readers, ai leveraged image description and others. She was often frustrated by the little speed of screen readers and how unfriendly (/unfamiliar they can be!). She hopelessly quitted her dream of becoming the best content creator because this experience was tiresome and non realistic for her as she did not have money to hire a social media manager for her in the beginning. Illuminate now came into place to address these similar problems faced by the visually impaired by providing independent, proactive solution delivering help and assistance in a friendly manner.

What it does

Illuminate is an ai based tool that runs in the background of social media applications [like Instagram, Facebook, twitter, Tiktok,...], professional networking platforms such [like Linked In, Upwork, Fiverr,...], and other digital platforms where a digital workplace can specifically be found by providing a proactive task-based assistance (like enabling complex tasks or job applications and email scheduling tasks) and endless in-app assistance guidance(like navigating complex UIs step by step when completing a desired action) without forgetting voice-driven controls (for controlling actions in the social media platform)

How we built it

I implemented the first version of Illuminate which is a browser extension. I used Html , CSS and Javascript for the frontend with tailwind web speech API for speech recognition and for tasks of generating post captions, summarizing job application resumes, responding to commands and job writing and Git , Github for version control

Challenges we ran into

I intended to use Whisper Api which is very accurate for speech recognition but encountered a strong barrier of how the Whisper API from OpenAI requires payment, as it’s part of OpenAI’s paid API services and not available for free initially. This forced me to work on the last minute pressure and not finish well my browser extension due to the use of webspeech for the last moment and it required a learning curver

Accomplishments that we're proud of

I am proud that I was able to develop a concept of using technology to empower visually impaired users by providing a friend like tool that helps them to be fit in the future workplace which is the digital workplace and i am proud of being able to prove that entertainment can not only be achieved to those who see only but also those who can not can perceive it in feelings!

What we learned

By building illuminate I learnt simple speech API integration for a seamless accurate User experience and we learned how to recognize the importance of cherishing and giving privileges to impaired individuals specifically (visually impaired) by learning their pain and how to solve it to make them involved in the world of digital workplace by making them fully independent

What's next for Illuminate

Building Illuminate with Whisper API for most accurate speech recognition and develop the real react native based mobile application that will support both ios and android platforms and include OpenAI's turbo api for a powerful API for the tasks related operations. I want to make illuminate a powerful and know tool to all people and am hence going to improve certain features such as coaching individuals remotely with technical and other skills needed on the digital market place, partnering with different NGOs to help in making the illuminate application popular and used by these individuals

Built With

• css
• git
• github
• html
• javascript
• webkitspeechrecognition