Smart Accessibility Reader

Inspiration

Access to information is a fundamental right, yet millions of visually impaired individuals still face barriers when interacting with printed or handwritten content. While screen readers and OCR tools exist, they often fall short in simplifying complex documents or making them truly accessible in real-world contexts like printed brochures, letters, or forms.

This project was inspired by the need to bridge that gap to create a tool that not only reads documents but also understands them and communicates their meaning in a simple, spoken format. By combining AWS services like Textract and Polly with the power of OpenAI's language models, I aimed to build something human-centered, practical, and inclusive—a small step toward making information truly accessible to everyone, regardless of their vision.

What it does

The Smart Accessibility Reader allows visually impaired users to understand printed or handwritten documents through a seamless, AI-powered experience.

📸 Image Upload Users upload a photo of any document—printed text, handwritten notes, forms, etc. 🔍 Text Extraction The app uses Amazon Textract to detect and extract text from the image. ✍️ AI Summarization The extracted text is sent to OpenAI’s GPT model, which rewrites it in clear, easy-to-understand language. 🔊 Text-to-Speech Finally, the summarized content is converted to speech using Amazon Polly, allowing the user to listen to it.

All this happens through a clean, accessible interface enabling visually impaired users to access, understand, and engage with written information effortlessly.

How I built it

I built the Smart Accessibility Reader as a full-stack AI solution using a combination of AWS services and modern APIs:

• Frontend Interface: Developed a clean and accessible UI using Streamlit to allow users to upload images and interact with the tool easily.
• Image Upload & Storage: Uploaded document images to an Amazon S3 bucket directly from the frontend using Boto3.
• Text Extraction (OCR): Used Amazon Textract to extract text from the uploaded images. This allowed me to handle both printed and handwritten text effectively.
• Summarization with AI: Instead of using Amazon Bedrock, I integrated OpenAI's GPT-3.5 API to simplify the extracted text into easy-to-understand language for visually impaired users.
• Text-to-Speech: The summarized content was converted to natural-sounding audio using Amazon Polly, making the final output accessible through speech.
• Integration & Logic: Used modular Python scripts to organize the OCR, summarization, and TTS logic, then connected everything through the main Streamlit app to create a seamless end-to-end experience.

Challenges I ran into

🧠 Bedrock Limitations Initially planned to use Claude via Amazon Bedrock, but ran into invocation issues due to inference profile requirements on Claude 3.5. Switched to Hugging Face Transformers for smoother integration under time constraints. 🖼️ OCR Accuracy on Complex Images Some handwritten or low-contrast images were difficult for Amazon Textract to process. Required testing multiple formats and preprocessing steps to ensure reliability. 🎧 Audio Playback in Streamlit Integrating real-time MP3 playback from Amazon Polly into Streamlit needed workarounds to properly stream binary audio data and ensure cross-browser compatibility. 🔐 Secure AWS Integration Managing credentials and permissions (for Textract, S3, Polly) in a way that’s both secure and developer-friendly was tricky—especially while balancing rapid development. 🧩 End-to-End Orchestration As a solo developer, stitching together multiple AWS services, OpenAI, and Streamlit in a modular, clean pipeline while maintaining accessibility and performance was a major challenge—but also the most rewarding part of the build.

Accomplishments that I'm proud of

As a solo developer, I successfully built a fully functional AI accessibility tool that brings together OCR, language understanding, and speech synthesis—all integrated into a seamless user experience. I'm proud that this project goes beyond just being technically sound; it solves a real, human-centered problem by making printed and handwritten information more accessible to visually impaired users.

Building this end-to-end system using Amazon Textract, OpenAI, Amazon Polly, and Streamlit from scratch—while ensuring usability, clarity, and inclusivity—was both a challenge and a meaningful achievement.

What I learned

• Integrating AWS Services at Scale: I deepened my understanding of working with AWS services like Textract, Polly, and S3, especially around access control, real-time processing, and service orchestration.
• Prompt Engineering & AI Summarization: I learned how to craft effective prompts to get accurate and user-friendly summaries from OpenAI's language models, tailoring output for accessibility use cases.
• Modular Application Design: Building this as a solo developer taught me the value of clean code structure—splitting logic into reusable modules made the system more scalable and maintainable.
• Accessibility-Centered Thinking: More than just tech, I learned to design with empathy—ensuring that visually impaired users could interact with and benefit from the application without barriers.
• Problem Solving Under Constraints: When Amazon Bedrock’s limitations blocked my initial plan, I quickly adapted by integrating OpenAI, proving my ability to pivot and keep building under pressure.

What's next for Smart Accessibility Reader

🗣️ Multilingual Support Integrate Amazon Translate to support document summaries and audio playback in multiple languages, expanding accessibility across regions. 📱 Mobile-Friendly Version Build a progressive web app (PWA) or mobile app using React Native or Flutter to make the tool more portable and usable on-the-go. 🧠 Personalized AI Summarization Add user-specific settings like reading level, tone (formal/friendly), or summary length to deliver more tailored content. 💬 Conversational Q&A Enable a follow-up chatbot interface where users can ask questions about the document and get natural answers using OpenAI. 🗂️ Document History & Cloud Save Add login + storage so users can save and revisit their document summaries and audio files. 🔐 Privacy-Focused Deployment Implement on-device or edge processing for organizations with strict privacy needs, like healthcare or legal.

Built With

• amazon-web-services
• huggingface
• python
• s3
• streamlit