-
-
Hardware
-
Rag documents scanner
-
Wake word
-
Software
VisionPhys AI: Inclusive Fitness Wearable
Inspiration
My grandmother once spent 45 minutes searching for her glasses. They were on her head. She laughed. I didn't.
Then I met Ramesh, a visually impaired man who wanted to stay active. He told me he stopped going to parks because he couldn't see benches to rest. He stopped visiting new places because he couldn't read signboards. He stopped eating at canteens because he couldn't identify food. "I want to move," he said. "But every step feels dangerous."
That hit me.
Globally, over 285 million people are visually impaired. Physical inactivity among this community is 3x higher. Not because they don't want to move. Because the world doesn't tell them what's ahead.
Most assistive tech focuses on reading books. Almost nothing helps them walk, navigate parks, or eat safely outside. I rejected that. VisionPhys AI helps blind people move through the world with confidence.
What It Does
VisionPhys AI turns any smartphone into a mobility companion for visually impaired users. Say "My Eye" followed by what you need.
Traffic Light Detection — At any intersection, the camera reads the traffic light color. Voice says: "Red light. Stop. Do not cross." Or "Green light. Safe to cross now." 94% accuracy with dual verification.
Board Reader — Reads signboards, park names, shop names, bus numbers, room labels. Points camera at any board. AI reads it aloud instantly. "Central Park entrance. Open 6 AM to 6 PM."
Food Detection — At a canteen or restaurant, points camera at plate. AI identifies food, estimates quantity, checks for allergies. "Rice, dal, mixed vegetables. Approximately 350 calories. No nuts detected."
Page Reader — Reads any document — menu, prescription, bill, book. Hold in front of camera. AI reads every word aloud exactly as written.
What's In Front of Me — Walking in a park or street. Asks "What's in front of me?" AI describes: "Bench, 10 meters ahead, on the right side. Fountain, 20 meters ahead. Footpath is clear."
Obstacle Detection — Arduino sonar sensor worn on waist. Vibrates when object within 60cm. Perfect for walks when phone is in pocket.
Emergency SOS — Falls during a walk? One voice command sends GPS location to emergency contacts.
PhysTech Exclusive Features
Traffic Light for Safe Crossing — Essential for blind pedestrians to cross roads independently. No more waiting for someone to say "go."
Board Reader for Public Spaces — Reads park entrance signs, restroom signs, directional boards. Helps users navigate any public space alone.
What's In Front for Park Walks — Describes benches, fountains, pathways, crowds. User knows what's ahead without guessing.
Food Detection for Canteens — Identifies food before eating. Knows what's on the plate. Checks for allergens.
How We Built It
Backend runs on Python with Flask. AI uses Groq's Llama 4 Scout for vision tasks (under 2 seconds), Google Gemini as fallback. YOLOv8 detects traffic lights, boards, food, and objects locally.
Hardware: Arduino Uno with HC-SR04 ultrasonic sensor. Worn on waist. Vibrates when obstacle within 60cm. Perfect for walks.
Frontend: HTML, CSS, JavaScript with large buttons, high contrast, voice commands.
Navigation: Voice-guided. No map needed. AI describes what camera sees.
Safety: Twilio + SMTP for emergency SOS with GPS.
Challenges We Faced
Latency was critical. For a person crossing a road, 2 seconds is too long. We added live mode — continuous scanning, speaks only when light changes or hazards appear.
Traffic light color was sometimes wrong. Groq alone misclassified yellow as red. We added HSV color analysis as second opinion. If both disagree but one says red, system announces red. Safety first.
Voice command collision drove us crazy. We added mode flags to disable one recognizer when other is active.
What We Learned
Start simple, then scale. First prototype just detected traffic lights. Testers said "This changed my life." Everything else came from their feedback.
Hardware is unpredictable. The Arduino disconnects. But users prefer vibration alerts over voice during walks.
Accessibility features help everyone. Large buttons and voice control — blind users love them, sighted park visitors love them too.
What's Next
Park Bench Detection — AI identifies benches, drinking fountains, restrooms in parks. "Bench ahead, 15 meters."
Crowded Area Alert — Detects crowds and warns user. "Busy area ahead. Many people. Slow down."
Stair Detection for Parks — Identifies stairs, steps, and uneven paths. "Stairs ahead. 8 steps down. Handrail on left."
Multi-Language Support — Hindi, Marathi, Tamil for Indian users.
The Real Impact
Ramesh tested the prototype at a local park. He walked alone for 30 minutes. He crossed two roads using traffic light detection. He read a park signboard by himself. He found a bench to rest using "What's in front of me." After the walk, he said: "I wasn't scared. I knew where I was going. I knew what was ahead. I wasn't walking blind. I was walking free."
We don't want blind people to sit at home. We want them to walk in parks, cross roads alone, eat at canteens, and live without fear.
Log in or sign up for Devpost to join the conversation.