WalkInMyShoes

WalkInMyShoes — AR scans any space for ADA compliance, VR lets you feel every barrier. Powered by Amazon Bedrock + Rekognition. Accessibility, finally personal.

Comment

About WalkInMyShoes

Inspiration

The idea for WalkInMyShoes was born from a deeply personal observation. Walking through public spaces every day, it's easy to take accessibility for granted — until you stop and look through someone else's eyes.

India has 2.68 crore people living with disabilities, yet fewer than 5% of public buildings comply with the Rights of Persons with Disabilities (RPWD) Act, 2016. The gap isn't just legal — it's empathetic. Architects, engineers, and facility managers know the rules, but they've never truly felt the barriers.

Traditional accessibility training is passive — slide decks, PDFs, compliance checklists. A person reads about wheelchair ramp gradients but never feels the physics of navigating one. They learn about hearing loss in theory but never experience the disorientation of muffled, directionally-confused sound in a crowded room.

Meanwhile, professional accessibility audits cost ₹50,000–₹2,00,000 per building, putting compliance out of reach for small businesses, schools, and NGOs.

We asked ourselves: What if AI could make empathy scalable — and compliance instant?

What We Learned

This project was a crash course in the intersection of AI, immersive tech, and real-world impact.

On the AI side:

  • How Amazon Bedrock's inference profile system works (us.amazon.nova-lite-v1:0 vs regional endpoints)
  • Structuring prompts so Nova 2 Lite returns deterministic, parseable JSON for audit results while remaining conversational for chat
  • Running DetectLabels and DetectText in parallel using Promise.all() for sub-second Rekognition throughput
  • The surprising depth of ADA 2010, WCAG 2.1, and RPWD Act 2016 — there are precise mathematical tolerances:

$$\text{Ramp gradient} \leq \frac{1}{12} \quad (\approx 4.76°)$$

$$\text{Door clear width} \geq 32\text{ in} \quad (\approx 813\text{ mm})$$

$$\text{Reach range height} \in [15\text{ in},\ 48\text{ in}] \quad (\approx 381\text{ mm} - 1219\text{ mm})$$

On the frontend side:

  • Building real-time disability filters with WebGL fragment shaders for 5 stages of visual impairment
  • Simulating sensorineural hearing loss using the Web Audio API BiquadFilter with frequency rolloff:

$$H(f) = \frac{1}{1 + j\left(\dfrac{f}{f_c}\right)^n}$$

  • Integrating WebXR Device API for headset-ready VR on the same codebase as the browser version

On AWS architecture:

  • The power of a truly serverless stack — zero servers to manage, pay-per-use billing
  • How IAM least-privilege roles eliminate the need to expose any API keys in the browser
  • CloudFront's edge caching strategy for near-instant global load times

How We Built It

The stack was designed around one principle: everything in the cloud, nothing on the device.

Architecture

Browser / Mobile
      │
      ▼
Amazon CloudFront (CDN · HTTPS · E2FKI267871EMW)
      │              │
      ▼              ▼
  Amazon S3       API Gateway (/ai · x5rbnqm2v4)
  (Frontend)           │
                       ▼
                 AWS Lambda (walkinmyshoes-ai · Node.js 20.x)
                  /          \
                 ▼            ▼
        Rekognition        Bedrock
    (DetectLabels +     (Nova 2 Lite ·
      DetectText)      us.amazon.nova-lite-v1:0)

The AR Audit Pipeline

  1. User taps "Scan Now" — camera frame is base64-encoded in the browser
  2. Lambda receives the frame and fans out two parallel AWS calls:
    • rekognition.detectLabels() — objects, bounding boxes, confidence scores
    • rekognition.detectText() — signage, labels, exit markers
  3. Results merged and sent to Bedrock with a structured system prompt containing the full ADA 2010 + RPWD Act 2016 clause reference
  4. Nova 2 Lite returns a JSON audit object: issues, severity, ADA section references, fix cost estimates
  5. Frontend renders color-coded bounding boxes on the live camera feed: 🟢 compliant · 🟡 warning · 🔴 violation

The VR Disability Simulations

Each simulation uses Three.js + WebGL 2.0 with custom post-processing:

Disability Technique
Visual Impairment Gaussian blur shader — 5 progressive stages
Hearing Loss Web Audio BiquadFilter — high-freq rolloff + reverb
Motor Disability Custom physics — wheelchair turning radius + ramp resistance
Color Blindness SVG color matrix filters — Deuteranopia, Protanopia, Tritanopia, Achromatopsia

The AI Expert Guide

Every simulation and AR scan has a persistent chat panel powered by Amazon Bedrock Nova 2 Lite. The Lambda function maintains conversation context using a rolling message history, and the system prompt instructs the model to:

  • Always cite the specific ADA 2010 section (e.g., §404.2.3)
  • Reference RPWD Act 2016 clauses for Indian context
  • Provide WCAG 2.1 success criteria for digital accessibility questions
  • Estimate remediation costs in INR when suggesting fixes

Challenges We Faced

1. The Bedrock Model ID Maze The single most time-consuming issue was discovering that amazon.nova-2-lite-v1:0 throws an "on-demand throughput not supported" error — you must use the cross-region inference profile us.amazon.nova-lite-v1:0. This cost us nearly a full day of debugging.

2. Rekognition + Bedrock Payload Structuring Nova 2 Lite expects messages as a JSONArray of content blocks — passing a plain string causes a Malformed input error. Getting the exact schema right ([{ "text": "..." }]) required careful reading of the Bedrock API docs.

3. Making JSON Deterministic Bedrock occasionally wraps responses in markdown code fences (json ...). We built a sanitisation layer in Lambda to strip fences before JSON.parse() — without this, the audit results would silently fail.

4. WebGL Performance on Mobile Running Three.js disability shaders at 60fps on desktop was straightforward, but mobile browsers hit memory limits quickly. We implemented adaptive quality scaling — detecting GPU tier at runtime and stepping down shader complexity to maintain a stable 30fps on mid-range Android devices.

5. CORS + Lambda Proxy Integration API Gateway's CORS configuration is deceptively complex with Lambda Proxy integration. Headers must be returned in the Lambda response itself, not just configured in API Gateway — a subtle but breaking distinction.

6. Accessibility Compliance Depth Writing a system prompt that reliably produces ADA-accurate audits required us to study the actual ADA 2010 Standards for Accessible Design in detail. The AI is only as good as its context — we embedded hundreds of specific tolerances and clause references directly into the Bedrock system prompt.

Built in India. For India. And every place in the world where a wheelchair can't get through a door.

Built With

Share this project:

Updates

Submission history