EdgeSense

## Inspiration 1 billion+ people globally live with visual, hearing, or speech impairments, and most assistive apps need constant internet — useless in hospitals, rural areas, or low-connectivity zones. We wanted one offline-first app that helps blind, deaf, and mute users, running entirely on-device using ARM-optimized AI.

What it does

EdgeSense is an offline Android accessibility app powered by 3 on-device AI models, covering blind, deaf, and mute users:

1. Object Detection + TTS model (YOLOv8n/TFLite) – detects surroundings and speaks them aloud for visually impaired users.
2. Speech-to-Text model (Vosk) – converts speech to text in real time for deaf/hard-of-hearing users.
3. Sign Language Recognition model (ASL/ISL trained) – converts sign gestures into text/speech for mute users.

All 3 models run fully offline on-device, optimized for ARM deployment — zero internet dependency, full privacy, low latency.

How we built it

We split into a 4-person team. Vision and speech modules were built and trained independently, then integrated into a single Android app, optimized and quantized for ARM deployment, and packaged into one final APK.

Challenges we ran into

Merging two independently-built AI pipelines into a single low-latency offline app, keeping model size small enough for on-device ARM inference without losing accuracy, and syncing TTS/STT timing for a smooth accessibility experience.

Accomplishments that we're proud of

A fully working offline, ARM-optimized accessibility app covering three disability categories in a single build — no cloud, no internet required.

What we learned

Deep-dive into TFLite quantization, ARM edge deployment constraints, and building AI pipelines that are inclusive for multiple accessibility needs at once.

What's next for EdgeSense

Expanding language support for regional Indian sign languages, improving detection accuracy in low-light, and exploring hardware acceleration (NPU/Jetson) for even faster on-device inference.

Built With

• android
• android-studio
• arm
• computer
• java
• kotlin
• machine-learning
• python
• tensorflow-lite
• tts
• vosk
• yolov8