Samvit

You were never lost. The world just didn't guide you.

Comment

Inspiration

The idea didn't start with a problem statement. It started with a scientist and a painter.

In the 1960s, Paul Bach-y-Rita demonstrated something that quietly changed neuroscience forever. She let blind people "see". A camera records what is seen through the glasses, transformed the image into electric impulses. The image is then was flashed onto the body via a tactile display. Through this vision substitution by tactile image projection, they made the blind person be able to see through his skin.

Then there was Neil Harbisson, born completely colorblind, who worked with researchers to mount an antenna on his skull that converted color frequencies into sound vibrations he could feel through bone conduction. He didn't gain sight. He gained something new entirely. He became the first person officially recognized as a cyborg, not because technology replaced him, but because it extended him.

Both of them pointed at the same truth: the brain is not fixed. The body is not the limit. The limit is whether someone bothered to build the bridge.

Samvit is that bridge. Not for the few. For the millions.

What It Does

Samvit is a two-layer assistive system, a voice AI on Android and a haptic wearable, that helps visually impaired individuals independently find, understand, and physically reach public services, starting with healthcare.

Layer 1: The App (The Brain)

The Android app acts as a Jarvis-like voice agent(we all know how cool Jarvis is). A user can say something like "Find me the nearest free clinic" and Samvit handles everything: it searches the web, reads and filters results, explains them in plain language, and can make the phone call on their behalf. Every action the agent takes is narrated aloud in real time, and the user can say "Stop" at any point to halt execution immediately. The user keeps every decision. The AI handles the complexity.

Layer 2: The Hardware (The Game-Changer)

Finding help is only half the problem. Getting there is the other half. The haptic vest uses two Intel RealSense D435i depth cameras to map the environment in real time, converting spatial information into a 12x12 grid of vibration motors worn on the body. The grid fires patterns that the brain learns to interpret as navigation signals, telling the user where to step, where to turn, and what is ahead. No guide. No dependency. Physical independence.

Together, the two layers solve both walls: finding the service and reaching it.

How We Built It

I took advantage of modern AI agents to complete tasks faster and make code cleaner and readable. Android App

I built the app in Kotlin using Jetpack Compose for the UI. The core agent loop runs on Gemini 2.5 Flash via the Genai AI, handling goal decomposition, step planning, and screen understanding. A FastAPI Python backend powers the full agentic flow for live use, while a MockVisionPilotClient ships with the app for demo purposes, simulating the complete plan, step, and confirm cycle without needing the backend running, the software was still tested and fixed so it just doesn't look good in demo's but actually works. Retrofit and OkHttp handle backend communication, Moshi handles serialization, and CameraX manages the camera pipeline.

Hardware and Firmware

The firmware is written in C++. Two Intel RealSense D435i depth cameras feed stereo depth frames into a heatmap generation pipeline. The heatmap output drives nine NXP PCA9685 PWM motor driver boards over I2C at 400kHz, each controlling 16 ERM vibration motors, 144 motors total across the 12x12 haptic grid.

Hardware Validation: Testing Without Hardware

Without physical components available, I built a datasheet-driven Hardware-in-the-Loop simulation in Python. Every component in the signal chain was modeled physically accurately, not mocked arbitrarily. Noise models were fitted to Intel's published D435i accuracy specifications. I2C protocol sequences were validated against NXP PCA9685 Datasheet Rev.4. Fault injection tested board failures, brown-outs, stuck bus conditions, and NACK storms. 1,000 continuous frames with moving obstacles ran without a single error at 0.28ms average loop time. 22 tests total. All pass.

Challenges We Ran Into

The single biggest challenge was time. One week. One person. A project that by any reasonable measure should have taken months.

A week before submission I had never written a line of Kotlin in my life. I had never built hardware firmware. I had never designed an I2C communication pipeline or worked with depth cameras. Every single technical domain in this project was new territory, learned under pressure, in real time.

The scale of Samvit compounded everything. This wasn't a single-layer app. It was an Android application, a Python backend, C++ firmware, an I2C hardware stack, and a full hardware validation suite, all built and integrated by one person inside seven days.

The hardware validation problem was its own mountain. Building something meant to run on physical components I didn't have, and making sure the gap between simulation and real world was as small as physically possible, not through shortcuts but through actual datasheet-driven accuracy, was one of the hardest technical and intellectual challenges of the entire build.

Accomplishments That We're Proud Of

It works.

That sounds simple. It isn't. Given the scope, the timeline, and where I started, the fact that Samvit functions end-to-end, the voice agent, the agentic call flow, the hardware simulation passing 22 datasheet-validated tests, is something I'm genuinely proud of.

But the accomplishment I'm most proud of has nothing to do with code.

A week ago I didn't know Kotlin. I didn't know hardware. I took on a project most people would have walked away from and I didn't. What Samvit taught me goes far beyond any technical skill. It taught me what willpower looks like when it's tested, what ambition feels like when it's inconvenient, and what you're actually capable of when time management stops being a productivity tip and becomes a survival skill.

Sometimes the best thing you can do is pick up something too heavy. You find out what you're made of.

What We Learned

Technically I learned Kotlin and Jetpack Compose from zero, agentic AI architecture with Gemini, FastAPI backend design, C++ firmware development, I2C protocol communication, depth camera integration, and datasheet-driven hardware validation methodology.

Beyond the technical, I learned that ambition without a team is still ambition. That a week is longer than it sounds if you don't waste it. That the gap between "I can't do this" and "I did this" is almost always just time and stubbornness.

What's Next for Samvit

The next step is the real world.

Everything built so far, the app, the firmware, the hardware validation suite, has been designed from the ground up to be deployable, not just demonstrable. The simulation was built to be as close to physical reality as datasheets allow. The firmware ready to be compiled and ready for a Raspberry Pi 4B the moment components are in hand.

What comes next is putting Samvit in the hands of the people it was built for, and finding out what the real world teaches us that no simulation can.

Built With

Share this project:

Updates