Iris

Iris is a voice first campus guidance system built to help and support visually impaired students navigate university support services, from registering for courses to replacing a lost ID card.

Comment

Inspiration

About 0.2% of the student body at my university (The University of Lagos) live with a form of visual impairment, which is about <100 students out of a total of 50,000+ registered on campus at any given time, a population the system is not designed around.

Many of these students rely on sighted human guides to get through the day; classmates, friends or sometimes hired assistants who walk with them between lecture halls, the library, the bursary, health centre and administrative offices. The problem is that these guides, while well intentioned, usually do not know how to access services specific to visually impaired students.

"Where do I get a permission slip for my exam scribe"

"Where do I go to replace my ID card if I can't see"

"What hostel has support staff for me"

Timothy and I built Iris to close that gap, not by replacing human guides, but by giving visually impaired students, a voice-first AI powered companion that knows the campus, understands their question and speaks the answers they need, on demand.

What it does

Iris does 4 things for visually impaired students:

  • Answer campus questions in plain language: Ask "Where do I register for the computer based exams" or "How do I get a transcript" and Iris interprets question, searches a curated knowledge bank of campus services, and speaks back a clear actionable answer, not just a building name, but a map route of the campus and description of how you can get there from where you are.

  • Tells where you are : Say "Where am I?" and Iris fuses 3 data sources in parallel; your GPS location, a camera snapshot interpreted by AI vision and reverse geocoding, into a simple spoken scene description. No guessing, no fabricated landmarks, if the camera can't see clearly it falls back to verified graph data instead of hallucinating.

  • Plans accessible routes : Iris maps campus as a weighted graph where doors, stairs, construction zones and inaccessible routes all add cost penalties. Community reported hazards feed directly into routing, so the path it recommends adapts to real conditions, not just what the map looks like on paper.

  • Detects obstacles in real-time : Point your phones camera forward and Iris uses Object detection to warn you as you approach people, furniture, and barriers, alerting users through audio cues and haptic vibration.

How we built it

During technical ideation, we started with the user and their real world constraints and worked backwards from there.

Browser-first, not app-first : Iris is a progressive Web App built with Flutter Web. No App store review, no download friction, a student opens up the link on and device with a browser and it works immediately. This was a non-negotiable, the people who need this tool the most are the least likely to navigate an app store to find it.

OpenStreetMap, not Google Maps: We needed a mapping solution that was free, offline-capable, and customizable. OpenStreetMap tiles gave us all three. We model campus grounds as a bidirectional graph of weighted waypoints and edges. Dijkstra computes the shortest accessible route without any API call.

TensorFlow.js SSD for obstacle Detection: Object detection runs entirely in the browser, using a pre trained model. No API calls, no cloud dependency, no latency and no cost.

Local-first with graceful degradation : Every AI feature has a fallback. Knowledge questions go to AI first, then fallback to keyword search against a local knowledge bank.

Challenges we ran into

Oh boy, do we have quite a few of these....

  • OUR GEMINI API KEY GOT BANNED: We had original built Iris with Gemini 3.1 Pro + grounding with Google maps, but in the last lap of development, our key got banned, so we quickly switched to Kimi K 2.6, another multimodal model.

  • Manual Map annotation : We had to go around campus and manually annotate problematic routes with obstacles.

-Scene Merging : fusing three parallel sources with their own individual overhead / latency(GPS, vision, geocoding) with independent timeouts and graceful degradation when one or more fail

-Speech to text in Firefox : Iris depends on the Chrome native speech to text api, unfortunately there is no equivalent of this in Firefox, we resorted to using a speech to text api as a fallback.

Accomplishments that we're proud of

Getting native browser based text to speech and speech-to-text to work : This was extremely important to us to as commercial api's for both services are quite expensive, we were so happy we got it to work

Weighted Djikstra for map routing: Implementing a weighted shortest possible part took us quite a while and we're extremely proud we get it to work within the provisioned time.

What we learned

Our major learning while building Iris is that our school has a lot of support services for visually impaired students, which we did not know of before building, if anything that exasperates the need for Iris.

What's next for Iris

  • Multilingual Support : Many UNILAG students think and speak in Pidgin, Yoruba, Igbo or Hausa before English. Iris should understand Where them dey collect transcripts for this school just as well as "Where is the transcript office"

  • More Campuses, more students : Iris currently models the University of Lagos. However, the architecture is campus agnostic, the graph, knowledge bank and hazard system are all data / community driven. A next step would be to onboard more campuses and students.

Built With

Share this project:

Updates