Hound Around

Inspiration

A BU student told us: "I had an event in the GSU. I couldn't find the room, so I didn't go." An MIT upperclassman didn't know where a lab was and spent 20 minutes finding it.

GPS stops at the front door. BU publishes no indoor maps, and its accessibility map ends at building entrances, so there are no published accessible indoor routes. It's hardest for first-years, visitors, and anyone who needs a route without stairs. Google built indoor AR navigation for airports and malls; nobody built it for students.

What it does

Open hound-around.tech on any phone: no install, no account, no location permission.

Snap: pick where you're going, point your phone down the hallway, and take a photo.
Locate: Hound Around names your spot and floor in under a second.
Confirm: when it isn't sure, it says so and shows its top 3 guesses or asks for a second photo. Tap the right one and it learns on the spot: the next photo there comes back right.
Route: it guides you one step at a time ("Keep straight", "Turn left toward the restrooms", "Take the stairs to Floor 1") without ever showing a map of the building. Elevators only swaps every stair for the elevator.

No good photo? Type the room number beside you instead.

Design: built for someone standing in a hallway with a class about to start. The four numbered steps stay on screen, red is used only for "you are here" and your path, directions come one card at a time with a full-screen card for floor changes, and unsure matches show plain-language places ("Hallway outside Room 164") instead of IDs. It supports light and dark themes and reduced motion.

How we built it

Data: no dataset of BU's interiors existed, so we filmed three floors of the Center for Computing & Data Sciences ourselves: 67 videos, about 7,000 face-blurred frames, 57 spots. We said each spot's name out loud while walking, and Whisper transcripts turned those callouts into labels. The building is a walkable graph traced from the floor plans in a public 2019 city planning filing, then corrected on foot.
Recognition: each photo becomes a MegaLoc place-recognition embedding, and the 10 nearest of 7,008 indexed frames vote for a spot (k-nearest-neighbor classification). A logistic regression on five match features predicts whether the answer is right and decides whether to answer, show the top 3, or ask for a second photo. EasyOCR reads room numbers when floors disagree.
Evaluation: leave-one-video-out cross-validation on 1,758 photos, so no frame from a test video is ever in the index. We compared a color histogram, CLIP, DINOv2 and MegaLoc.
Routing: networkx shortest path over the building graph; accessible mode drops stair edges.
Stack: React + Vite + Tailwind on Vercel; FastAPI in Docker on Hugging Face Spaces; PyTorch, MegaLoc, EasyOCR; private data in a private Hugging Face dataset.
Kiro: we worked spec-first (requirements → design → tasks), with steering docs that turn our privacy rules into a per-change checklist, hooks (tests on save, an API-contract guard, a privacy guard on stop), and property tests that check, on 200 random graphs per run, that accessible routes never take the stairs. KIRO.md in the repo details what Kiro built.

Results

Measured on our own held-out videos (n = 1,758 photos), first version (DINOv2) → shipped (MegaLoc):

Exact spot: 28.6% → 48.2%
Correct spot in top 3: 52.7% → 74.9%
Right floor: 74.2% → 89.4%

When it answers on its own (63% of photos), it is within 10 m on the right floor 91% of the time. With the top-3 picker and second photo, 84% of simulated users end up at the right spot. A live match takes about 0.6–0.7 s on a CPU server.

Challenges we ran into

Look-alike hallways: the F2 east corridor, and B1 vs. F2 near the elevators, fooled our first model. Room-sign OCR, vote pooling across nearby spots, and the top-3 picker are the fix.
More data isn't always better: more frames from the same video stopped helping past about 40 per spot; new filming sessions and new angles helped far more.
Honest evaluation: random train/test splits leak near-identical frames, so we hold out whole videos.

Accomplishments that we're proud of

A working, deployed app that a judge can use on their own phone.
Right floor 74% → 89% in one weekend, with a confidence model that asks instead of guessing.
A live correction moment: tap the right spot and the next photo is right.
Privacy from the first frame: faces blurred on the laptop, photos deleted after matching, no public map or dataset.

What we learned

Place-recognition models beat general-purpose image features by a wide margin, calibrated uncertainty matters more than raw top-1 accuracy for a navigation app, and the way you split your data decides whether your numbers mean anything.

Ethics and responsibility

Responsible data sourcing and use. We filmed only public hallways of the floors open to the event, excluding service corridors and security rooms. Floor plans come from a public 2019 BPDA filing, credited in our README, used only to build the routing graph and never displayed. Frames, plans, graph and index stay private: a complete photo map of a building is a security risk, so the public repo holds code only.

Privacy, bias and fairness. Faces are blurred before any frame leaves the laptop. User photos are resized and stripped of metadata on the phone, then deleted after matching. No GPS, Wi-Fi positioning, accounts or location history. Accuracy differs by floor and lighting, and we filmed at chest height, so wheelchair-height photos are still untested. Elevators-only routing serves the people with the most to lose first.

Potential harms and mitigation. A wrong answer could send someone the wrong way, so low-confidence matches ask instead of guessing. Corrections could be abused, so they are rate-limited, and at scale would require BU login and three agreeing users. Users see only their next step, never a full map.

Transparency and accountability. The app says when it is unsure and that photos are deleted. Our README documents data sources, methods, the evaluation protocol, measured accuracy and limitations, and labels internal numbers as internal.

What's next for Hound Around

Not just CDS: each building is its own data package, not new code. Film it, label it, blur it, index it, from the GSU to a lab at another college. Next, we'll run the matcher on the phone so it works with no signal, test with seated and first-time users, and partner with Disability & Access Services, Admissions, Orientation and IS&T for BU's first accessible indoor routes.

Team

David Ribeiro (ML and data) · Jun Olea (backend) · Avery Lee (frontend and Kiro) · Tobi Olawale (data and workflow) · Bryan Martinez (pitch, design and testing). All five of us worked on the pitch, design, verification and regression testing through GitHub.