Homeclear: film a home with your phone, and we simulate the walker's routes to find fall risks before discharge.
We are Team Splatter , say hi to our product Homeclear!
GIF
Splat of Century Tower at night
Splat of Newell hall 2nd floor. Lets you move objects around to clear the path
sampleRenderTimes_notaccurate
double_rende/workspace!
minimal renovations with second walker and widened steps. Built on splat on Newell Hall's stairs to improve accessibility on campus.
Larger renovations for wheelchair accessibility

Inspiration

A grandparent comes home from the hospital, and the first two weeks are the most dangerous part of their recovery. Falls happen at 8.0 per 1,000 person-days in the first two weeks after discharge, compared with 1.7 three months later.

The usual safeguard is a home visit from an occupational therapist (OT), which takes close to four hours of their time. When a visit can't be booked in time, the patient's discharge gets delayed or they go home unchecked.

Researchers already scan homes so an OT can review them remotely. We couldn't find a tool that simulates this patient, with their walker, on the routes they will actually walk. That is what we built.

What it does

A family member films the home with a phone. Homeclear turns the video into a measured 3D model of the room, accurate to within a few percent of a tape measure. Then it:

Finds the floor and maps the room: obstacles, floor bumps, and the walker's route (for example, kitchen entrance → sink → chair).
Simulates hundreds of trips along that route with a learned step model and walker physics, counting toe catches, walker snags and tight gaps.
Ranks the fixes (remove the mat, tape the cord, move a chair, add a nightlight) by how much risk each one removes and what it costs in effort and money.
Writes a report for the OT, with every result flagged for their review.

In the kitchen we scanned, removing one mat cuts expected toe catches from 1.14 to 0.07 a month.

You can explore the result three ways:

In the browser: drag furniture around and watch "ghost" walkers re-plan live, with a night view.
In VR on a Quest 2: the patient stands at their real walker with a controller strapped to it, and the virtual walker follows. They can walk their own home before they're back in it.
As a Cosmos Transfer clip: a realistic video that shows the family what that first morning will look like.

Turning a risk into a plan

Knowing the mat is a trip hazard isn't enough. A family also needs to know what to do about it, what it will cost, and whether it will help. So every change Homeclear suggests comes with all three.

How it works: pick a change in the room, like moving a chair, removing a rug or adding a grab bar. Homeclear reruns the simulated walks on the changed room and shows how many toe catches, snags and tight squeezes go away. The result comes from a new simulation, not a guess.

What a family sees for each change:

Cost: dollars for anything you buy, and "no purchase" for moving something you already own.
Effort: how long it takes, and whether it needs one helper or two.
Who might pay: self-pay, Medicare, Medicaid or VA. This tells the family what to ask about, not what is covered.
Hazards removed: the measured hazards close to that change that it clears.

Totals add up across all the changes, and each one can be undone, so a family sees the whole bill and the whole effort before moving anything.

Only fixes that matter on the real route. Everything is judged against the path the patient actually walks. A door fix on a route they never use is left out. If a route is blocked, Homeclear tries widening each door in turn to find the one that is really in the way.

Stairs and wheelchairs. The same approach covers stair fixes (a rail refit, a grab bar, step-edge strips, step lights, a second walker parked on the landing) and a wheelchair check against ADA numbers. A stairlift is offered as an option.

Preview first. Before anyone lifts a thing, the room slides into the new layout with outlines at the old spots. You can compare before and after, or switch to the night view.

What it can't tell you. Dollar figures are catalog estimates, not quotes, and a missing number reads "Pending", never zero. Anything we designed rather than scanned is tinted and labeled "imagined". If something can't be measured, such as turning room on a staircase, it is marked "OT to verify" instead of passing quietly.

How we built it

The rule behind everything: every risk number traces back to the scanned geometry, physics, or published data. It never comes from a generated image. Generated pictures are only used for show, and they are labeled "imagined".

The core is a 13-stage pipeline (stages 0–12 in our repo). Each stage reads the last stage's output and writes a record, so any result can be traced back to the video:

Stages	What happens	Where it runs
0–2	Set up the environment, extract frames, estimate camera positions (COLMAP / GLOMAP)	HiPerGator CPU and GPU
3–4	Train the splat, then fix its real-world scale from a tape-measured table	HiPerGator B200
5–6	Fit the floor, build the occupancy and floor-bump maps	HiPerGator B200
7–8	Plan the walker's route, find hazards	Laptop
9–10	Render each hazard, check the splat in the viewer	HiPerGator, laptop
11–12	Write the OT report, export the scene to the viewer	Laptop

Optional stages add the room shell (walls and ceiling), object detection, floor fills under moved objects, and a re-scan diff that compares a second video after the fixes to show which hazards are gone.

1. Reconstruction (UF HiPerGator, B200 GPUs through Slurm)

Phone video becomes frames, then SIFT features with exhaustive matching on CUDA pycolmap. Camera poses come from GLOMAP global mapping, followed by a bundle adjustment pass. Raw GLOMAP poses gave PSNR 14.6, while GLOMAP plus bundle adjustment matched standard COLMAP at 28.5.
After stray-camera cleanup, a 30,000-step gsplat run takes about 6–11 minutes on a B200. Our demo kitchen reaches 28.7 dB PSNR on held-out frames.
Real-world scale comes from one tape-measured table, checked with a leave-one-out test. The scan was off by under 1% overall.

2. Scene understanding

A RANSAC floor fit, an occupancy grid, and a floor-bump grid measured against a locally fitted floor. Walls and ceiling are fitted to the splat as a room shell.
Grounding DINO + SAM 2 masks are voted onto the Gaussians, so every chair, rug and box becomes its own movable object.
LaMa fills in the floor under each moved object, tinted "imagined, not measured".

3. The step model (trained by us)

Trained on PhysioNet Multi-Gait-Posture, which has 14 people walking with a walker.
A small MLP (two layers of 128 units, SiLU activations) predicts the next footstep as a mixture density network over the next heel position, plus Gaussian outputs for the walker's pose and the head position. An 8-number "style" embedding can be fitted to one person from 30 seconds of walking.
Trained with AdamW and a cosine learning-rate schedule, then tested by leaving two people out at a time. With the style fitted, head-position error drops from 9.8 cm to 5.2 cm, and held-out likelihood improves for all 14 people. It does not improve heel placement, and we say so in our docs.
Exported to ONNX and run in the browser, in the headset and in the Python planner. The browser and PyTorch outputs match to about 5e-6.
Toe-catch odds come from published minimum-toe-clearance data (lognormal, mean 14.1 mm, sd 8.3 mm). That is about 1 catch in 578 strides over a 1 cm bump. Catches are rare, which is why we need many simulated trips.

4. Physics

Rapier runs in a Web Worker for live walker and furniture contact.
A MuJoCo batch on HiPerGator (3 × 3,200 simulated pushes, with properties sampled from assumed ranges and the walker taken from its spec sheet) gives P(chair moves | touch) ≈ 0.85 and P(mat slides) ≈ 0.8.
The MuJoCo cloth-rug model was calibrated against our own filmed walker-over-rug takes, and the simulation agrees with the footage on where the rug gets disturbed.

5. Planner

A B200 search evaluated 52,510 layouts (about 10 million imagined trips, 1,053 seconds of wall time) with shared random seeds, so layouts are compared fairly.
A learned critic (an MLP with two layers of 64 units, R² = 0.999 on held-out layouts) predicts a layout's risk, so the browser can search again live without the GPU.
We tested whether walking style changes the plan across 15 fitted styles. It doesn't: the best option and the fix order were identical for all 15.

6. Viewer and VR

three.js + Spark render the splat, with WebXR for the Quest 2. We thinned the scene and trimmed splat sizes to hold 72 Hz from every viewpoint on the route.
A controller zip-tied to a real walker drives the virtual one, which pivots about its own center and refuses turns into furniture.

7. NVIDIA Cosmos

Cosmos Reason 2 proposes hazards from the video frames. We lift them to 3D and confirm them against the measured geometry before they count.
Cosmos Transfer 2.5 renders realistic "first morning" clips from our depth and segmentation controls.

Challenges we ran into

Measured vs imagined: every number had to trace back to geometry, physics or published data, never to a generated image.
Scale: we tape-checked the scan in the real kitchen and found it had lost the top of the fridge, which no metric on screen would have shown.
GLOMAP needs bundle adjustment: raw GLOMAP poses gave PSNR 14.6 until we added that pass. Camera placement is CPU-bound, so we split the GPU matching and CPU placement into separate jobs to stop HiPerGator cancelling idle GPU jobs.
A real walker in VR: a controller at an arbitrary angle has to drive a walker that pivots about its own center.
Splats on a Quest 2: we thinned the scene until the headset held 72 Hz.

Accomplishments that we're proud of

A real kitchen, filmed on a phone, measured to tape accuracy and simulated end to end.
The physics simulation agrees with our real footage on where the rug gets disturbed.
The step model and the planner run live in the browser and in a headset.
Every result in the repo links to its proof: job IDs, logs, numbers and videos.

What we learned

A tape measure beats a clever model. One measured table fixed the scale of the whole scan.
Label what is measured and what is imagined. Generative tools can show a family what a fix looks like, but risk numbers only earn trust when they trace back to geometry, physics or published data.
Rare events need many trips. A toe catch happens about once in hundreds of strides, so hundreds of simulated trips are what turn "this mat looks risky" into a number an OT can act on.
Hardware shapes the design. The 72 Hz Quest budget and a strapped-on controller taught us more than desk testing did.