Fit Check

Fit Check: scan your room once, text it a furniture listing and your goals, and know it fits spatially and financially before you carry it up three flights of stairs.

Comment 1

Inspiration

Living in New York City, space is always a constraint.

When buying furniture, especially from places like Facebook Marketplace, it is surprisingly difficult to know whether something will actually fit in your apartment. You usually have a listing photo, a few measurements, and your memory of the room — and often do not know whether the desk will block the door, whether the shelf will fit beside the window, or whether adding one more object will make the room feel unusable.

We wanted to build something that lets people answer those questions before they buy or physically rearrange anything.

That became our idea for the Live Better track: a persistent digital version of your room that you can experiment with as your space and needs change.

What it does

The project begins by scanning a real room using Apple's RoomPlan API and LiDAR.

From that scan, we capture the structure of the space, including:

  • Walls
  • Doors
  • Windows
  • Room shape
  • Real-world dimensions

We then recreate that space as an interactive 3D room using Three.js.

Instead of behaving like a traditional CAD or floor-plan tool, the room is designed to feel more like a cozy top-down game. Users can add furniture, move it around, rotate it, lock objects in place, and experiment with different layouts.

The scanned room becomes a persistent base space, while the user's actual furniture becomes their Current Room.

Bringing real furniture into the room

We wanted the product to fit into how people already shop.

Using Photon, users can send furniture links or images through iMessage. For example, someone browsing Facebook Marketplace could send a desk listing and ask:

"Will this fit beside my window without moving my bed?"

The system uses the saved room dimensions and furniture information to bring that object into the planning workflow and test it against the user's actual space.

This makes iMessage the bridge between furniture people discover in the real world and their digital room.

AI-assisted spatial planning

Inside the application, the Gemini API acts as a conversational spatial assistant.

Users can describe how they want their room to change using natural language, such as:

  • "I want a reading corner here."
  • "Make space for yoga."
  • "Keep my bed where it is."
  • "Place the desk near the window."
  • "Can I add another storage unit without blocking the door?"

Instead of replacing the user's room, the system can create new layout variants that the user can compare, adjust, and save.

The goal is not to have AI completely redesign the room for the user, but to help the user reason about what can realistically fit inside the space they already have.

How we built it

We used Swift with RoomPlan and ARKit to capture the physical room through LiDAR.

The scanned room data is passed into our application and reconstructed using Three.js and React Three Fiber.

Furniture assets were created in Blender, exported as .glb models, and stored using Supabase so they could be loaded dynamically into the 3D scene.

Our backend uses FastAPI to manage room data, furniture, layout variants, and communication between the different services.

We used:

  • RoomPlan + ARKit for room scanning
  • Swift for the iOS scanning experience
  • Three.js + React Three Fiber for the 3D editor
  • Blender for furniture assets
  • Supabase for storing assets and application data
  • FastAPI for backend orchestration
  • Photon for iMessage-based furniture input
  • Gemini API for conversational spatial assistance

Challenges we faced

The biggest technical challenge was integrating LiDAR and RoomPlan.

Capturing the room was only the first step. We then had to translate the scanned spatial information into geometry that could be reliably reconstructed inside the Three.js environment.

Another challenge was maintaining consistent real-world scale throughout the pipeline.

A room captured in meters, a furniture object created in Blender, and an object rendered in Three.js all need to agree on dimensions. Otherwise, something might appear to fit digitally even though it would not make sense physically.

We also had to balance visual quality with performance. Rather than trying to make every object photorealistic, we focused on recognizable furniture, accurate proportions, and a stylized but believable 3D environment.

Finally, connecting RoomPlan, Three.js, Photon, Gemini, Blender, Supabase, and the backend into one continuous workflow was a challenge of its own.

What we learned

One of the biggest things we learned was that spatial computing does not always need to mean a headset or a fully immersive XR experience.

Sometimes the most useful spatial applications are the ones that help people better understand the physical spaces they already live in.

We also learned how important it is to separate AI reasoning from spatial validation. AI can understand what someone means when they say:

"Make space for yoga without moving my bed."

But the geometry of the room still determines whether that request is physically possible.

That relationship between human intent, AI reasoning, and real spatial constraints became one of the most interesting parts of the project.

What's next

We want to continue expanding the project beyond a single-room planner.

Future directions include:

  • More accurate furniture reconstruction from product photos
  • Automatic extraction of dimensions from retail listings
  • Smarter layout optimization
  • Multi-room planning
  • Accessibility-aware layouts
  • AR previews of proposed furniture
  • More persistent personalization based on how the user actually uses their space

At its core, the project is built around one simple question:

Will this actually fit in my space?

Instead of guessing, measuring repeatedly, or moving furniture around physically, users can test the idea first.

Built With

+ 2 more
Share this project:

Updates

Submission history