MemoryHome

Turn four room photos into an editable, measured 3D plan and practical furniture layouts—without requiring design expertise.

Comment

Inspiration

Most people can photograph a room, but turning those photos into a usable plan still requires spatial reasoning, measuring, and design experience. Existing interior tools often start with a blank CAD-like canvas or create a beautiful image that cannot be edited.

We wanted a simpler path: take four ordinary photos, confirm the measurements that matter, and get a room that remains understandable and changeable. The guiding principle became: AI proposes. Code verifies. You decide.

What it does

MemoryHome turns four directional room photos into an editable room plan. It identifies visible walls, doors, and windows, explains the evidence and limitations for every photo, and uses the user's wall width and room depth as the source of truth for geometry.

Users can:

  • review the reconstructed room in synchronized 3D and 2D views;
  • choose from checked room sets and a reviewed furniture catalog;
  • move, rotate, replace, recolor, add, or remove furniture;
  • see room, opening, obstacle, and furniture collision checks;
  • preserve existing furniture through recommendations and page refreshes;
  • choose wall and floor finishes; and
  • use the complete experience in English or Korean.

MemoryHome is designed for ordinary people planning a real room, not professional CAD users.

How we built it

The consumer experience uses Next.js, React, TypeScript, React Three Fiber, and Three.js. A shared Kotlin and Spring Boot service owns room-plan sessions, photo-analysis orchestration, geometry validation, and recommendation contracts. The validation frontend runs on Vercel at bangtaste.com, while deterministic 2D SVG and 3D projections share the same canonical room and furniture data.

Furniture comes from a pinned, locally served catalog of 51 reviewed models. Every item has a complete identity, model-derived dimensions, preview provenance, and room-compatibility rules. The browser never trusts a model-supplied URL or arbitrary asset name.

The AI boundary is deliberately narrow. The live model may suggest visible openings, allowlisted furniture identities, roles, and bounded relationships. Deterministic Kotlin code owns physical geometry, coordinates, room bounds, collisions, door and window clearance, persistence, and the final 2D/3D projection. Raw photo bytes are bounded, metadata-stripped, and not retained in response state.

How we used Codex and GPT-5.6

MemoryHome began with product-definition research in Codex. During Build Week, GPT-5.6 was meaningfully integrated into the engineering loop as the reasoning model inside Codex. We used it to turn that research into a tested consumer flow and to make the AI/code trust boundary explicit. The live end-user provider path uses a separate cost-controlled OpenAI API profile, so this submission does not confuse the builder model with the runtime model.

Codex helped us:

  1. research and falsify product directions before selecting MemoryHome;
  2. define contracts across the frontend, backend, catalog, and geometry system;
  3. build and extend the React/Three.js flow and active Kotlin/Spring path;
  4. inspect real screenshots and screen recordings;
  5. reproduce failures across preview, application, storage, and reload; and
  6. turn each failure into a shared invariant and regression test.

This was not a one-prompt generation exercise. Codex reproduced stale web/API contracts, unplaceable rug candidates, preview/model mismatches, weak furniture assets, and video frames that escaped their reviewed source windows. Each concrete failure became a reusable validator, test, or render gate.

Problem-definition Codex session ID: 019f5d6f-8ca2-7000-875c-8000589d5138

Primary implementation Codex session ID: 019f6eba-b742-7ec3-8f2c-ec2f53cb851c

Challenges we ran into

The hardest problem was not drawing a 3D room. It was keeping every representation honest and consistent. Four photos can contain repeated objects, adjacent walls, partial openings, blinds that resemble windows, and perspective distortion. We had to prevent uncertain visual evidence from silently becoming precise geometry.

We also had to maintain one furniture identity and footprint across recommendation cards, the 2D plan, the 3D scene, direct manipulation, browser storage, and refresh. A recommendation originally replaced accepted furniture even when the user asked to keep it. Screen-recording-based debugging revealed the broader invariant: when “keep this furniture” is selected, the accepted room must survive preview, application, persistence, and reload.

Accomplishments that we're proud of

  • A working validation deployment at bangtaste.com
  • Four-direction photo evidence with explicit limitations and rejection reasons
  • One canonical room rendered consistently in editable 2D and 3D
  • A reviewed catalog of 51 browser-optimized furniture assets and 12 relation-based room sets
  • Deterministic collision checks for room boundaries, doors, windows, obstacles, and furniture
  • Direct furniture editing with saved identity, placement, and finish
  • Complete English and Korean product experiences
  • Regression coverage for user journeys found through real screen recordings
  • A reproducible 170-second demo pipeline with local narration, caption QA, source fingerprints, reviewed scene windows, and immutable review files

What we learned

AI-generated spatial suggestions are most useful when bounded by explicit evidence and deterministic rules. A confident-looking image is not the same as a trustworthy plan. Showing uncertainty, preserving user intent, and making every error actionable improved the product more than hiding complexity behind a loading screen.

We also learned to treat each bug as a missing invariant. Fixing the shared boundary and adding a generalized regression test was more reliable than patching the single screenshot that exposed the problem.

What's next for MemoryHome

Next we want to complete the owner and target-user walkthroughs, add durable server-side projects and deletion operations, and make plans shareable. We also want to improve irregular-room support and classify photographed furniture into the reviewed catalog—or keep it explicitly unplaced until its dimensions and identity are known.

Existing and third-party work

The meaningful Build Week extension includes product selection and the trust contract, the consumer flow, catalog and direct editor, bounded recommendation system, active Kotlin/Spring path, production-shaped browser verification, regression gates, and the submission video pipeline described above.

The project uses standard open-source frameworks and reviewed models from the Amazon Berkeley Objects dataset. Asset provenance and license notes are tracked with the catalog, and commercial publication remains gated while an upstream license discrepancy is resolved.

Built With

Share this project:

Updates