Inspiration

In Peru, most homes are self-built. Families, foremen (maestros de obra) and small contractors build floor by floor, often over years, without a schedule, a material takeoff or a budget they can follow. Plans exist on paper or as a phone photo, but nobody turns them into "what do we build this week, what do we buy, and how much will it cost?" We wanted to give these families the kind of 4D/5D planning that big construction companies use, starting from the only thing they already have: a photo of their floor plan.

What it does

ObraKadabra turns a phone photo or PDF of a 2D floor plan into a 3D home linked to a construction schedule (4D) and a cost plan (5D).

  1. Upload a photo of the plan and mark one known length to set the scale. Blurry photos get a warning first.
  2. AI reads the plan: walls, doors, windows, columns and rooms, each with a confidence score.
  3. Review in a 2D editor: uncertain elements are flagged and can be fixed with snapping and undo.
  4. See it in 3D: a procedural model of the home.
  5. 4D + 5D: a week-by-week schedule that follows how confined-masonry homes are really built in Peru (foundation → walls → columns → slab → roof → finishes), a budget in soles with an S-curve, and the shopping list for each week, which can be sent to the hardware store on WhatsApp or downloaded as a PDF.
  6. Log real progress: upload a site photo, the AI compares it with what was planned for that week, the foreman approves, and the 3D model shows what was really built.
  7. Share: a read-only link lets the family follow progress, money invested and the expected move-in date from their phone.

Home

2D editor

4D/5D workspace

Progress from a site photo

How we built it

  • Next.js 16, React 19, TypeScript and Tailwind CSS 4 for the app; React Three Fiber, drei and three.js for the 3D viewer.
  • One shared contract, FloorPlan: the AI produces it, the editor edits it, and the 3D viewer, schedule and budget all consume it.
  • AI behind one door: Gemini vision with structured JSON output reads plans and site photos. Every answer is validated with Zod and retried once with the validation errors as feedback. The model can be swapped (Gemini, OpenAI, or offline fixtures for demos and tests).
  • Pure domain logic (geometry, takeoff, schedule, earned value) written as tested functions with Vitest.
  • Supabase (Postgres with row level security, anonymous auth, Storage) for persistence, with a localStorage/IndexedDB fallback so it works without an account.
  • Playwright + axe-core end-to-end tests on desktop and mobile, plus Lighthouse checks.
  • Screens designed in Google Stitch, and the app pair-programmed with Claude Code.

Challenges we ran into

  • Reading real plans: phone photos come skewed, blurry and full of dimension lines. We added cropping, rotation, contrast, scale calibration and a blur check, cleaned the AI geometry (closing corners, merging walls), and only ask the user to confirm the uncertain elements.
  • A schedule that matches how Peru builds: confined masonry raises the walls first and pours the columns against them, floor by floor, so a generic Gantt chart wasn't enough.
  • Honest progress: the AI proposes and the foreman approves. Only approved logs count, so the family never sees "on track" until a photo confirms it.
  • Performance on mid-range phones: the 3D viewer, PDF library and database SDK load only when needed.

Accomplishments that we're proud of

  • A full pipeline, from photo → AI → 2D editor → 3D → schedule → budget → progress → family link, working end to end in one week.
  • A schedule and budget that reflect real Peruvian construction practice and prices in soles, not a generic template.
  • AI that recovers from its own mistakes thanks to schema validation and a feedback retry.
  • An accessible product (WCAG 2.2 AA contrast, keyboard support, large touch targets) tested on both desktop and mobile.
  • Two interactive demos that work with no API key and no sign-in.

Family view

What we learned

  • Validating AI output against the same schema the app uses makes errors recoverable: one retry with the validation feedback fixes most bad answers.
  • Running one contract test suite against two storage backends caught real bugs we would have missed.
  • Exposing 3D state as HTML attributes made a WebGL app testable end to end.
  • Designing for real users (families and foremen, not engineers) means simpler words, fewer steps and keeping a human in the loop.

What's next for ObraKadabra

  • Test the AI against a larger set of real hand-drawn and printed plans, and tune it per plan style.
  • Add stairs, beams and roof slopes to the takeoff, plus regional price lists.
  • Offline-first progress logging for construction sites without signal.
  • An account page to manage share links and invite the foreman.
  • Deploy publicly and pilot it with families and foremen in Lima.

Built With

Share this project:

Updates

Submission history