Home Page / Room View
Dollhouse model of a real room
Photo-realistic room rendering
Vibe-based redesign
Vibe-based product recommendations

Forma

Carry your room in your pocket.

About Our Project & Significance

Forma is an AI-powered interior design and furniture shopping application that allows users to scan, visualize, and redesign their living spaces in 3D. Our goal is to make furniture shopping more intuitive, personalized, and accessible by combining spatial computing, generative AI, and autonomous shopping agents into one platform.

Furniture shopping can be surprisingly difficult, especially when trying to determine whether a piece will fit in a room, match existing furniture, or complement a particular aesthetic. While traditional online shopping platforms provide product images and dimensions, they leave much of the visualization work up to the customer. Even existing augmented reality furniture applications are often limited to specific products or they only allow users to visualize individual pieces of furniture, rather than considering how they fit into the overall layout and aesthetic of a room.

Forma fixes this problem by connecting the user's actual living space directly to the shopping experience. Using LiDAR technology, users can scan their rooms and generate true-to-scale, editable 3D models that preserve the original room's dimensions, layout, and existing furniture. Users can then scan furniture they encounter in physical stores or discover products online and visualize how those objects would look inside their own rooms before making a purchase.

We also wanted Forma to help users who have a general idea of their ideal room aesthetic in pictures but struggle to translate that inspiration into actual words. To address this, we developed Vibes, an AI-powered personalization feature that analyzes inspiration images to identify users' aesthetics. These preferences are then translated into actionable design profiles that can be saved and applied to different rooms.

From there, AI shopping agents search real retailers for furniture that matches the user's preferred aesthetic, spatial requirements, and budget. Users can view recommended products, place them inside their 3D rooms, and generate photorealistic previews of their redesigned spaces.

Rather than treating inspiration, furniture discovery, spatial visualization, and purchasing as separate processes, Forma brings them together into one cohesive shopping experience.

What Makes Us Unique

Although there are existing tools for interior design, 3D modeling, and furniture shopping, these experiences are often disconnected.

Traditional furniture retailers primarily focus on displaying their own products, while interior design applications may require users to manually construct their rooms or choose furniture from limited catalogs. Professional 3D modeling software can provide detailed visualization, but generally requires a much greater learning curve than the average furniture shopper would want to navigate.

Forma brings it all together. Our primary differentiators include:

Real-world spatial accuracy: Instead of relying on approximate room templates, Forma uses LiDAR scans to create room models based on actual measurements.
Physical-to-digital furniture shopping: Users can scan furniture they encounter in stores and visualize those objects in their own rooms.
Personalized aesthetic understanding: Users can provide inspiration images ranging from fully decorated bedrooms to artwork, posters, and nature photography. AI interprets the overall aesthetic rather than simply identifying individual objects.
Cross-retailer AI shopping: Rather than restricting users to one retailer's inventory, our shopping agents discover products from real online stores.
Budget-aware recommendations: Forma considers the combined prices of recommended furniture, helping users design within their intended spending limits.
Visualize before purchasing: Users can preview products in their actual room layout and generate realistic images of possible designs.

Process

1. 3D Room Scanning and Visualization

We developed Forma using Swift and SwiftUI, integrating Apple's RoomPlan framework to capture room dimensions, walls, doors, windows, and furniture through LiDAR scanning. Using RealityKit, we built a custom 3D room editor with an orbit camera and touch controls, allowing users to move, resize, replace, or hide furniture and customize wall and floor finishes.

To improve accuracy, we use images captured during scanning to match wall colors and Apple Vision to identify materials and suggest objects that RoomPlan may have missed. We also implemented spatial constraints to keep furniture within the scanned room's boundaries, ensuring that designs remain grounded in real-world measurements.

2. Real-World Object Scanning and 3D Generation

Using Apple's Object Capture, Forma allows users to scan physical furniture and create textured 3D models that can be placed into their saved rooms. For online products, we integrated fal's SAM 3D Objects to generate 3D models directly from product images.

Because AI-generated models do not inherently preserve real-world dimensions, we scale them using measurements from retailer listings. This allows users to visualize both in-store finds and online furniture at realistic sizes within their own rooms.

3. AI-Powered Vibe Extraction and Personalization

Our forma-Vibe feature allows users to upload up to 15 inspiration images. These images can range from decorated rooms to artwork, clothing, nature, or anything else that represents their personal style. Using Claude's multimodal image analysis, we identify recurring aesthetics and extract preferences such as colors, materials, textures, lighting, and furniture styles.

These preferences are organized into editable, reusable design profiles called Vibes, which users can save and apply to different rooms. Rather than requiring users to know specific interior design terminology, Forma allows them to communicate their aesthetic through visual inspiration.

4. Feng Shui-Based Furniture Arrangement

We also developed a feng shui-inspired automatic furniture arrangement system within our 3D room editor. The system uses six parallel search agents to explore and evaluate different furniture layouts based on spatial constraints and traditional feng shui-inspired placement principles.

Each agent scores potential arrangements using several criteria, including but not limited to:

Bed and sofa positioning: Beds and sofas should have their backs against walls while maintaining a view of the entrance.
Unobstructed pathways: Doorways and walking paths should remain clear.
Balanced furniture placement: Furniture should be balanced throughout the room without overlapping or extending beyond walls.
Natural lighting: Tall shelves should not block windows.

5. Autonomous AI Shopping Agents

Our Python FastAPI backend integrates Browser Use to power autonomous shopping agents that browse real retailers and discover furniture matching the user's aesthetic preferences and budget. We also use SerpAPI as a faster search fallback when appropriate.

Rather than simply recommending individual products, our backend evaluates combinations of furniture to keep the recommended merchandise total within the user's budget. Users can review product information, access original retailer listings, and place recommended items into their virtual rooms.

6. Photorealistic Room Rendering

We integrated Gemini image editing to generate photorealistic previews of redesigned rooms based on their actual layouts, finishes, and selected furniture. This allows users to visualize how their design choices might look beyond the basic 3D model.

To prevent AI-generated images from inventing walls, changing room dimensions, or misplacing furniture, we developed a geometry-grounding and bounding system that uses real scan measurements and reference images to constrain the rendering process. This helps preserve the accuracy of the original room while providing realistic visualizations.

7. Cloud Storage and Data Synchronization

We use Supabase, including PostgreSQL, Storage, anonymous authentication, and row-level security to manage users' saved rooms, scanned objects, and Vibes.

We also implemented a local-first synchronization system, allowing users to access and edit their rooms without a continuous internet connection. Changes are synchronized with the cloud when connectivity returns, making Forma especially useful for shopping in physical stores.

8. User Interface and Design

We designed Forma around a warm, earthy, minimalist aesthetic. Using Paper, we developed and refined our interface with a focus on visual hierarchy, readability, consistency, and intuitive interactions.

Our goal was to create a distinctive, cohesive design that makes advanced technologies easy to use, rather than overwhelming users with complicated control mechanisms.

Challenges We Ran Into

1. Limitations of RoomPlan

Although Apple's RoomPlan provides a strong foundation for room scanning, it only recognizes a limited number of furniture categories, leaving some objects missing from the generated models. To address this, we integrated manual tap-to-mark functionality and Apple Vision suggestions to help identify objects that RoomPlan might overlook.

2. Preventing AI Hallucinations in Room Rendering

Arguably, one of our biggest challenges was ensuring that AI-generated room previews accurately reflected the user's actual space. Early renders sometimes invented walls, altered room dimensions, or misplaced furniture.

To address this, we developed a geometry-grounding and bounding system that represents the room using bounding boxes and instructs the AI to preserve existing objects and their positions. We also provide reference photos of real furniture and product listings to ground the generated images in reality, helping produce more accurate and consistent room visualizations.

3. Maintaining Real-World Scale

AI-generated 3D models often lack accurate physical dimensions, making it difficult to determine whether furniture will actually fit. We solved this by scaling generated models using dimensions obtained from retailer listings, allowing products to be visualized at realistic sizes within scanned rooms.

4. Managing Shopping Agent Latency

Autonomous shopping agents can take several minutes to navigate retailers, inspect listings, and collect product information. To prevent these processes from slowing down the app, we implemented asynchronous backend jobs and integrated faster search alternatives when appropriate.

5. Reliable Cloud Synchronization

Implementing local-first synchronization introduced challenges involving conflicting edits, deleted records, and files changing during uploads. We developed a synchronization system to preserve users' saved rooms, objects, and Vibes while supporting offline editing and cloud backups, reducing the risk of data loss across sessions.

Future Expansions & Applications

Although Forma currently focuses on furniture shopping and interior design, we envision expanding its capabilities to make the entire shopping and design process more seamless.

1. Secure Agent-Assisted Checkout

We hope to implement secure, AI-assisted checkout powered by payment technologies from Visa and Stripe. Shopping agents would first gather final prices, including taxes and shipping, before presenting users with a consolidated order summary for approval.

Once confirmed, agents could complete purchases using single-use virtual cards issued through Stripe Issuing and supported by Visa's payment network, with merchant restrictions and spending limits to help prevent unauthorized or excessive charges.

Sensitive payment information would be handled securely without exposing card credentials to the AI, while users would retain full control over purchase authorization. By combining autonomous shopping with trusted payment infrastructure, we aim to make AI-powered purchases more seamless, transparent, and secure.

This is a proposed future integration rather than an existing feature or partnership.

2. Augmented Reality Visualization

We plan to integrate live AR furniture placement, allowing users to visualize selected products directly through their phone cameras in addition to Forma's existing 3D room models.

Users could walk around furniture virtually placed in their space, view it from different angles, and see how it interacts with their existing layout. This would make furniture shopping even more interactive, helping users better understand a product's size, placement, and overall appearance before purchasing.

3. Collaborative Room Design

We hope to introduce shared rooms and collaborative editing, allowing roommates, families, and friends to design spaces together.

Users could invite others to view their rooms, suggest furniture, experiment with different layouts, and share their saved Vibes. We also envision features like comments, voting on furniture choices, and comparing different room designs, making it easier for multiple people to contribute ideas and agree on a final layout.

4. Expanded AI Design Assistance

We plan to expand our AI assistant to support accessibility needs, personalized room functionality, and combinations of multiple saved Vibes.

For accessibility, users could request layouts that accommodate wheelchair movement, wider walking paths, or furniture that is easier to reach and navigate around. For room functionality, the AI could suggest arrangements based on how users actually use their spaces, such as maximizing storage in a small bedroom, creating a study area, or designing a multifunctional living room.

We also want users to be able to combine multiple saved Vibes, such as blending the natural textures of cottagecore with the clean simplicity of minimalism, and adjust how much each style influences the final design.

Ultimately, our goal is to make Forma more than just a furniture visualizer by helping users create spaces that reflect both their personal style and practical needs.

With Forma, you can forma room, forma vibe, and forma dream you can bring to life.