Inspiration

What it does🚀 The Vision

RealityMerge transforms how maker spaces, engineering teams, and creators collaborate on physical products. Instead of huddling around a single screen or passing around static renders, teams can now step inside their CAD models together — in real-time, from anywhere in the world.

Imagine picking up a robot arm design, rotating it in your hands, scaling it to examine a gear mechanism, then shrinking it back down — all while your colleague in another country does the same, seeing your movements live. That's RealityMerge.

⚡ Core Features 🎯 Dynamic CAD Loading from Google Drive The Problem: Traditional CAD collaboration requires everyone to have expensive software licenses, deal with file versioning nightmares, and be in the same physical location.

Our Solution: RealityMerge pulls GLB/GLTF models directly from Google Drive shared folders — no CAD software required.

📁 Maker Space Drive (Shared) ├── 📂 Current Projects │ ├── 🤖 robot_arm_v3.glb ← Click to load in VR! │ ├── 🔧 custom_bracket.glb │ └── 🛠️ assembly_fixture.glb └── 📂 Archives How It Works:

Designer exports GLB from OnShape, Fusion360, or Blender Uploads to the shared Google Drive folder Instantly accessible in VR for the entire team No sync delays, no version conflicts, no software installs 🔄 CLI-Powered Sync Pipeline We built a powerful command-line interface that keeps everything synchronized:

Authenticate once

./sh/authenticate_gdrive.sh

Sync models from shared drive

./sync_from_drive.sh

Push updates back (if bidirectional)

./sync_to_drive.sh Why This Matters:

Continuous Integration for Hardware — Every time a design is updated, the VR experience reflects it immediately No Manual Downloads — The pipeline handles authentication, caching, and file management Works Headless — Can run on a server, triggering updates without human intervention OAuth2 Security — Proper Google authentication, not password storage 🤝 Multi-User Networking (NGO) Built on Unity Netcode for GameObjects, RealityMerge supports real-time multiplayer collaboration:

Feature Description Shared Spatial Anchors Everyone sees models in the same physical location Ownership Transfer Pick up a model another user was holding Transform Sync Position, rotation, and scale synchronized at 60Hz Client-Authoritative Whoever grabs it owns it — no server lag The Magic: When you grab a model, NetworkPhysicsInteractable instantly transfers ownership to you. Your movements are synced via ClientNetworkTransform . When you release it, ownership transfers back — seamlessly.

✋ Full VR Manipulation Every CAD model becomes fully interactive in your hands:

One-Handed Controls Grab & Move — Natural hand tracking, pinch to grab Rotate — Twist your wrist to spin the model Throw — Release with velocity for physics-based toss Two-Handed Controls Scale — Pinch with both hands, pull apart to enlarge Precision Rotate — Two-point rotation for exact angles Inspect Mode — Hold steady while scaling for detailed examination Powered by:

XRGrabInteractable with Velocity Tracking XRGeneralGrabTransformer for one/two-hand manipulation Continuous Speculative collision for smooth physics 📍 Smart Spawn System Models don't just appear randomly — they spawn exactly where you need them:

🎯 SpawnPoint (GameObject) ↓ 📦 CAD Model appears HERE ↓ 🦿 Bottom of model sits on surface Intelligent Positioning:

Automatically calculates model bounds Positions the bottom of the model at the spawn point (not the center) Horizontally centered, vertically correct Works with any GLB regardless of its internal origin point 🪄 Physics That Make Sense We carefully tuned the physics to feel natural:

Setting Value Why Movement Velocity Tracking Responsive but smooth Gravity OFF Models float where you leave them Interpolation Interpolate No jitter on network sync Collision Detection Continuous Speculative No tunneling through objects Table Collision IGNORED Models can clip through table (intentional) 🏗️ Architecture Deep Dive The Loading Pipeline 👆 User Clicks Button CadSelectorUI Spawn NetworkedCadModel Network Spawn Event CadContainer.SetUrl CadLoader.LoadGlb glTFast Parser GameObject Created 🤖 Model Visible! Key Components Script Responsibility CadSelectorUI UI button handling, prefab spawning CadContainer Network variable sync, URL management CadLoader GLB download, parsing, interaction setup NetworkPhysicsInteractable Ownership transfer, physics sync ClientNetworkTransform Client-authoritative transform sync 🔧 The Tech Stack Layer Technology Runtime Unity 6 (6000.3.0f1) Rendering Universal Render Pipeline (URP) VR SDK Meta XR SDK + XR Interaction Toolkit Networking Unity Netcode for GameObjects (NGO) Asset Loading glTFast 6.15.0 Cloud Storage Google Drive API Auth OAuth2 (CLI-based) Target Platform Meta Quest 3 🌟 What Makes This Special

1. Zero Friction CAD Sharing No plugins, no installs, no "let me send you the file". Just upload to Drive and it's available in VR.

2. True Spatial Collaboration Not just screen sharing — you and your teammate are both holding the same part, examining it from different angles, at the same moment.

3. Hardware-Agnostic Design Files GLB is the universal format. Export from ANY CAD tool:

OnShape → GLB Fusion360 → GLB SolidWorks → GLB (via plugin) Blender → GLB

1. Maker Space Ready Built specifically for collaborative fabrication environments where:

Multiple members need to review designs Physical proximity isn't always possible Quick iteration is essential Budget constraints limit software licenses

1. Developer Friendly CLI tools for automation Clean component architecture Well-documented code Artifact-based development with clear plans 🎬 Demo Flow Host starts the session on Quest 3 Client joins via lobby system Both users see the Virtual Table with UI buttons Host clicks "Load Robot" GLB downloads from Google Drive in ~2 seconds Robot appears hovering above the table Host grabs the robot, rotates it Client sees movement in real-time Client takes over the grab — ownership transfers Both users pinch with two hands to scale up Examine internal mechanisms in giant detail Scale back down, place on table Done — Design review complete without a single email attachment 📈 Future Roadmap Annotation System — Draw in 3D, attach notes to specific parts Version History — Swipe to see previous design iterations Measurement Tools — Ruler, caliper, section views in VR Export Back — Modifications saved to a new GLB Voice Chat — Integrated spatial audio for discussions AI Assistant — "Show me the gear assembly" voice commands 🏆 Built With Passion RealityMerge represents the future of collaborative product development — where geography disappears and ideas come to life in your hands.

No more "can you rotate it a bit more?" emails. No more "wait for the render to finish" delays. No more "we need everyone in the same room" constraints.

Just pick it up. Look at it. Build it together.

RealityMerge — Collaboration Without Boundaries 🚀

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for 1200-SF24 Reality Merge

Built With

• bash
• c#
• gltfast
• google-drive-api
• meta-quest-3
• ngo
• oauth2
• openxr
• rclone
• unity
• unity-relay
• urp
• xr-interaction-toolkit