Inspiration

A disrupted schedule can create a chain reaction. One meeting running late can push back subsequent appointments, consume focus time, and put important deadlines at risk.

Traditional calendar applications can display these conflicts, but users still have to figure out how to rearrange their entire day.

I wanted to explore a different approach: what if users could physically interact with their schedules in a virtual environment while an intelligent assistant helped them understand and resolve disruptions?

This led to PlanRescue Spatial, an immersive productivity experience designed around natural hand interactions and intelligent schedule recovery.

What it does

PlanRescue Spatial transforms schedule management into an interactive spatial experience designed for Meta Quest and WebXR.

Users enter a virtual productivity studio where they can explore their schedules, inspect tasks, report disruptions, and examine alternative recovery plans.

Its core capabilities include:

  • Spatial Calendar: Visualize daily schedules as interactive spatial timelines.
  • Natural Interactions: Use hand-based pointing, pinching, grabbing, and selection to interact with spatial elements.
  • Disruption Management: Report meeting overruns and understand their effects on subsequent commitments.
  • Conflict Analysis: Identify affected events, scheduling conflicts, and potential deadline risks.
  • AI-Assisted Recovery: Explore alternative scheduling strategies generated through PlanRescue's deterministic recovery engine.
  • Spatial Comparison: Compare original and proposed schedules while inspecting individual changes.
  • Independent Validation: Evaluate recovery proposals against scheduling constraints.
  • Safe Execution: Explicitly approve a recovery proposal and separately confirm its execution.
  • Audit History: Review previous recovery decisions and completed actions.

The central principle is that AI assists with interpretation and explanations, deterministic logic handles scheduling decisions, independent validation checks proposed changes, and users retain control over execution.

How we built it

PlanRescue Spatial extends an existing AI-powered scheduling and recovery system with a dedicated immersive application.

Technology Stack

Spatial Application

  • Meta Immersive Web SDK (IWSDK)
  • Three.js
  • TypeScript
  • Vite
  • WebXR
  • Hand-tracking and spatial interaction systems

Backend

  • Node.js
  • TypeScript
  • Next.js
  • Model Context Protocol (MCP)
  • SQLite

AI and Intelligence

  • Existing OpenAI and Amazon Bedrock provider architecture
  • Deterministic scheduling and recovery algorithms
  • Independent constraint validation
  • Explainable recovery recommendations

Architecture

The VR application communicates with the existing PlanRescue backend through server-side API routes.

The backend remains responsible for retrieving schedule data, analyzing conflicts, generating recovery proposals, validating candidates, managing approvals, and executing authorized changes.

This separation allows the immersive interface to focus on spatial interaction without duplicating the underlying scheduling intelligence.

Challenges we ran into

1. Translating 2D productivity interfaces into spatial experiences

Traditional dashboards are designed for flat screens. Designing readable, comfortable, and interactive panels for a three-dimensional environment required reconsidering layout, depth, viewing distance, and spatial organization.

2. Natural hand interactions

Implementing intuitive pointing, pinching, selection, and grabbing required careful handling of XR input, interaction states, tracking interruptions, and object movement.

3. Preserving scheduling correctness

A visually appealing schedule is not enough. Event durations, priorities, protected commitments, and task deadlines must remain consistent with the underlying scheduling engine.

4. Safe schedule execution

A recovery proposal must never automatically modify a user's schedule. Separating proposal generation, validation, approval, and execution required careful integration with the existing backend lifecycle.

5. Integrating multiple systems

Connecting an independent VR application with an existing Next.js application, MCP server, deterministic recovery engine, and persistent database required maintaining consistent API contracts and clear application boundaries.

Accomplishments that we're proud of

  • Built a dedicated spatial productivity application using Meta IWSDK and Three.js.
  • Developed a three-zone virtual productivity studio.
  • Created reusable spatial interface components.
  • Implemented hand-interaction and desktop fallback systems.
  • Translated calendar schedules into interactive spatial timelines.
  • Connected disruption analysis and recovery proposals to the existing scheduling architecture.
  • Developed spatial Before-and-After recovery comparison.
  • Integrated independent validation and explainable recovery information.
  • Preserved the explicit approval and separate execution-confirmation workflow.
  • Hosted the WebXR application over HTTPS through GitHub Pages.

Most importantly, PlanRescue Spatial explores how spatial computing can make complex productivity decisions more tangible and understandable.

What we learned

Building PlanRescue Spatial taught us that successful VR productivity applications require much more than placing conventional interfaces inside a headset.

Spatial interaction must feel natural, information must remain readable, and the environment must be designed around human comfort.

We also learned the importance of separating AI-generated explanations from deterministic business logic, especially when applications can modify important user data.

Finally, we learned that a reliable immersive experience depends equally on interaction design, backend architecture, data integrity, and performance.

What's next for PlanRescue Spatial

PlanRescue Spatial is a foundation for a broader spatial productivity platform.

Future development will focus on:

  • Further validating and optimizing the experience on physical Meta Quest hardware.
  • Improving spatial panel positioning, readability, and interaction ergonomics.
  • Expanding accessibility and alternative interaction methods.
  • Integrating real external calendars, including Google Calendar and Microsoft Outlook.
  • Introducing more intelligent productivity insights and disruption scenarios.
  • Improving personalization and multi-user collaboration.
  • Strengthening synchronization, security, monitoring, and production deployment.
  • Exploring mixed reality experiences that bring productivity interfaces into users' physical workspaces.

Our long-term vision is simple: instead of manually reorganizing your day whenever something goes wrong, you should be able to interact with your schedule naturally, understand the consequences, and recover your day with confidence.

Built With

Share this project:

Updates

Submission history