holoaid

This guide outlines the development and vision for holoaid, a full-stack navigation system designed to revolutionize spatial awareness and routing through Mixed Reality (MR).## InspirationThe inspiration for holoaid came from the limitations of traditional 2D GPS interfaces. Whether for disaster relief responders in chaotic environments or visually impaired individuals navigating urban spaces, looking down at a flat screen is often dangerous and unintuitive. We wanted to create a "heads-up" world where navigation is woven directly into the user's field of view, making the digital path as real as the physical one.## What it doesholoaid is a full-stack spatial navigation system that projects 3D holographic waypoints and directional paths onto the real world.Real-time AR Waypoints: Dynamic "holograms" (arrows and path lines) appear on the ground to guide users.Contextual Information: Floating data panels provide distance, ETA, and environmental alerts.Smart Routing: A custom backend calculates the most efficient path, prioritizing safety and accessibility.Cloud Synchronization: Allows for remote monitoring where a central operator can "drop" waypoints into a user's field of view in real-time.## How we built itThe project was built using a robust full-stack architecture tailored for low-latency spatial data:Frontend (Mixed Reality): Developed using Unity and the MRTK (Mixed Reality Toolkit). We optimized it for devices like the HoloLens 2 and AR-capable smartphones.Backend: A Node.js/Express server handling API requests and user authentication.Mapping & Logic: Integrated Google Maps Platform (Directions API) for raw path data, which our system then translates into Cartesian coordinates for the AR space.Database: MongoDB stores user profiles, frequent routes, and custom spatial markers.Real-time Communication: Socket.io enables the live "operator-to-user" waypoint dropping feature.## Challenges we ran intoSpatial Drift: Keeping holograms "locked" to the ground over long distances was difficult. We addressed this by implementing Azure Spatial Anchors to ensure markers stayed persistent.Occlusion: It was a challenge to make holograms appear "behind" real-world objects (like walls or pillars). We utilized LiDAR and depth-sensing shaders to improve environmental mesh mapping.Battery & Performance: Rendering complex 3D paths in real-time is resource-heavy. We moved heavy pathfinding calculations to the backend to preserve the wearable device's battery life.## Accomplishments that we're proud ofZero-Latency Sync: Successfully achieving a sub-100ms delay between an operator placing a marker on a web dashboard and it appearing in the user's AR headset.Accessibility Design: Developing a "vocal-haptic" feedback loop that vibrates the device when a user veers off the holographic path, making it usable for those with low vision.Seamless Integration: Building a system that feels "native" to the environment rather than an overlay.## What we learnedWe learned that Human-Computer Interaction (HCI) in 3D is vastly different from 2D. We had to unlearn many UI habits buttons are less effective than gestures, and too much information in the field of view causes "cognitive overload." We also gained deep experience in Geospatial Coordinate Mapping, specifically converting Lat/Long GPS data into Unity's $X, Y, Z$ local space.## What's next for holoaidThe future of holoaid involves expanding beyond simple navigation:Indoor Mapping: Integrating BLE (Bluetooth Low Energy) beacons for precise navigation inside hospitals and malls where GPS fails.AI Hazard Detection: Using the device's camera to identify and highlight real-time obstacles (like construction or spills) and rerouting the user automatically.Multi-user Collaboration: Allowing multiple users to see each other's "digital breadcrumbs," creating a shared navigational space for search and rescue teams.

Built With

• typescript