The Underlayer

Inspiration

Cybersecurity is one of the hardest subjects to learn because almost all of it is invisible. You cannot see a packet, an open port, an outdated firmware version, or a vulnerability. Students and new practitioners learn it abstractly, from textbooks and scrolling terminal output, with nothing to ground it in the physical world. Meanwhile, the stakes could not be more concrete: throughout the history of the internet, attackers have compromised entire organizations through a single overlooked device. The whole discipline hinges on being able to understand something you fundamentally cannot see. We wanted to change that. We asked what it would look like if the hidden layer of the connected world, the devices, their software, their weaknesses, were made visible and spatial, so that a person could walk through a real room and learn to see and understand threats the way an expert does. That is what The Underlayer is: a tool for seeing the underlying factor behind each device's security, with an AI tutor that explains what it means.

What it does

The Underlayer is a spatial learning experience for cybersecurity, built for Snap Spectacles. A chest-mounted ESP32 scanner detects nearby Wi-Fi and Bluetooth devices and streams what it finds to a backend relay. The backend identifies devices, checks operating system and package versions against vulnerability data, and generates AI-powered explanations and remediation guidance.

Challenge prompt

We’re unleashing the future of learning with Spatial AI at Reality Hack at AWE! Join Reality Hack, AWE, and Spectacles to build the future of learning with Spatial AI and Extended Reality. Teams will create experiences that levels up how people learn by blending intelligent digital assistance with the physical world in an education context. Using XR hardware and AI services, design experiences that help humans see more, understand more, and learn better in real-world environments. From real-time individual guidance and spatial learning to multi-user education experiences, the sky’s the limit on how you imagine learning and educational experiences that use AI to understand space, objects, and context to transform spaces for learning, whether that’s a museum or a classroom. Join us to unleash experiences where AI amplifies human learning, turning the world around us into contextual, intelligent learning experiences.

How we built it

The project consists of three primary layers:

Hardware Layer ESP32-based Wi-Fi and Bluetooth scanner Custom wearable enclosure Real-time device discovery SSH-based device interrogation and remediation execution Backend & Intelligence Layer Python FastAPI backend MongoDB database WebSocket-based real-time communication Device enrichment and vulnerability analysis pipeline AI-assisted risk assessment and remediation recommendation engine AR Layer Snap Spectacles Lens Studio Spatial device visualization and anchoring Interactive device cards and approval workflows Real-time rendering of device status, vulnerabilities, and AI recommendations Challenges we ran into

One of our biggest challenges was integrating multiple real-time systems across hardware, backend services, and AR rendering. We needed reliable communication between the ESP32 scanner, FastAPI backend, MongoDB, WebSocket services, and the Spectacles experience while maintaining low latency.

Another challenge was transforming raw device discovery data into meaningful cybersecurity insights. Mapping discovered devices, correlating software versions, and presenting that information effectively inside an AR interface required multiple iterations.

We also experienced significant team changes during the hackathon. We originally started as a four-person team, but two members became inactive and ultimately left the project. The organizer, Brian, reviewed and approved the team changes and confirmed that our remaining team would still be eligible despite the published team size requirements. This required the remaining contributors to absorb additional responsibilities and complete portions of the project outside of their original assignments.

Accomplishments that we're proud of

Built a complete end-to-end pipeline from physical device discovery to AR visualization. Successfully integrated hardware scanning, backend intelligence, WebSockets, MongoDB, and Snap Spectacles. Created a real-time cybersecurity experience that allows users to visualize nearby devices and their security posture in physical space. Implemented an AI-assisted workflow that helps users understand risks and remediation options. Successfully delivered the project despite significant team attrition during development. What we learned

This project taught us how challenging it can be to connect hardware systems, backend infrastructure, AI services, and AR interfaces into a single real-time application.

From a technical perspective, we gained hands-on experience with ESP32 device scanning, MongoDB, FastAPI, WebSockets, Lens Studio, and real-time AR data visualization. We also learned the importance of designing clear communication protocols between hardware and software components early in the development process.

Beyond the technical work, we learned how important adaptability and ownership are in a hackathon environment. When team composition changed unexpectedly, remaining contributors had to take on additional responsibilities, reprioritize tasks, and focus on delivering a functional end-to-end experience within a limited timeframe.

Contribution

Sami Maghnaoui Project management and overall technical coordination ESP32 hardware development and device scanning C++ bridge connecting hardware telemetry to the backend pipeline SSH interrogation and remediation execution framework Hardware enclosure design and integration Additional frontend and Lens Studio development after team restructuring

Han Lee MongoDB database design and integration Python FastAPI backend development Real-time WebSocket infrastructure Backend-to-Lens Studio communication pipeline Device data processing and relay services

Team Changes

The project was originally formed as a four-person team. During development, two team members became inactive and voluntarily withdrew from the project. The organizer, Brian, approved the team changes and confirmed that the remaining contributors would not be disqualified despite the originally stated team-size requirements. Following the restructuring, the remaining work was completed by Sami and Han, who assumed responsibility for the unfinished tasks and final project integration.

What's next for The Underlayer

Future development will focus on expanding device support, improving vulnerability detection, and enabling automated remediation across a wider range of operating systems and network devices.

We would also like to replace static device mappings with dynamic localization, allowing devices to be positioned automatically in physical space. Additional plans include deeper AI reasoning capabilities, integration with enterprise security platforms, historical asset tracking, and support for larger environments such as data centers, manufacturing facilities, and corporate campuses.

Built With

• c++
• digitalocean
• esp32
• fastapi
• gemini
• lens
• mongodb
• openai
• python
• snap
• websocket