SentinelRT: Surveillance Rover

Inspiration

Modern defense and security operations require both reliable real-time systems and higher-level decision support. Inspired by the Thales challenge on persistent surveillance and strategic planning, we wanted to explore how QNX can power dependable embedded control while cloud-based AI assists with interpreting surveillance data and threats.

What it does

SentinelRT is a real-time surveillance rover built on QNX and Raspberry Pi. The rover streams live camera footage to a web interface and receives movement commands over TCP. Surveillance data is summarized and interpreted using the Gemini API to highlight potential threats and provide higher-level situational awareness, similar to decision-support systems used in defense and maritime operations.

How we built it

The system runs on a Raspberry Pi using the QNX Neutrino RTOS. Motor control, networking, and camera handling are separated into real-time tasks to ensure deterministic behavior. A TCP server receives commands from a web interface, while live video is streamed from an onboard camera.

To align with the Thales challenge, the rover represents a mobile surveillance asset that could support patrol, inspection, or monitoring scenarios. Gemini API is used to process and summarize surveillance observations, demonstrating how AI can assist operators and leadership with faster interpretation of field data.

Challenges we ran into

Balancing real-time motor control with networking and camera streaming was a major challenge, especially under tight time constraints. Mechanical prototyping required rapid iteration to securely mount motors, wheels, and the camera. Integrating AI-generated insights while keeping the real-time system deterministic also required careful separation of responsibilities between QNX tasks and cloud-based processing.

Accomplishments that we're proud of

We successfully built a working surveillance rover that integrates QNX real-time control, a live camera feed, and AI-assisted analysis. The project demonstrates how embedded systems, defense-oriented surveillance concepts, and modern AI tools can work together in a reliable and modular way.

What we learned

This project deepened our understanding of real-time operating systems, particularly how QNX enables predictable task scheduling and system isolation. We also learned how AI tools like the Gemini API can complement real-time systems by providing higher-level insights without compromising deterministic behavior.

What's next for SentinelRT: Surveillance Rover

Future work includes mapping multiple surveillance assets, simulating patrol coverage similar to maritime operations, adding autonomous navigation, and expanding AI analysis for risk assessment and decision-support dashboards inspired by real-world defense use cases.