RubbleEcho by McNeeseNexus for Nebius X NVIDIA

A search-and-rescue research rover using NVIDIA Nemotron on Nebius to turn sensor evidence into inspection priorities for human review.

Comment

Inspiration

RubbleEcho began with concern about people trapped after earthquakes and floods, including disasters in Nepal and Türkiye. The difficulty of identifying where to search led us to investigate a specific problem: how could a small rover gather observations and help an operator decide which location deserves closer inspection?

We built a controlled research testbed to explore that question. Our goal is to connect physical measurements with an understandable assessment that a person can review.

What it does

RubbleEcho combines a wheeled rover, a measured nine-cell sand table, and a browser-based operator dashboard.

An operator can drive the rover from a phone or initiate a fixed grid sweep. The system organizes available sensor observations by cell. After the sweep, NVIDIA Nemotron models accessed through Nebius Token Factory review the recorded evidence and recommend a location for further inspection. The dashboard presents the assessment alongside the observations and a search-priority map.

The prototype studies inspection prioritization. It has not been validated for locating buried people, detecting vital signs through rubble, or confirming that an area is clear.

How we built it

We divided the system into three connected parts:

  • Rover and sensing: A Raspberry Pi gathers ultrasonic range readings, 60 GHz radar observations, acoustic recordings, and camera frames when available. An Arduino Uno controls the wheels through an L298N motor driver.
  • Operator console: A React and TypeScript frontend communicates with a Python FastAPI backend. The backend stores cell observations, processes audio, manages command queues, and supplies dashboard updates.
  • AI assessment: The backend calls NVIDIA models through Nebius Token Factory’s OpenAI-compatible API. The configured text model, NVIDIA Nemotron 3 Nano, reviews cell evidence. Nemotron 3.5 Super VL supports descriptions of submitted camera frames.

For acoustic evaluation, the software processes recordings captured during array testing. A local threshold classifier uses signal features such as peak amplitude, RMS, and the proportion of loud samples to distinguish knock-like, scratch/scrape-like, quiet, and ambient windows. These labels are uncalibrated and do not establish where a sound originated. Nemotron reviews the reported classification; it does not perform that classification itself.

We kept AI recommendations separate from motor commands. Model responses are checked against allowed recommendation types, known evidence IDs, and known grid points. The validator rejects fields such as motor speed, PWM, voltage, and GPIO settings. Recommendations require operator approval.

Challenges we faced

Connecting hardware and software. Wiring, motor direction, power delivery, serial-port ownership, and network configuration all affected integration. We had to establish clear responsibilities: the Raspberry Pi communicates with the Arduino, while the laptop manages the dashboard, evidence, and model requests.

Obtaining useful acoustic evidence. Quiet signals, motor noise, and microphone connection damage complicated testing. We learned to distinguish captured test recordings, missing audio, and genuinely quiet recordings. Missing input must remain unavailable rather than become a negative detection.

Making the dashboard understandable. Raw sensor values and multiple controls competed for attention. We focused the interface on the rover’s state, recorded observations, the latest assessment, and the operator’s next action.

Keeping conclusions within the evidence. A radar response, an acoustic event, or a model-generated explanation cannot establish that a person is present. We built explicit limitations into the processing and assessment workflow.

What we learned

Physical AI requires careful coordination between measurement, interpretation, and action.

A sensor reading needs context: where it was collected, which device produced it, and what conditions could affect it. An AI assessment needs traceable inputs and clearly stated uncertainty. Movement needs a control path that remains separate from the model’s interpretation.

We also learned that a controlled sand-table result is only an early research step. Claims about disaster conditions require substantially broader testing, calibrated sensors, and evaluation with search-and-rescue professionals.

Accomplishments we are proud of

We brought together mechanical, civil, computer, and software work into a physical research prototype. The project includes a rover, a constructed testbed, sensor collection, an operator console, and a structured NVIDIA model-integration path through Nebius Token Factory.

Our MIT-licensed repository documents the architecture, setup, hardware connections, and current limitations so others can inspect and extend the work.

What’s next

Our next priorities are to:

  • Measure repeatability, classification errors, and false positives using labeled trials.
  • Improve microphone reliability and evaluate multichannel processing.
  • Make camera capture consistent across observation locations.
  • Investigate GPR integration and optimization, planned LDR installation, and higher-performance sensors.
  • Evaluate performance on larger testbeds and more varied materials before considering field use.

These are planned research directions, separate from the capabilities demonstrated in this submission.

Team and acknowledgments

McNeese Nexus — McNeese State University

Prince Pudasaini, Dipendra Kandel, Cooper Tadlock, Caden Fontenot, Yuni Lin, Pratikshya Parajuli, Rajesh Karki, Pratik Thapa, and Arjun Gautam.

Special thanks to the McNeese ACM Student Chapter and the McNeese Department of Engineering & Computer Science for supporting the project.

Explore the source code and setup instructions.

Built With

Share this project:

Updates

Submission history