Noise2Node

In high acuity fields such as autonomous vehicles

Comment

## Inspiration

My FSD tesla saved my life preventing a unforeseen car crash in a split second. It was so aware of its surroundings that no average human driver could see. From then on, I was in awe how it achieves this but also feared when it would fail just once. Furhermore being part of my school's FSAE team, I am serious about serious about AV safety. I used this hackathon to demonstrate I can implement a state estimation module but with specific constraints and newer lightweight NVIDIA technologies like Nemotron and TensorRT. This hackathon is just a start. Autonomous systems and edge robotics require split-second situational awareness, but traditional pipelines struggle to balance heavy deep learning inference with deterministic, real-time safety constraints. I wanted to build a hybrid perception pipeline that bridges high-performance hardware acceleration with robust state estimation.

## What it does

Noise2Node (OmniSense-Edge) is a real-time, edge-native ROS 2 perception and sensor fusion framework. It ingests raw LiDAR point clouds and IMU streams, leverages custom CUDA kernels for zero-copy point cloud filtering, runs accelerated 3D object detection via TensorRT, and stabilizes vehicle trajectories using a robust C++ Extended Kalman Filter (EKF), all while integrating a Nemotron diagnostic agent for intelligent pipeline monitoring.

## How I built it

  • Framework: Built using ROS 2 (Humble/Iron) C++ lifecycle nodes (rclcpp_lifecycle) for deterministic startup and runtime safety.
  • GPU Acceleration: Developed custom CUDA shared-memory kernels (.cu/.cuh) managed via ament_cmake_cuda to filter point clouds directly on device memory.
  • Inference: Converted pre-trained 3D detection models into high-performance TensorRT .engine formats for lightning-fast forward passes.
  • State Estimation: Implemented an error-state EKF using the Eigen linear algebra library to fuse high-rate IMU kinematics with sparse bounding-box detections.
  • Infrastructure: Utilized an NVIDIA L40S cloud instance via Brev for CUDA compilation and hardware-accelerated pipeline execution.

## Challenges I ran into

  • Managing zero-copy tensor bindings between CUDA filter outputs and the ONNX/TensorRT inference engine without triggering host-device memory bottlenecks.
  • Setting up robust cross-compilation and workspace packaging (colcon) to handle C++20, CUDA nvcc, and ROS 2 dependencies cleanly.
  • Balancing EKF covariance noise tuning ($Q$ and $R$) to maintain smooth trajectory tracking during rapid sensor updates.
  • Time Constraints: Due to a tight development crunch, we were unable to implement a graphical visualization system such as RViz2 or Foxglove to visually render the real-time point clouds and trajectories.

## Accomplishments that I am proud of

  • Successfully engineering a fully asynchronous perception loop capable of sub-10ms point-cloud filtering and inference on an enterprise NVIDIA GPU.
  • Implementing a clean hybrid workflow and coding locally on Mac via Claude Code and pushing to GitHub before building and executing on the remote NVIDIA L40S instance.

## What I learned

  • The immense power of zero-copy device memory allocation when passing data directly from custom CUDA pre-processing filters into TensorRT runtimes.
  • The structural advantages of ROS 2 lifecycle nodes in managing complex, multi-threaded robotic initialization sequences.

## What's next for Noise2Node

  • Integrating visualization tools like RViz2 and Foxglove Studio for live 3D telemetry rendering.
  • Expanding live hardware integration onto physical mobile robots and Jetson edge kits.
  • Deepening the Nemotron agent integration to autonomously flag perception edge cases and log dynamic safety warnings in real time. *Keep following the repository post hackathon to see new developements and links!

Built With

Share this project:

Updates

Submission history