TeraNode Multimodal Sensor Fusion for Terrain Classification

An embedded AI rover that learns to distinguish terrain in real time using onboard sensor data.

Comment

Inspiration

Most small robotic rovers are designed to move across terrain without really understanding what is happening underneath them. A robot can experience a change in traction, vibration, or load long before a human can clearly observe the difference.

That made me interested in a simple question: Can a rover identify the terrain it is driving on using only signals generated by its own movement and motors?

This idea led to TerraNode, a 4WD robotic platform designed to explore terrain classification using multimodal proprioceptive sensing and edge machine learning.

Rather than relying on a camera, the project focuses on signals such as acceleration, angular velocity, vehicle orientation, and motor-current behavior. The long-term goal is to build robots that can use this information to adapt how they move through unfamiliar environments.

What it does

TerraNode collects sensor data while driving across different terrain surfaces.

The current system uses:

          1. MPU6050 IMU for acceleration, angular velocity, and orientation-related information
          2. INA219 for measuring motor-current behavior
          3. PWM information from the motor-control system

From these signals, the system extracts features such as:

          1. Mean and standard deviation of acceleration
          2. Mean and standard deviation of gyroscope measurements
          3. Mean pitch angle
          4. Mean and maximum current
          5. Motor PWM value

These features are used to classify the terrain using a classical machine-learning model.

The trained model is then converted into a lightweight C++ implementation and deployed directly on an ESP32, allowing inference to happen on the robot rather than sending data to a laptop or cloud service.

How I built it

I built TerraNode as a combination of mechanical hardware, embedded software, data collection, machine learning, and deployment.

The rover uses a 4WD chassis, ESP32, motor driver, IMU, current sensor, and battery system. I first built the hardware prototype and created a data-collection pipeline to record sensor measurements while the rover traveled across different surfaces.

The collected data was then transferred to a laptop, where I cleaned and analyzed the measurements, created features, and trained the terrain-classification model.

The workflow was approximately:

Rover → Sensor Data → Feature Extraction → ML Model → C++ Conversion → ESP32 → Real-Time Prediction

One of the main goals was to keep the inference pipeline lightweight enough to run on a microcontroller with limited computational resources.

The project is being developed iteratively, with the current prototype serving as the foundation for a more reliable hardware and robotics platform.

Challenges I ran into

The biggest challenge was that the sensor signals were not as clean or straightforward as they initially appeared.

Different terrain surfaces affected multiple signals simultaneously. Changes in acceleration, vibration, pitch, and motor current could also be influenced by factors such as motor speed, PWM, chassis movement, and environmental conditions.

This made data collection much more important than simply training a model on a small dataset.

Another challenge was moving from a Python-based machine-learning workflow to real-time C++ inference on an ESP32. The model had to be represented in a lightweight form that could run within embedded-system constraints.

The physical prototype also became difficult to manage as more components were added. The large number of jumper wires made the system harder to debug and maintain, which motivated one of my next major upgrades: designing a custom PCB using KiCad.

Accomplishments that I'm proud of

I'm proud that TerraNode goes beyond being a basic remote-controlled rover.

I built the system from the hardware level upward, collected my own real-world sensor dataset, developed the machine-learning pipeline, and deployed the resulting model onto the actual robot.

The most important part for me is the complete hardware → data → ML → embedded deployment loop.

The project also helped me identify weaknesses in my original prototype instead of hiding them. Those weaknesses are now driving the next iteration of the system, including better hardware organization, improved mechanical design, and more rigorous experimentation.

What I learned

TerraNode taught me that robotics is not just about getting a robot to move.

I learned how strongly hardware, sensing, data quality, machine learning, and embedded software depend on each other.

I also gained practical experience with:

      1. Sensor-data collection and preprocessing
      2. Feature engineering for classical ML
      3. Real-world experimental testing
      4. C++ implementation of ML inference
      5. ESP32-based edge computing
      6. Hardware/software integration
      7. Debugging noisy and imperfect sensor data
      8. Iterative robotics prototyping

Most importantly, I learned to treat a robotics project as an experimental system rather than just a collection of components.

Built With

Share this project:

Updates