Terrain Aware Speed Control via Event Streaming

This is a terrain aware speed control loop for a walking robot using ML on streaming sensor data through Confluent in real time.

Comment

Inspiration

Robots have to use more than vision in order to navigate the physical world. In order to provide real time updates to determine safe and reliable mobility conditions, I wanted to build a realistic end to end demo of a terrain aware control using machine learning on streaming data, even without physical hardware.

What it does

This project is a closed-loop, real-time terrain classification and speed control system. A simulated robot streams foot force vectors (x,y,z forces from four feet) as a live data stream. A machine learning model predicts the terrain class: 0: Concrete, 1: Grass, 2: Gravel, 3: Mulch, 4: Dirt, 5: Sand The prediction from the ML is streamed back and used to adjust the robot’s speed in real time (slower on loose/unstable surfaces, faster on high-traction surfaces).

The result is a loop: Foot forces -> Confluent Topic -> ML Surface Prediction -> Speed Adjustment -> Step/Move

I also track end-to-end latency (currently ~200 ms) so the system is measurable.

How I built it

-Used Google Cloud Shell + Cloud Editor as the main dev environment -Used Gemini Code Assist to speed up iteration (boilerplate, debugging hints, refactors) -Streaming backbone: Confluent topics carry telemetry and predictions/commands (data in motion). -Producer: A Python producer publishes simulated foot-force data as messages. -Consumer + inference: A Python consumer reads the stream, extracts the 12 force features, runs the trained model, and publishes the predicted surface (and/or speed command) back into the stream. -Controller behavior: The simulated robot updates speed based on the predicted surface, closing the loop. -UI + logging: The demo UI shows current surface state, predicted surface, speed changes, and logs/latency so it’s easy to validate what’s happening. -Simulation: Using HTML and JavaScript and CSS with a Google Cloud VM a simple grid is implemented in order to allow the robot to change terrain and have different forces given by QCAT Legged Robot Dataset, cited below

Ahmadi, Reza ; Nygaard, Tønnes ; Kottege, Navinda ; Howard, David ; Hudson, Nicolas (2020): QCAT legged robot terrain classification dataset. v2. Commonwealth Scientific and Industrial Research Organisation.dataset. https://doi.org/10.25919/5f88b9c730442

Challenges I ran into

  • Making a robot like loop without a physical robot: I had to make the simulator produce forces that are realistic enough for the model and still be stable for streaming.
  • Learning Google Cloud and Confluent: This was my first time using both of these software's so they seemed a little foreign and took playing with in order to get them to talk together and send data and receive data from Confluent topics.

Accomplishments that we're proud of

-Built a true closed-loop system: streamed foot-force telemetry → ML surface prediction → speed control → updated telemetry -My terrain classifier hits ~98% F1 on held-out data, and in the live streaming loop it drives real-time speed adaptation with ~200 ms end-to-end latency. -Successfully classified 6 terrain types (Concrete, Grass, Gravel, Mulch, Dirt, Sand) from 12-axis foot-force features with -Integrated Confluent event streaming as the backbone for both telemetry and control commands -Created a working robot simulator/digital twin that demonstrates the pipeline without requiring physical hardware -Added detailed logging and observability to verify predictions, commands, timing, and system behavior -Demonstrated terrain-aware speed adaptation that changes robot behavior dynamically based on predicted traction -Packaged the project so it’s reproducible (clear run steps + consistent demo behavior)

What we learned

-Streaming ML isn’t just “run a model” — it’s contracts, timing, and reliability: message structure, ordering, throughput, and safe actuation. -Closed-loop systems need stability logic (not just classification accuracy) to prevent control oscillations. -Even a simple model becomes much more interesting when it drives a real-time decision loop and is measurable with latency/throughput metrics.

What's next for Terrain Aware Speed Control via Event Streaming

The next step is moving from simulation to hardware: collect real terrain force signatures on a legged robot and retrain the classifier on real-world data. From there, this can be used to make robots safer and more efficient in messy environments—construction sites, farms, search-and-rescue, by automatically adapting speed and gait when traction changes.

Built With

Share this project:

Updates

Submission history