Kunkun

The inspiration for this project came from a practical problem in autonomous cleaning robots: a robot must understand not only where it can move, but also what type of floor it is currently cleaning. Different floor materials and textures may require different cleaning strategies. However, real-world floors often contain reflections, repeated patterns, weak textures, shadows, and transitions between multiple materials, making reliable recognition difficult.

To solve this problem, I developed a ground-material recognition and mapping system that combines deep-learning-based visual detection with spatial and temporal decision logic. The system is designed to recognize floor characteristics during both rapid mapping and automatic cleaning, and then associate the recognition results with specific areas on the robot’s map.

I built the project in several stages. First, I trained and evaluated lightweight YOLO-based models for structured-light point detection and floor-material recognition. Because the model needed to run on embedded hardware, I tested different model architectures, pruning ratios, confidence thresholds, and post-processing parameters. I compared the models using detection recall, predicted point counts, stability across different datasets, and actual inference latency on the target device.

The selected model was then integrated into a ROS-based processing node. The node receives AI detection results, robot motion states, working modes, and area information. It aggregates observations within map regions and applies different decision rules depending on the robot’s operating mode. Rapid mapping uses responsive rules to generate results quickly, while automatic cleaning uses accumulated grid statistics and straight-line motion information to improve reliability. Multithreading and mutex protection were also introduced to ensure that AI results, map data, and robot states could be processed safely without blocking the main system.

Through this project, I learned that model accuracy alone is not sufficient for a production robotics system. A model with good offline metrics may still fail after deployment because of domain differences, confidence collapse, unstable lighting, hardware-specific execution behavior, or errors in temporal aggregation. I also learned that smaller models are not always faster on an NPU. Channel alignment, operator fusion, memory access, padding, and scheduling overhead can have a greater influence on latency than the model’s parameter count.

The main challenge was achieving a balance between accuracy, speed, and stability. Some models performed well on the original dataset but produced significantly fewer detections on newly collected data. Lowering the confidence threshold did not fully solve the problem, which showed that the root cause was domain mismatch rather than simple post-processing. Another challenge was converting frame-level predictions into stable area-level conclusions. Individual detections could fluctuate, so I introduced minimum observation counts, label-ratio thresholds, spatial coverage conditions, and motion-based refinement rules.

Overall, this project taught me how to build a complete applied AI pipeline—from dataset analysis and model training to embedded deployment, ROS integration, performance profiling, rule design, and real-world validation. More importantly, it showed me that a reliable robotic perception system must combine machine learning, domain knowledge, engineering constraints, and evidence-based testing.

Built With

• chinese