Forbi

An open-source autonomous robot fleet with AI-powered exploration, privacy-first vision, and natural language control — built on a $30 microcontroller.

Inspiration

Commercial inspection and patrol robots start at $50,000. Fleet management platforms add another six figures in annual licensing. Meanwhile, small warehouses, greenhouses, and remote facilities still send humans on repetitive, dangerous routes — not because better solutions don't exist, but because the existing ones are financially out of reach.

We asked a simpler question: what if an ESP32, a handful of sensors, and some clever software could do the same job?

Forbi was inspired by the gap between what the maker community can build (impressive single robots) and what the real world needs (managed fleets with intelligence, safety, and privacy). We wanted to prove that autonomous robotics doesn't require a corporate budget — just curiosity, open tools, and a willingness to build in the open.

What it does

Forbi is a complete robot fleet platform with two components:

The Robot (ESP32 Firmware)

4-wheel mecanum drive with per-wheel optical encoders for odometry
LiDAR SLAM navigation using a 360-degree LD6 laser scanner
Ultrasonic obstacle detection with audible buzzer alarm
INA226 battery monitoring over I2C
WiFi STA/AP with automatic fallback to a configuration access point
OTA firmware updates directly from the web interface
8-direction motor control with a 500ms safety timeout to prevent runaway

The Platform (Edge BotHub)

Fleet management — register unlimited robots, assign them to sites with floor plans, track their real-time position on a map
Autonomous exploration — assign a mission ("find the north exit"), and the robot navigates using its camera + Qwen-VL AI, anonymizing every photo it takes along the way
Natural language control — tell your robot "go forward until you see a door" in plain English (or Spanish, or any language), and it translates your words into movement commands via Qwen LLM
Real-time monitoring — live ping, camera proxy, ultrasonic readings, battery status, encoder counts, and SLAM position for every robot
Privacy-first vision — every captured photo is automatically processed with OpenVINO + YOLOv8n-seg to detect people and overlay emoji on their faces before storage
SLAM mapping — particle filter SLAM builds occupancy grid maps from LiDAR data, with configurable origin offsets per robot
JWT authentication with role-based access (admin/user) and HTTP-only cookies
Full REST API for custom integrations and dashboards

How we built it

Hardware

The robot chassis uses 4 DC gear motors with quadrature encoders, driven by an L298N dual H-bridge. The brain is an ESP32-WROOM-32 dev board. Sensors include an INA226 power monitor (I2C), an LD6 LiDAR (UART at 230400 baud), and an HC-SR04 ultrasonic sensor. Everything runs on a 2S LiPo battery.

Firmware (C, ESP-IDF v5.5)

The ESP32 firmware is written in bare-metal C using FreeRTOS. We built modular drivers for each subsystem — motors with PCNT hardware encoders, INA226 I2C communication with CRC validation, LD6 LiDAR packet parsing, ultrasonic timing, and a full HTTP REST API with 18 endpoints. Configuration is persisted in NVS flash and editable through an embedded web UI. The web pages (dashboard, config, events, OTA) are compiled directly into the firmware binary.

Backend (Python, FastAPI)

BotHub runs on FastAPI with SQLAlchemy ORM backed by MySQL. We implemented a complete CRUD system for users, sites, devices, and missions, plus async background tasks for autonomous exploration and auto-move. The exploration engine captures camera images, anonymizes them with OpenVINO, sends them to Qwen-VL for AI navigation decisions, and executes movement commands in a loop. SLAM processing uses BreezySLAM's particle filter algorithm.

AI Integration

We integrated two Qwen Cloud models: Qwen-Plus for natural language command parsing and Qwen-VL-Plus for vision-based autonomous navigation. The LLM system prompt maps natural language to exactly one of 9 robot commands, with automatic obstacle avoidance via ultrasonic feedback.

Privacy Pipeline

Every image the robot captures passes through an OpenVINO-optimized YOLOv8n-seg model that detects persons and overlays smiley emoji on detected faces. This runs locally on the edge server before any image is stored or sent to the cloud.

Challenges we ran into

GPIO pin conflicts. The ESP32-WROOM has limited pins. Front encoder pins (GPIO4/13/33/34) had to be shared with the ultrasonic sensor and buzzer, forcing us to implement two distinct sensor modes rather than supporting everything simultaneously.
Motor safety. Without a safety timeout, a lost WiFi connection could leave the robot moving indefinitely. We added a 500ms dead-man's switch that stops all motors if no fresh command arrives.
LiDAR data parsing. The LD6 outputs raw binary packets with CRC8 validation. Getting the packet framing, angle interpolation, and point extraction right required careful byte-level work.
OTA multipart handling. Implementing firmware OTA upload directly on the ESP32's HTTP server meant parsing multipart form data by hand — finding boundaries, stripping headers, and streaming binary data to the flash partition.
Anonymization performance. Running YOLOv8n-seg inference on every captured frame needed optimization. We settled on OpenVINO's CPU backend with a 640px input resolution to balance accuracy and speed.
Multi-language AI. The Qwen prompts needed to work in any language the user writes in, not just English or Spanish, while still mapping to the same fixed set of robot commands.

Accomplishments that we're proud of

A $30 robot that does what $50K machines do. Full fleet management, SLAM navigation, AI-powered exploration, and natural language control — all on commodity hardware.
Privacy by design. Not an afterthought, not a setting you can toggle off. Every single photo is anonymized before storage. The robot sees the world but doesn't remember faces.
Speak to your robot. "Avanza hasta que veas una puerta" — the robot understands, maps it to a command, starts moving, and stops when its ultrasonic detects an obstacle at 50cm. No code required.
Autonomous exploration actually works. Give it a destination description and it navigates using camera vision + AI, making its own decisions about which way to go.
Two sensor modes, one robot. LiDAR-based SLAM for mapping, or ultrasonic-based obstacle avoidance for simple patrol — switchable from the web UI without reflashing.
Complete documentation. Every pin, every API endpoint, every configuration option — documented in markdown, ready for anyone to replicate.
Full-stack in a weekend. From bare-metal C firmware to FastAPI backend to AI integration to web dashboard — built end-to-end as a cohesive system.

What we learned

Hardware constraints force creative software. The GPIO pin conflict between encoders and ultrasonic led us to design a mode-switching system that actually made the robot more versatile.
AI is a tool, not magic. Getting Qwen to reliably map natural language to exactly one of 9 commands required careful prompt engineering, response parsing, and confidence handling.
Privacy and functionality aren't enemies. The anonymization pipeline adds milliseconds of processing but makes the entire system ethically sound and deployable in real spaces.
The boring stuff matters most. Motor safety timeouts, NVS configuration persistence, WiFi reconnection logic, event logging — none of it is flashy, but all of it is what makes a robot actually usable.
Open source compounds. Every module we built is independent enough to be reused. The anonymizer works standalone. The SLAM engine is a drop-in. The motor driver handles any L298N setup.

What's next for Forbi

WebRTC live video streaming — replace static camera snapshots with real-time video feeds from the robot
Multi-robot coordination — let multiple robots share SLAM maps and divide exploration tasks autonomously
Voice control — integrate speech-to-text so operators can control robots by talking, not typing
Path planning algorithms — add A* and RRT* pathfinding on top of the SLAM maps for goal-directed navigation
Mobile app — a native iOS/Android companion app for field operators
ROS2 bridge — integrate with the ROS2 ecosystem for advanced robotics workflows
Cloud dashboard — a hosted multi-tenant version for managing robots across different sites and organizations
Custom sensor modules — support for gas sensors, thermal cameras, and environmental monitoring payloads