Semidirectional YOLO-Based Aquatic Unit (SYBAU)

AI vision system that detects and tracks targets to direct a 100 PSI smart water stream — enabling the future of interactive productivity tools.

Comment

Inspiration

Imagine jokingly trolling your friend with a face-tracking water gun — that playful idea sparked our exploration into embodied AI, systems that not only perceive the world but physically interact with it in meaningful ways.

The project began as a fun concept brainstormed over dinner: a face-tracking system capable of directing a high-pressure water stream. As development progressed, the idea evolved beyond novelty into a broader investigation of human-centered applications, where we believe in a strong use-case for productivity/education, allowing everyone to "lock-in" for their meetings with a fresh blast of water.

How We Built It

SYBAU combines real-time computer vision, robotic arm manipulation, and high-pressure fluid control into a modular platform.

Vision and AI

We used YOLOv8-medium for real-time object detection and tracking, running locally on an NVIDIA RTX 4070 GPU to enable low-latency edge inference. A 360° fisheye camera provides wide environmental awareness, allowing the system to detect targets anywhere within its field of view.

The detection pipeline includes:

  • Image acquisition from the fisheye camera
  • Object detection using YOLOv8
  • Target selection and tracking
  • Conversion of bounding box coordinates into servo motion commands
  • A real-time feedback control loop that continuously adjusts the robotic arm to keep the target (e.g., a face) centered in the frame

Servo positioning maps image-space coordinates into angular control:

$$ \theta_x = f_x(u), \quad \theta_y = f_y(v) $$

where (u,v) are the detected object coordinates in image space.

Mechanical and Control Systems

The physical actuation system consists of:

  • Dynamixel servos (daisy-chained) for smooth pan/tilt targeting
  • OpenRB-150 microcontroller running low-level Arduino firmware for deterministic control
  • A 12V diaphragm water pump (RV/marine-grade)
  • A 12V relay to safely switch pump power
  • Interchangeable nozzle tips for different water stream profiles
  • Custom 3D-printed mount

The architecture separates perception and control:

  • High-level edge AI and computer vision are implemented in Python on the GPU
  • Low-level firmware on the microcontroller handles real-time servo positioning and relay switching

This separation allows fast visual inference while maintaining stable, responsive hardware control through a closed-loop feedback system.

Challenges We Faced

  • Selecting and sourcing the right materials and components to balance durability, safety, and performance
  • Calibrating the fisheye camera and compensating for lens distortion in the vision pipeline
  • Interfacing high-level AI software with low-level embedded firmware
  • Designing reliable wiring and power systems for stable hardware operation
  • Ensuring mechanical alignment so all moving parts operate smoothly as an integrated system
  • Working with wide-angle optics and translating distorted image coordinates into accurate servo control
  • Designing and CAD-ing custom 3D-printed parts robust enough to withstand motion, water pressure, and repeated use

What We Learned

  • How to design and integrate a complete moving robotic system combining AI, electronics, and mechanical engineering
  • Practical lessons in hardware reliability, real-time control, and system integration
  • How to build and tune a fully functional AI-controlled water delivery system

Future Directions

While currently focused on face tracking, SYBAU can generalize to many detection tasks in the future. Potential applications include:

  • Fire detection and suppression assistance
  • Agricultural or home pest mitigation
  • Interactive entertainment and amusement use cases, and haptic feedback for movies etc.

Built With

Share this project:

Updates