Emotion-Aware Autonomous Service Robot

Inspiration

We were inspired by the need for service robots that can do more than simply move around a space. In cafés, offices, and public venues, human service depends not only on physical assistance but also on understanding people. We wanted to build a robot that could navigate, interact with objects, and respond to customers in a more natural and socially aware way.

What it does

Emotion-Aware Autonomous Service Robot combines the mobility of the Go2 robot with the dexterity of the S01 manipulator. It can navigate indoors, approach tables and chairs, pick up everyday objects, and assist with practical service tasks. By integrating the Interhuman API, it can also detect customer mood and adapt its responses. Moreover, it asks for customer's confirmation before it executes an action like grasping an object.

How we built it

We built the project by integrating the Go2 mobile robot platform with the S01 robotic arm to create a complete indoor service assistant. We used the Cyberwave Python SDK to communicate with the robots and send commands between our software system and the hardware.

During the hackathon, Cyberwave provided us with their Vision-Language-Action (VLA) models, which generate control commands for the robots. We integrated these models with autonomous navigation, object manipulation, and the Interhuman API for emotion awareness, enabling the robot to understand customer mood and adapt its responses accordingly.

As part of the development process, we first connected to the robots in simulation and sent basic control commands to verify communication and system connectivity. Getting reliable command execution and establishing stable communication channels was challenging at the beginning, but it helped us better understand the robot interfaces and SDK workflow.

At the same time, we worked on physically integrating the S01 robotic arm with the Go2 quadruped platform. Mounting the arm on top of the robot required careful consideration of hardware compatibility, balance, and system integration to create a unified mobile manipulation platform capable of both navigation and object handling.

Challenges we ran into

One of the main challenges was the time constraint. To properly test a VLA model on our specific service task, we needed to collect a custom dataset, fine-tune the model, and then test the policy on the robot. However, the hackathon timeline was not long enough to complete the full data recording, training, and evaluation pipeline.