Amo-Bot

Inspiration

The inspiration for Amo-Bot was born from the desire to make long-distance connections feel tangible. We wanted to move beyond flat video calls and create a Valentine's Day themed "Personality-Driven" companion. By simulating the unique personality of a chosen person through generative AI, Amo-Bot provides a physical presence that can listen, respond, and emote in real-time.

What it does

Amo-Bot is an interactive AI desktop pet that mimics human personality:

Speech Interaction: It captures voice input via a web interface, converts it to text, and sends it to the Gemini API to generate context-aware, personality-driven responses. Visual Feedback: The bot displays a live transcript of the conversation on a dedicated screen and expresses emotions through an RP2040-based facial expression system. Dynamic Motion: Using four MG90S servos, the bot can move forward and backward or perform a "Happy Shake" when it detects positive sentiment in the conversation.

How we built it

We utilized a multi-board architecture to balance high-level AI tasks with low-level hardware control:

The Brain (Raspberry Pi 5): This serves as the central hub, managing the web server, audio-to-text conversion (Whisper), and Gemini API calls. The Face (RP2040 Game Kit): A dedicated microcontroller that handles the "eyes" and facial expressions, allowing for smooth, flicker-free animation. The Body (ESP32-S3): This controls the 4-servo drive system on pins 15, 16, 17, and 18.

Challenges we ran into

Serial Communication Noise: We initially faced "Write timeout" and packet corruption errors when uploading firmware to the ESP32-S3. We resolved this by ensuring a shared ground across all modules and using an external power board to isolate motor noise from the logic circuits. Audio Latency: Since the Pi 5 lacked a direct microphone, we had to build a custom web-based "recording-to-upload" pipeline to bridge the audio data from a smartphone to the processing unit. Driver Synchronization: Ensuring the facial expressions on the RP2040 updated simultaneously with the text on the main screen required precise timing in our message-passing protocol.

Accomplishments that we're proud of

Seamless Integration: Successfully connecting the Raspberry Pi, ESP32-S3, and RP2040 to work as a single, cohesive unit. Personality Emulation: Tuning the Gemini prompts so that the bot's responses felt authentically human and consistent with the chosen personality. Robust Firmware: Writing a stable MicroPython driver that manages 4-channel PWM for the servo "legs" while maintaining a responsive UART link.

What we learned

We gained deep experience in embedded system architecture and asynchronous API integration. This project taught us that in complex robotics, power management is just as important as code. By implementing a common ground plane and external power regulation, we learned how to stabilize a system prone to electrical interference.

What's next for Amo-Bot

Computer Vision: Integrating the ESP32-S3 camera for gesture recognition so Amo-Bot can "see" and wave back. On-Device LLM: Exploring the use of localized models to reduce the latency of the personality-driven responses. Enhanced Mobility: Upgrading the 4-servo system to a multi-joint leg design for more expressive "dancing" and walking.