Mizuna - AI Companion Pet Robot

An intelligent AI companion robot powered by GPT-OSS 20B for personalized daily assistance and home monitoring

Comment 1

Mizuna - AI Companion Pet Robot

An intelligent AI companion robot powered by GPT-OSS 20B for personalized daily assistance and home monitoring.

Meet Mizuna—your chatty little side-kick on wheels. Instead of just running apps or flashing LEDs, Mizuna sits nearby, listens to your everyday conversations through the Omi Friend Device, and figures out how to make your life smoother: jotting down tasks, lining up reminders, and suggesting breaks when you sound stressed. At the same time, its onboard sensors keep a gentle watch over your room, alerting you to motion or changes in temperature so you always feel safe and informed. Everything happens locally on an open-source GPT-OSS 20B brain, so your memories stay yours, and Mizuna’s personality is yours to tweak—from energetic morning motivator to laid-back evening storyteller. In short, it’s not just a robot—it’s the friend who’s always excited to help.

🤖 Powered by Groq's GPT-OSS

Mizuna is built using leveraging Groq's GPT-OSS-20B model. Developed with feedback from the open-source community, these models push the frontier of open models at their size.

This project is a creative demonstration of what open models can do—combining real-world robotics, local reasoning, and a fully customizable AI agent. Whether you want a robot that plans your day, a local agent that thrives offline, or a hardware hack that surprises everyone, Mizuna is designed to inspire and empower.

Inspiration

Mizuna was inspired by the dream of having a truly personal, privacy-first AI companion that lives in your space—not just on your phone or in the cloud. We wanted to create a robot that could listen, learn, and help in daily life, while being open, affordable, and customizable. The OpenAI Open Model Hackathon and the release of Groq's GPT-OSS models made it possible to bring advanced reasoning and memory to a real-world robot, all while keeping user data local and secure.

🚀 Features

  • 💾 Persistent Memory - Remembers your schedule, tasks, and personal events across sessions
  • 🔒 Privacy-First Design - Open-source LLM ensures full data transparency and control
  • 🏠 Home Security Integration - Motion detection, environmental monitoring, and live surveillance
  • 🎭 Customizable Personality - Adjustable tone, style, and behavioral responses
  • 📱 Remote Control - Manual robot control and real-time home monitoring
  • 💰 Cost-Effective - Affordable alternative to premium personal robots
  • 🧠 Intelligent Daily Planning - Automatically organizes tasks by understanding natural conversations through Omi Friend Device

🎯 Target Users

  • Students - Daily task organization and study planning
  • Remote Workers - Schedule management and home office monitoring
  • Tech Enthusiasts - Customizable open-source AI platform
  • Families - Home security and personalized assistance
  • Privacy-Conscious Users - Local AI processing without cloud risks

How we built it

  • Hardware: Raspberry Pi (robot brain), NodeMCU (motor/sensor control), motor shield, camera, LED matrix, sensors, and custom chassis (can be built from any box or board).
  • Software: Python backend (Flask) for robot control and API, React Native (Expo) mobile app for chat, video, and teleoperation
  • AI: Integrated Groq's GPT-OSS-20B (and optionally 120B) for all reasoning, memory, and conversation, using Groq API or local inference via Hugging Face/Ollama and uses Azure AI Speech to enable high-quality speech-to-text (STT) and text-to-speech (TTS) capabilities. This allows the robot to understand spoken commands and respond with natural-sounding voice output.
  • Features: Persistent memory, agentic planning, customizable responses, and modular hardware/software design.

Challenges we ran into

  • Hardware integration: Ensuring reliable communication between Raspberry Pi, NodeMCU, and all sensors/motors.
  • Real-time AI: Achieving low-latency, high-quality responses from large LLMs on affordable hardware.
  • Privacy: Designing a system that keeps all personal data local, with no cloud lock-in.
  • Cross-platform app: Building a mobile app that works seamlessly on both Android and iOS, including live video and real-time controls.
  • Power management: Supplying enough current for motors, LEDs, and Pi without voltage drops or resets.

Accomplishments that we're proud of

  • Built a fully functional, real-world robot that uses open-source LLMs for real-time conversation and planning.
  • Achieved seamless integration of hardware, AI, and mobile app.
  • Created a privacy-first, customizable, and affordable AI companion that anyone can build and extend.
  • Developed clear, reproducible documentation and open-sourced all code and assets.
  • Designed Mizuna to be accessible and beneficial for a wide range of users, from students to families.

What we learned

  • The power and flexibility of open-source LLMs like GPT-OSS for real-world robotics.
  • The importance of privacy and local processing for user trust and adoption.
  • How to design modular hardware and software that can be adapted to many use cases.
  • The value of clear documentation and community-driven development.
  • That even with limited resources, it's possible to build something truly novel and impactful.

Why Mizuna Qualifies - Competition Categories

🏆 Best Overall

Transforms GPT-OSS from stateless reasoning tool into stateful companion using 128k context for persistent personal relationships. First application to make mixture-of-experts architecture seamlessly switch between emotional intelligence and task planning in real-time.

🤖 Best In Robotics

Showcases GPT-OSS 20B's agentic workflows in real-world robotics with local 16GB deployment. Adjustable reasoning levels become dynamic robot behaviors—quick responses for casual chat, deep reasoning for home security decisions.

⚡ Weirdest Hardware

Uses passive conversation eavesdropping as primary interface instead of traditional inputs. Creates hardware that's part pet, part security system, part assistant—with AI-mood dependent behavior changes.

🎯 Wildcard

Subverts OpenAI's intended use (coding/analysis) by applying chain-of-thought reasoning for social emotional intelligence. Uses transparency features as emotional audit trails showing why the robot cares about your wellbeing.

🌍 For Humanity

Democratizes AI companionship via Apache 2.0 licensing, addressing global loneliness epidemic without corporate data harvesting. Local deployment on affordable hardware reduces digital divide for AI-powered mental health support worldwide.

Quick Start

  1. Review Materials.md and gather all required hardware.
  2. Follow Assembly.md to build and wire the robot.
  3. Set up the Raspberry Pi using Assembly.md.
  4. Install and configure the mobile app using Assembly.md.
  5. Power on, connect, and enjoy your new AI companion!
  6. Add Omi Friend Integration Omi

What's next for Mizuna - AI Companion Pet Robot

  • More sensors: Add support for additional environmental and health sensors.
  • Better autonomy: Enhance navigation and obstacle avoidance with computer vision.
  • Voice interface: Improve speech recognition and synthesis for hands-free interaction.
  • Community modules: Enable plug-and-play extensions for new features and hardware.
  • Educational kits: Develop kits and curricula for schools and makerspaces.
  • Accessibility: Add features for users with disabilities.
  • Open collaboration: Invite the community to contribute, customize, and deploy Mizuna in new
Share this project:

Updates