-
-
Project Name
-
Dashboard
-
chaotic movement
Inspiration
Our inspiration came from observing how homes remain reactive, not proactive — even with modern “smart” devices. We wanted to create a home that doesn’t just automate lights or locks but truly understands human context and events — like noticing when someone’s at the door, when an elderly person misplaces a cane, or when routines change subtly. The goal: a compassionate, event-aware AI home that assists, learns, and protects.
What it does
HomeMindAI is an autonomous home intelligence system powered by dual AI agents: 🧩 Head Robo (Reasoning Agent) — interprets sensory inputs, classifies events, and decides responses. ⚡ Assist Robo (Execution Agent) — performs tasks, communicates via voice or text, and logs events. It detects and responds to real-world household events such as: 👋 Visitor detected at the door (camera vision) 🦯 Empty cane or misplaced item alert (object absence detection) 🔊 Sudden noise or motion anomalies (sound/motion analysis) 🧾 Daily activity and safety monitoring for elderly care 🖥️ Event timeline visualization and voice summary generation The system acts autonomously while maintaining transparency through a beautiful React + FastAPI dashboard and voice reports.
How we built it
We built HomeMindAI using a modular, event-driven AI architecture: Backend: FastAPI (Python) for event processing, REST APIs, and AI routing. Frontend: Vite + React Dashboard with live event cards and reports. AI Stack: GPT-OSS (via Together.ai), AWS Bedrock Agents, and custom event logic. Storage & Processing: Local + cloud event store with timeline and logs. Voice/Report System: pyttsx3 and natural language generation for audible updates. Agents: Twin Robo system — Head Robo (reasoning) + Assist Robo (execution). we also integrated image and video event uploads, timeline feeds, and manual event simulation tools for testing and demoing AI-driven responses.
Challenges we ran into
Designing a stable multi-agent workflow (reasoning → execution → feedback loop). Handling multi-modal event inputs (text, image, NFC, sound) consistently. Achieving low latency between event detection and AI-generated response. Building an intuitive dashboard UI that conveys AI reasoning clearly. Balancing local autonomy with cloud-based intelligence for privacy and scalability.
Accomplishments that we're proud of
Built a fully functional AI home assistant with real-time event detection and reasoning. Designed a Twin Robo architecture — modular, explainable, and scalable for future IoT expansion. Created an elegant React dashboard that visualizes events like “Visitor on Door” or “Empty Cane detected.” Integrated voice-based summaries and daily status reports for accessibility and elder care. Demonstrated true AI agency — the home doesn’t just respond; it thinks and explains why.
What we learned
How to connect multi-agent reasoning systems (LLMs + local automation). Importance of explainable event flow for user trust in autonomous AI. How to optimize AI pipeline latency for real-time responses. Designing with human emotion and empathy in mind for smart home interactions.
What's next for HomeMindAI: Autonomous Event-Driven Home Intelligence
Integrate IoT sensor network (temperature, movement, air quality, fall detection). Add real robotics layer — robotic arm / cleaning bots responding to AI decisions. Expand event taxonomy for health, safety, and energy management. Develop mobile companion app with live event alerts and voice interaction. Launch open-source community version — inviting developers to create “event packs.”
Log in or sign up for Devpost to join the conversation.