Inspiration

When we started this project, we weren’t thinking about AI or hardware. We were thinking about the people we care about. We thought about our mothers, who stop cooking every time the doorbell rings. We thought about our grandmothers, standing alone in front of the intercom, wondering whether the person outside can be trusted. We also thought about ourselves—missing important deliveries because we were in meetings or away from home. We also realized another problem. For many women living alone, simply answering the door can feel unsafe. Speaking through the intercom may reveal that they are home, that they are living alone, or even allow a visitor to guess their age or gender. Ignoring the doorbell isn’t always a solution either, since visitors may continue knocking or return later. An intercom is supposed to make people feel safe. Instead, it interrupts daily life and forces people to make quick decisions every time someone rings the bell. Modern smart doorbells offer remote monitoring, but most require replacing the existing intercom or installing new outdoor hardware. That’s expensive, difficult in rental apartments, and unnecessary when the current intercom already works. So we asked ourselves a simple question: What if the intercom people already own could understand visitors, hold conversations, and respond on their behalf? That question became GenkanAI.

What it does

GenkanAI is an attachable AI system that transforms an existing intercom into an intelligent front-door assistant. When someone rings the doorbell, GenkanAI operates the existing intercom, answers the call, listens to the visitor, understands why they are there, and responds naturally based on rules defined by the resident. For example, it can:

Ask delivery drivers about their package and preferred delivery method.
Politely decline unwanted sales visits.
Verify maintenance workers by asking for their company and appointment details.
Take messages for the resident.
Notify the resident only when human attention is actually needed. For users living alone, GenkanAI also provides an additional layer of privacy and security. Instead of using their own voice, residents can choose an AI voice—including a lower-pitched voice—to answer visitors. This helps reduce the chance of revealing personal information such as gender, age, or whether someone is living alone. The goal is not to impersonate another person. GenkanAI clearly identifies itself as an AI assistant while protecting the resident’s privacy during conversations. Most importantly, users don’t need to replace their existing intercom. GenkanAI simply adds intelligence to the device they already have.

How we built it

Our prototype combines embedded hardware, real-time voice AI, and software into one complete system. At its core is an ESP32 microcontroller. A microphone captures audio from the existing intercom, while a servo motor physically presses the intercom’s talk button. This allows GenkanAI to work without opening or modifying the intercom itself. The interaction flow is:

Detect an incoming visitor.
Press the existing intercom button using a servo.
Capture the visitor’s voice.
Send audio to a real-time AI model.
Generate and play an appropriate spoken response.
Continue the conversation until the visitor’s request is handled.
Summarize the interaction and notify the resident. Instead of building a simple chatbot, we designed GenkanAI as an AI agent capable of understanding context and following household-specific rules. For example, it can automatically reject salespeople, collect information from delivery drivers, or escalate important situations to the resident. To maximize safety, we separated conversation from physical actions. The AI can talk, understand, and make recommendations, but sensitive actions—such as unlocking doors—always remain under human control.

How we used Codex

Codex became much more than a coding assistant—it became an engineering and creative partner throughout the project. We used Codex to:

Design the overall system architecture.
Develop ESP32 firmware.
Build Python tools for serial communication and audio transfer.
Debug hardware communication issues.
Investigate incomplete audio recordings.
Improve reliability through iterative testing.
Break a complex hardware project into manageable development steps.
Explore safer ways to separate AI reasoning from physical device control.
Plan the structure and flow of our presentation video.
Support the video editing process, including organizing footage, refining subtitles, adjusting timing, and preparing the final export. Hardware development rarely works perfectly on the first try. Instead of asking Codex to build everything at once, we used an iterative workflow: implement one feature, test it on real hardware, analyze logs together, improve the design, and repeat. We used the same approach when creating our submission video. Codex helped us organize the story, decide which parts of the prototype to show, refine the narration and subtitles, and improve the overall pacing of the edit. This approach allowed us to gradually transform an idea into a working prototype and communicate the product clearly through the final video.

Challenges we ran into

The biggest challenge was connecting modern AI to hardware that was never designed for it. Every intercom is different. Button positions, speaker volume, microphone quality, and call behavior vary significantly. Instead of modifying the internal electronics, we decided to build an external device that physically interacts with the intercom. While this makes installation much easier, it also introduced mechanical challenges such as button alignment and servo reliability. Capturing clean audio was another major challenge. During testing, we occasionally received fewer audio samples than expected, forcing us to redesign our buffering and synchronization strategy. We quickly learned that building a reliable voice system requires much more than simply connecting a microphone to an AI model. Latency was equally important. A visitor standing at the front door expects an immediate response. Every stage—recording, transmission, speech recognition, reasoning, speech synthesis, and playback—adds delay. We spent considerable effort minimizing unnecessary processing to keep conversations feeling natural. Finally, we had to think carefully about safety. Should the AI unlock the door? What personal information should it share? How should it respond if it isn’t confident? Our answer was simple: the AI should only do what can be done safely. It can talk, verify, summarize, reject unwanted visitors, and notify residents—but high-risk decisions always remain under human control.

What we learned

The biggest lesson we learned is that building AI for the physical world is fundamentally different from building software. When software makes a mistake, users can often click a button and try again. When an AI agent makes a mistake at someone’s front door, it affects a real person in a real situation. We also learned that the true value of an AI agent isn’t simply speaking naturally. Its real value lies in understanding context, following household preferences, protecting privacy, and knowing when to involve a human. Finally, we realized that innovation doesn’t always require replacing existing hardware. Millions of homes already have working intercoms. Instead of asking people to throw them away, we wanted to make those devices smarter. GenkanAI isn’t just about building a better intercom. It’s about helping families, protecting people living alone, and giving everyone peace of mind that—even when they can’t answer the door—someone they trust still can.