Keny: Your Proactive Voice Intelligence Operative

💡 Inspiration

We live in an era of severe information overload. Professionals and executives lose hours every day doom-scrolling news feeds, drowning in newsletters, and managing dozens of open browser tabs just to find a handful of critical updates.

At the same time, modern AI chatbots are completely passive—they sit idle in a browser tab until you remember to open them and type a prompt.

We asked ourselves: What if your AI didn't wait for you to ask? What if it acted like a real executive assistant—monitoring the web 24/7, filtering out 99% of the noise, and simply calling your mobile phone the moment a critical development breaks? No apps to install, no screen required.

⚙️ What It Does

Keny is an autonomous digital PA that bridges background web surveillance with real-world cellular phone calls:

1. Continuous Monitoring: Keny continuously scans the web for the exact industries, competitors, and topics you care about.
2. AI Relevance Filtering: Google Gemini evaluates every discovered article, discarding noise and verifying whether an update is genuinely novel and impactful.
3. The Phone Call: When a critical signal is verified, Keny dials your phone using the CALL-E telephony network.
4. Two-Way Dialogue & Memory: Keny delivers a crisp, spoken briefing. You can interrupt, ask questions, tell it to "call me back in 20 minutes", or request a different tone. Keny captures structured feedback from your voice and updates its persistent memory.

🛠️ How We Built It

We engineered Keny as an end-to-end autonomous agent pipeline:

• Next.js & React: A modern, clean frontend featuring a step-by-step onboarding flow and a unified card-list dashboard to review past calls and discovered resources.
• LangGraph Orchestrator: A multi-step state graph that orchestrates the entire intelligence lifecycle—from background discovery and vector matching to call dispatch, interruption handling, and callback scheduling.
• Google Gemini & Vector Search: Discovered content and user preferences are converted into high-dimensional vector embeddings using Google's text-embedding-004. Semantic alignment is calculated mathematically via cosine similarity: $$\text{Similarity}(\mathbf{A}, \mathbf{B}) = \frac{\mathbf{A} \cdot \mathbf{B}}{|\mathbf{A}| |\mathbf{B}|}$$ Matches are indexed and searched directly using Google Cloud Firestore Vector Search.
• CALL-E SDK: An outbound telephony engine that places genuine two-way cellular calls, streams low-latency speech, and returns structured user intent directly back to our agent orchestrator.

🚧 Challenges We Faced

• Purging Fake Mocks for Real Infrastructure: Early in development, it was tempting to simulate phone calls inside a web browser or use in-memory vector mocks. We made the strict architectural decision to rip out all fake fallbacks—connecting real Google Cloud Firestore vector search, configuring live GCP Application Default Credentials, and routing calls through actual cellular telephony networks.
• Managing Live Conversational State: Handling phone calls with a stateful AI agent is challenging. If a user interrupts mid-sentence to say, "I'm walking into a meeting, call me back in 45 minutes," the agent cannot crash or lose its place. Using LangGraph memory savers, we had to ensure the orchestrator cleanly paused the briefing, logged the structured callback time, and hung up politely.
• Zero-Fluff Simplicity: We repeatedly stripped away unnecessary complexity—removing complicated multi-tab dashboards, gimmicky graphics, and rigid time caps—to deliver an ultra-minimalist experience centered entirely on high-signal voice intelligence.

🎓 What We Learned

• Voice Demands Absolute Brevity: Unlike text chatbots where long paragraphs are tolerated, voice interfaces require high-density, concise communication. Every sentence delivered over a phone call must earn its place.
• Proactive beats Reactive: The real frontier of AI is not larger chat windows, but autonomous background systems that know when and how to deliver signal directly into the user's real-world routine.

Built With

• calle
• gemini
• javascript
• parrallel