VoiceFlow AI

Turn messy voice thoughts into real actions—automatically routing bugs, ideas, reminders, decisions, and Slack requests to the right tools.

Comment

The Problem

We all think out loud.

A single voice note can contain a bug to fix, a UI idea, a reminder for tomorrow, a decision made with a teammate, and a message that needs to be sent to a team channel.

But after speaking, someone still has to manually turn those thoughts into GitHub issues, tasks, notes, and messages.

VoiceFlow AI removes that friction.

Instead of forcing users to organize their speech before they speak, VoiceFlow AI lets them talk naturally and turns the conversation into structured actions.

The Inspiration

The idea came from a simple question:

What if a voice interface could understand not just what I said, but what I actually need to do with each part of it?

For example, a user can say:

"The result page has a bug. I also think the dashboard needs a better dark theme. Remind me tomorrow at 10 AM to check the API key. We decided that the onboarding flow should require email verification."

VoiceFlow AI understands that this is not one task. It is multiple independent thoughts with different destinations.

The bug and UI idea can become GitHub issues, the reminder becomes a Todoist task, and the onboarding decision becomes a Notion note.

How It Works

VoiceFlow AI follows:

Speak → Transcribe → Segment → Understand → Route → Execute

  1. Voice Capture The user speaks naturally through the mobile application.

  2. Transcription AssemblyAI converts the voice input into text.

  3. Semantic Segmentation The transcript is separated into individual thoughts instead of treating the entire recording as one instruction.

  4. Intent Classification Each thought is classified as a bug, idea, reminder, decision/note, explicit Slack request, or something that should be skipped.

  5. Smart Routing

  • GitHub → bugs and ideas
  • Todoist → reminders
  • Notion → decisions and notes
  • Slack → only when the user explicitly asks to send something
  • Skip → secrets, credentials, or irrelevant content
  1. Execution VoiceFlow AI performs the appropriate action through the connected APIs.

The system also uses confidence-aware handling so ambiguous instructions are not blindly executed.

Mobile + Monetization

For Shipaton, we brought VoiceFlow AI into a mobile experience and integrated "RevenueCat" for subscription management.

The free experience lets users try the core voice-to-action workflow, while VoiceFlow Pro provides the premium experience through RevenueCat's entitlement and purchase system.

This makes the project more than a voice demo: it is designed as a product that can actually be shipped and monetized.

What We Built

The project combines a mobile interface with an agentic processing pipeline.

The backend handles transcription, semantic segmentation, classification, routing, and execution, while the mobile application provides the user-facing voice experience and subscription layer.

We also made an important architectural decision: external integrations run after voice capture finishes.

That means a GitHub, Todoist, Notion, or Slack failure does not interrupt the user's recording.

Challenges

The hardest challenge was understanding the boundaries between thoughts.

Human speech is messy. People change topics, add supporting sentences, correct themselves, and combine multiple requests into one voice recording.

For example:

"Ali will handle the onboarding flow that should require email verification."

The second part should remain attached to the decision instead of becoming a separate action.

We therefore combined semantic AI processing with deterministic routing rules for explicit actions such as Slack requests.

Another challenge was avoiding unwanted automation. An agent that creates tasks or sends messages incorrectly can be more frustrating than an agent that does nothing.

This led us to design explicit rules and confidence handling around high-impact actions.

What We Learned

Building an AI agent is not simply about connecting an LLM to APIs.

The difficult part is creating "reliable boundaries around AI decisions".

We learned how important semantic segmentation, deterministic safeguards, confidence thresholds, API failure handling, and delayed execution are when building an agent that can take real-world actions.

We also learned that a good voice interface should adapt to the way people naturally communicate rather than forcing users to speak in commands.

What's Next

Our long-term vision is to make VoiceFlow AI a personal action layer between people and their digital workspace.

Instead of remembering which application to open, users could simply speak naturally and let VoiceFlow AI determine where each thought belongs.

One voice. Multiple thoughts. Real actions.

Built With

Share this project:

Updates

Submission history