VoiceX

Inspiration

Most voice assistants are limited to timers, weather, and music. Meanwhile, the real value of an assistant, actually completing tasks like sending emails, making calls, and scheduling things, has stayed locked behind typing into apps one at a time. I wanted to build something that treats speech as a real command interface, where saying a task out loud is enough to get it done, end to end, with no follow up steps required from the user.

What it does

VoiceX is a voice native AI assistant powered by the Gemini API. A user speaks a task in natural language, and VoiceX understands the intent and executes it autonomously. It can draft and send real emails, send SMS and WhatsApp messages through Twilio, schedule messages for later delivery, place outbound phone calls using text to speech, autonomously shop on Amazon, perform web searches, manage calendar events and reminders, and trigger automations across more than five thousand apps through Zapier. Every feature is built to actually execute, not simulate, the requested action.

How we built it

The Gemini API powers intent understanding and task orchestration, interpreting a spoken command and determining which action or combination of actions to take. Firebase handles backend data and user state. Twilio integration enables real SMS, WhatsApp, and outbound calling with text to speech. Zapier integration connects VoiceX to thousands of external apps for broader automation beyond the built in features. Nearly all of the code was written using Google Antigravity, an AI coding agent, allowing a single developer to build and ship a wide range of real, working integrations.

Challenges we ran into

Making sure every feature performed a real action rather than a mocked or simulated one, since the entire premise of VoiceX depends on tasks genuinely completing, not just appearing to. Reliably parsing natural spoken language into structured actions across many different task types, from email drafting to shopping to calendar management. Coordinating multiple third party integrations, including Twilio and Zapier, alongside the Gemini API, so that a single spoken sentence could correctly route to the right service and complete successfully.

Accomplishments that we're proud of

Building a voice assistant where speaking a task actually results in that task being completed, across a wide range of real world actions, rather than a narrow demo of one or two capabilities.

What we learned

Building VoiceX reinforced that voice interfaces are only as useful as the actions behind them, since understanding a spoken command is meaningless without genuinely executing it. We also learned that coordinating several real third party services, such as Twilio for messaging and calls and Zapier for cross app automation, requires careful handling of failure cases, since a voice command that partially executes can be more confusing to a user than one that clearly fails. Finally, building almost entirely through an AI coding agent taught us to verify functionality directly rather than assume a reported implementation actually works end to end.

What's next for VoiceX

xpanding the range of natural language commands VoiceX can reliably interpret without needing rigid phrasing. Adding proactive suggestions, where VoiceX recognizes patterns in a user's requests and offers to automate recurring tasks on its own. Deepening Zapier based automation so more of a user's daily app ecosystem can be triggered entirely by voice. Improving call and messaging reliability across different carriers and regions as usage grows.

Built With

• firebase
• gemini-api
• google-antigravity
• javascript
• rest-api
• twilio
• zapier