Handle

Inspiration

My flight to MHacks this year almost didn't happen. I was trying to change my flight to a more convenient time, so I called Frontier Airlines' customer support. However, the support agent was having difficulty understanding me and accidentally cancelled my flight. So, I had to call Frontier customer support while on the way the airport to rebook my ticket an hour before takeoff. This was quite stressful and a really close call, and just felt like something I should NOT have to do. Why couldn't these annoying customer support calls be automated from the customer's side? So many businesses automate their calls nowadays. We should be able to as well. This was the inspiration for Handle, an AI voice agent that works through iMessage and takes the annoying calls for you.

What it does

All you have to do is text Handle on iMessage. You don't need to install or sign up for anything. Just text Handle the problem and who to call. Handle will call the number, stay on hold even if they play music for 30+ minutes, explain your problem, talk to support, and get the issue taken care of. Need to get a reservation canceled, but with no cancellation fee? Handle will make sure you aren't charged a dime. If support ever asks for something that needs your permission, like a payment or plan change, Handle will text you and get your approval before saying anything. If it doesn't get your approval, Handle will play it safe and will not put you at risk.

How we built it

I built Handle in TypeScript, using Photon for iMessage and Gemini to understand requests and check whether calls actually accomplished what the user wanted. ElevenLabs handles the voice conversations, while Twilio connects them to real phone numbers. SpacetimeDB keeps cases, transcripts, and approvals synced with the live dashboard. When the agent needs approval during a call, the backend texts the user through iMessage and makes their response available to the agent so it can continue. I also added optional Gmail OAuth support so Handle can find relevant receipts and confirmations.

Challenges we ran into

Working as a solo hacker was definitely a challenge. Beyond that, getting messaging, voice, and phone providers to work together was harder than expected. I also had to find the right balance between letting Handle act on its own and making sure it asks before agreeing to something you didn't authorize.

What we learned

I learned a lot about how personal agents like Muse and Instinct work behind the scenes. Personal agents are one of the hottest trends in tech right now, and I'm super happy that I was able to learn about how these work and perform complex tasks.

What's next for Handle?

Next, I want to generalize Handle to more use cases. Different types of recurring workflows can be set up as well so that users can type just one or two familiar words, and Handle knows exactly who to call, when to call them, and what to say.