KindlyCall

Inspiration

Everyday appointments often hinge on one English phone call. Someone can know exactly what they need and still stall on explaining it, following the receptionist's questions, or negotiating another time. We wanted that task to happen in the language the user is comfortable with, without handing over control of who gets called or what gets committed.

What it does

Speak or type a request in your own language. KindlyCall translates it, finds a business, and shows you the business, number, and a plain-language readback to approve. Only after you approve does CALL-E place the English call. The result comes back in your language as text and speech: a confirmation when the task is genuinely done, or the specific missing information when it is not.

There are three task shapes:

Direct booking.
Inquiry-only comparison: ask several places, commit to none, and you choose.
Availability discovery: collect open slots on one call, you pick one, then a separate approved call books that exact slot.

Saved details and preferences pre-answer predictable questions. An uncertain call outcome stays pending and is re-checked by its existing run id rather than redialed. A dozen languages are selectable, with right-to-left layout for Arabic. The phone call itself is always in English while the user's side is translated. Our demo video uses Hindi as one example.

How we built it

iOS app in SwiftUI, using Apple Speech for speech-to-text, AVSpeech for text-to-speech, Location, and EventKit calendar. It talks over HTTP to the backend.
Backend in TypeScript and Fastify: translation, intent classification, grounded business search, brief composition, a session state machine with the confirm gate, and background polling.
CALL-E is invoked at runtime through its OAuth-protected MCP tools (plan_call, run_call, get_call_run), with an optional REST Developer-API transport (POST /v1/calls, GET /v1/calls/{id}) selectable by API key.
Gemini powers translation, search grounding, classification, and ranking. Keyless fake transports make the whole workflow reproducible offline, which is how the 28 backend tests run without spending call quota.

Challenges we ran into

The core insight is that CALL-E's call status is not the same as the user's task outcome. A call can reach COMPLETED while the appointment was not actually booked. We preserve both, and only show success on explicit task completion. Monitoring timeouts and uncertain network responses hold a pending state instead of inviting a duplicate call. We also worked through account-linked OAuth requirements and a nested result envelope in CALL-E responses, both of which we documented as reproducible integration feedback in the repo.

Accomplishments that we're proud of

A complete native path: request, approval, live call progress, and an honest outcome.
Comparison briefs that prohibit any commitment before the user chooses.
Recovery that re-checks an existing run id rather than starting a second call.
Guardrails enforced in code and tests: an AI disclosure in every brief, card-like input rejected before it leaves the device, and redacted operational logging.

What we learned

The hard part is not making the call. It is knowing when the task is truly done and communicating uncertainty honestly, while never taking away the user's authority to approve each call or their chosen business and constraints.

What's next for KindlyCall

Validate the full real-call flow with intended users across languages, expand localization from that feedback, improve structured appointment extraction, and add authenticated hosting, durable sessions, and encrypted local storage before any public launch. Mid-call intervention and robust hold or IVR handling are future work.