networkAssistAI

Use Case 1: Phone Call Agent + SMS AI Agent Use Case 2: visit internet through phone call and sms not rely on internet.

Comment

Inspiration

Many older adults find modern AI tools difficult to access because they require a smartphone, an app, reliable mobile data, and confidence using unfamiliar interfaces. People living in rural or poorly connected areas may also have phone and SMS coverage when reliable 4G or 5G data is unavailable.

We wanted to make AI accessible through interfaces people already understand: answering a phone call and reading or sending an SMS.

Senior Phone AI was initially designed around Australian information and phone-number rules, but the workflow can be adapted to other countries with localized sources, phone configuration, and safety requirements.

What it does

Senior Phone AI combines a phone-call agent with an SMS follow-up agent.

Before a call, the application uses OpenAI web search to prepare a shared, source-backed daily briefing. The briefing can contain local news, scam alerts, digital-safety information, community services, government updates, benefits, and retirement information. CALL-E receives this briefing before placing the call, allowing the phone agent to discuss its verified contents with the recipient.

An operator can place or schedule a reviewed CALL-E outbound call. The application then monitors the call status, summary, and transcript.

If the senior asks a question requiring new internet research, CALL-E records the request and asks whether the person wants the answer sent to the same phone number by SMS. After the call finishes, the application verifies the request and consent, searches current public information with OpenAI, and prepares a short, customer-friendly answer with a source.

The Calls page keeps the phone call and its corresponding SMS together, making it easy to see:

  • when the call occurred;
  • whether it was completed;
  • what was discussed;
  • whether an SMS follow-up was requested;
  • the exact SMS content; and
  • whether the message is a preview, sent, delivered, or failed.

The local demonstration uses SMS preview mode, so judges can inspect the complete workflow without sending a real message. Twilio delivery can be enabled separately for allowlisted test numbers.

Use case 1: Phone Call Agent and SMS AI Agent

Older adults do not need to learn how to operate an AI application. They can have a normal phone conversation and receive the result as an SMS.

The phone call provides a familiar conversational interface. The SMS provides a durable answer that the recipient can read again, retain, or show to someone helping them.

A planned extension will support two-way SMS conversations. A senior will be able to reply with another question, and the application will search current public information and respond within the same consented SMS conversation.

Use case 2: Accessing online information without mobile data

In some rural and regional locations, conventional voice calls or SMS may remain usable even when mobile-data coverage is weak or unavailable.

When voice or SMS service is still available, Senior Phone AI can act as a bridge to online public information. A person could ask about local services, transport updates, weather information, news, scam warnings, or nearby businesses without browsing the web themselves.

The application does not replace emergency services or provide personal medical, legal, or financial decisions. Emergency requests must be directed to the appropriate local emergency service.

How we built it

Senior Phone AI is a Next.js and TypeScript application integrating:

  • CALL-E for outbound AI phone calls;
  • OpenAI for daily briefing generation and current web research;
  • Twilio for optional SMS delivery and delivery-status tracking;
  • React for the operator interface;
  • provider adapters for safe preview and live modes; and
  • deterministic tests that do not place calls or send messages.

The application normalizes telephone numbers to E.164 format, masks numbers in the interface, requires call review and confirmation, prevents duplicate dispatches, encrypts local operational state, and allows pending scheduled calls to be cancelled.

Calls, summaries, transcript turns, SMS previews, and delivery results appear together in one monitoring table.

Challenges we ran into

The main technical limitation was that the current CALL-E integration cannot invoke our OpenAI web-search tool during a provider-hosted phone call.

We adapted the architecture into two complementary paths:

  1. Information likely to be useful during the call is searched beforehand and included in the daily briefing.
  2. A new question is captured during the call, together with SMS consent, then searched after the call and returned as a sourced SMS.

We also had to handle transcript availability, duplicate provider requests, phone-number privacy, explicit SMS consent, delivery uncertainty, Twilio trial restrictions, and the difference between a previewed message and a message actually accepted by a carrier.

Accomplishments that we are proud of

We created an end-to-end workflow that connects call preparation, AI phone conversations, current-information research, SMS composition, and monitoring.

We are especially proud that the demo:

  • gives older adults a familiar way to access AI;
  • keeps every SMS traceable to its related call;
  • converts raw call summaries into useful customer-facing messages;
  • supports source-backed daily knowledge;
  • makes all real-world actions explicit and reviewable; and
  • remains safe to demonstrate without live calls or SMS by default.

What we learned

Voice and SMS can make AI more accessible when conventional applications create usability or connectivity barriers.

We also learned that voice and SMS work best as complementary channels. A phone call is natural for asking questions, while SMS is better for preserving detailed answers, links, addresses, and sources.

Building a trustworthy system required more than connecting APIs. Consent, privacy, idempotency, delivery status, cancellation, source quality, and honest failure handling were essential parts of the product.

What's next for Senior Phone AI

Our next steps are:

  • support consented two-way AI conversations over SMS;
  • accept inbound phone calls as well as outbound calls;
  • localize daily briefings for countries outside Australia;
  • add production authentication and durable multi-user storage;
  • deploy public Twilio webhook endpoints and verify carrier delivery;
  • improve source selection and concise SMS formatting; and
  • support escalation to a trusted person when human help is appropriate.

The current demo supports outbound CALL-E calls, shared daily briefings, post-call OpenAI search, SMS previews, combined call/SMS history, and optional Twilio delivery. Inbound calls and two-way SMS questions remain planned features.

Built With

Share this project:

Updates

Submission history