SiteSignal

Inspiration

In large facilities, people notice problems faster than systems do : a broken elevator, a vehicle incident, or a security threat. We wanted to make reporting as simple as making a phone call or sending an SMS, while giving operators the exact location and next action.

What it does

SiteSignal turns a phone call or SMS into a structured, location-aware incident.

A caller can describe the problem and location by voice, confirm the location with DTMF, or send an SMS such as broken elevator 3143. The incident appears in an operator queue and on a spatial facility map. Operators can acknowledge it, notify maintenance, dispatch security, resolve it, and automatically notify the original reporter by callback or SMS.

How we built it

We built SiteSignal with Vonage Voice API and Vonage Messages API, using inbound webhooks, speech recognition, DTMF input, call events, outbound callbacks, and SMS status events.

The backend is a Fastify/TypeScript service running on Google Cloud Run with Firestore. The frontend is a React interface with a 3D architectural map, incident queue, location markers, and operational actions.

The real-time parsing path is deterministic for reliability. Location codes such as 3143 map reports to a specific area: Zone B, Floor 4.

Challenges we ran into

Speech recognition is not always perfect, especially with location names such as “Zone B.” We added normalization and fallback handling so that imperfect transcripts can still become useful incidents instead of being lost.

We also had to handle webhook retries and delivery events safely. Status callbacks must not create duplicate incidents, so inbound SMS processing is now idempotent.

Another challenge was closing the loop with the reporter. The outbound Voice API callback required a valid NCCO flow and careful separation between incoming calls and resolution callbacks.

Accomplishments that we're proud of

We built a complete operational loop:

Call or SMS
→ structured incident
→ location
→ operator action
→ maintenance or security response
→ resolution
→ reporter confirmation

The system supports both voice and SMS reporting, maps incidents to physical locations, and gives operators clear actions instead of just displaying raw messages.

Most importantly, SiteSignal works through channels people already understand. No new mobile app is required for the person reporting the problem.

What we learned

Voice interfaces need strong fallbacks. Speech recognition should be combined with DTMF and deterministic parsing rather than treated as perfect.

We also learned that webhook systems must be designed for retries, duplicate events, and asynchronous state changes from the beginning.

Finally, the spatial interface is most useful when it supports an operational decision. The map is not just visualization, it helps the operator understand where the problem is and what response is needed.

What's next for SiteSignal

Next, we want to add post-call AI enrichment for transcript normalization, classification, priority scoring, duplicate clustering, and multilingual summaries without putting AI in the critical real-time capture path.

We also plan to add call recording playback, richer facility location registries, more maintenance workflows, and integrations with existing security and work-order systems.

Built With

• 3d
• api
• cloud
• dtmf
• fastify
• fiber
• firestore
• google
• messages
• node.js
• react
• recognition
• run
• sms
• speech
• three
• three.js
• typescript
• voice
• vonage
• webhooks