CareLink

• 

  Automated call

• 

• 

  PowerAutomate Workflow

• 

  Dashboard

Inspiration

The inspiration for CareLink came from emergency response hotlines, and how they help callers get the help they need.

What it does

CareLink is an automated hotline assistant. CareLink is designed to help unhoused people find nearby resources such as shelter, legal aid, mental health care and food kitchens. According to the US National Library of Medicine National Institutes of Health, only 55% of the unhoused population has regular access to the internet. This means that they need an alternative way to seek out these resources. Our hotline uses an NLP model in conjunction with Esri ArcGIS products to search the Greater Los Angeles Area to find these resources and connect them with the help they need. CareLink also maintains a detailed dashboard overviewing relevant call statistics to better help organizations use and interpret data to aid the unhoused population.

How we built it

• Phone audio transcription was done using Twilio, a communications library.
• Extracted key terms for location and call category using a natural language processing model from spaCy in python
• Geocoded caller’s location from key terms using Esri’s geocoding service in the ArcGIS API for Python.
• Log caller location and context. Use caller location and context to find the closest resource (referencing real data in Feature Layers).
• Return address and phone number of resource to caller
• Update feature layer hosted in a web map
• Configure an ArcGIS Dashboard that displays data and calculates statistics for visualization
• Integrate feature with Teams: ArcGIS for Microsoft PowerAutomate

Challenges we ran into

On first pass, we used google's speech to text api to transcribe our calls real-time. What we found was that this package did not integrate well with Twilio for our use case. We had to shift gears, but ultimately found there was a really good built in transcription service provided by Twilio. We ran into trouble accessing editable Open Data resources for shelter locations, resource locations, and other features due. We also had to create a large repository of our own test data, as our project is based on the data ingested to the visualization process, so we did not have access to prepped data.

Accomplishments that we're proud of

Configuring an app that has full functionality at every step of the process was a very exciting accomplishment. We are proud to have interpreted audio from calls into data on a map and extracting the relevant context from each request to deliver a useful response. Being able to collaborate as a team that has members in different offices and create a successful product.

What we learned

• ArcGIS API for Python
• Integrate various products along the ESRI Suite with an open source API.
• ArcGIS for Microsoft PowerAutomate.
• Arcade Expressions in ArcGIS Online.
• Twilio cloud communications sdk for javascript
• Natural Language Processing

What's next for CareLink

Expansion of CareLink’s services largely relies on the integration of more data sources of available resources. We could also implement an LLM to converse with the caller in natural language rather than templated response Due to limitations with Twilio’s free tier, we were unable to utilize SMS messaging. An expansion could include follow ups through SMS and increased versatility when it comes to providing a user with their available resources. We could introduce pagination of resources via SMS, so if a user is looking for options other than those provided, they could have the ability to do so.

Built With

• axios
• express.js
• flask
• node.js
• python
• spacy
• twilio