Locately

Enhance business communications
Get more details about locations you read about
Beautiful and easy to use
Get help while writing research papers or essays
Choose from data that's relevant to you and your workflows

Inspiration

We wanted to create an easy to use web browser extension that provides geospatial context to your workflows, right at your finger tips.

What it does

Locately analyzes your web pages and looks for city names in articles, research papers, google searches, and even business communications. We then geo-enrich these locations and provide a popup with useful metrics like demographics, at risk communities, and economics; as well as a map with city boundaries visualized, and a summary from wikipedia.

How we built it

On the frontend...

The user interface of locately is implemented as a browser extension (just supporting Chrome for now). Once installed by an end-user, it works right out-of-the-box thanks to some default settings, allows end-users to set their own preferences, and does some pretty interesting things with the DOM, including traversing all elements to find text, reacting to DOM updates via the MutationObserver API.

The page's text is sent to the backend for some natural language processing (NLP) goodness (more on that later), then returned as recognized place entities. The extension takes over again and matches all these entities with the DOM nodes where they're mentioned, adds some styling to alert users they've been recognized, and waits for user interaction. Finally, when a user interacts with a locately place, the extension talks to Wikipedia and ArcGIS Platform (see below) and constructs a beautiful popover with relevant information.

The backend is pretty rad, too...

Once the backend receives text, it uses the SpaCy library to execute NLP. NLP is a form of deep learning that understands linguistic context and dependencies between words to split text into meaningful tokens, and then names them as real-world objects (e.g., person, date, or place). The trained model leveraged by the backend has learned from conversational phone speech, news articles, blogs and more to complete this task.

From the collection of meaningful tokens identified, just the location-types are isolated. The location tokens are then supplemented with additional spatial information. An open source geocoder, from the geopy library, is used to match location entities with geographic keywords that can be used as inputs to the Geoenrichment API.

Enter ArcGIS Platform...

TL;DR: locately uses the following ArcGIS Platform services, APIs, and tools:

locately uses ArcGIS Platform in a few distinct ways:

After the backend runs the NLP to identify place entities, it uses the Geocoding service to gather more details [view code]
Back on the frontend, the browser extension uses both the Standard geography query and Enrich tasks from the GeoEnrichment service in order to fetch data variables for displaying to the end user. All the heavy lifting here is done by one of our favorite ArcGIS Platform APIs, ArcGIS REST JS. [view code]
Finally, the map users see in the locately popover is built with another ArcGIS Platform API, the ArcGIS API for JavaScript. The JSAPI leverages the geometry returned by the GeoEnrichment service as well as ArcGIS Platform's basemap layer service to show a contextual map to users. Users can even click the map to go directly to that place in the Map Viewer! [view code]

Solution Overview

locately solution overview

Challenges we ran into

Modern web pages load data in unexpected and unpredictable ways, content isn't always available when a chrome extension is loaded. To overcome this challenge we use a new capability called the MutationObserver, this allows us to be notified whenever a section on the page updates its content so we can find city names again.

On the backend, we spent time researching the appropriate models and approaches to solving this kind of problem. We identified the Spacy NLP Library and were able to incorporate that third party library in our GeoProcessing tools. This led to problems during publishing to ArcGIS Enterprise as a GP Service. In the future, this service would be published to a highly scalable service like AWS Lambda or similar to handle the number of requests that our extension would need for real world use.

Accomplishments that we're proud of

The overall polish of the extension, we spent a lot of time trying to make it a joy to use and provide useful information that actually enhances your browsing experience. Being able to use so many tools from the ArcGIS platform together in a way that creates a great end user experience. We're also proud of incorporating a third-party Deep Learning NLP model in a GP tool to complete token classification on place names and geocode those tokens against the ArcGIS Geocoding Service. While we weren't able to publish as a GP Service, we would look to alternative platforms due to the type of traffic this service will ultimately receive.

What we learned

Browser extensions have their own apis and unique challenges when you're trying to support content for any website on the internet; we learned how to create a solution that's robust and has a light touch so we don't break styles or functionality on pages we enhance.