Cluster Close Up
Let web traffic intelligently inform you about your next market move. Everyone gets so many people visiting their websites let's do something with that data.
The main idea behind IP Informer is to make it easy for merchants to combine their Capital One transaction information with their web traffic. By finding an area where web traffic is high but sales are lacking it will make targeted advertising high effective and new lead generation more economically.
Structure of the Project
I have broken the project up in Django services allowing easy import and upkeep of data.
End User Setup for their Site
- User adds a single decorator to their existing Django views.
@record_ip def index_view(request): pass
The decorator tracks uses are everyone access the site.
Note: The record_ip only logs ips initially. Services will batch the reverse lookup later so ensures has no decease in web site speed.
General Data Gathering
I pull data from several sources. They are outlined below:
- Transactions and Merchants from your history are pulled from Capital Ones API's. This is done with the
merchantsservice. (Additionally, I generated demo merchant relationships.)
- Each merchant from your transactions is reverse looked up using latitude and longitude to determine the FIPS county code. This is done with the
- Once each merchant is mapped to a FIP additional FIBS data is added from US Census data. This is done with the
ML and Analysis (all the fun stuff!)
I use two main techniques to identify potential market leads and marketing targets: dbscan clustering and multi. variable nearest neighbors. Additionally, I leverage MatterMark for local companies once cluster locations are determined. Below is a walk through of the data flow:
- Transactions and Traffic is difference based on linear distance computed from latitude and longitude
- Points are plotted and clustered.
- Clusters are scored in terms of similarity to the merchants past transactions demographics based on FIBS data
- Highly rated clusters are displayed and use can select of their choice.
- Companies in the cluster area with similar tags to the merchants past transactions are gathered from MatterMark along with proper information for contact and promotion.
List of services with a brief description
Pulls all merchants associated with your account from Capital One and into the local database.
Computes a score for how close a FIB is to the market demographics of your past transactions. Does so through normalization and a linear combination.
Maps latitudes and longitudes to FIBS. This is done so it can be used with US Census data.
Imports and maps US Census data to each FIB in local database.
Runs FIBS and tags through MatterMark to gather related businesses in the cluster area.
Demo Seeding Services
Sets up the main account. This is what an ensusers account would look like.
I don't have a long time to monitor people visiting a site so I simulated some traffic.
Wanted more transactions than could pull from API. I simulated some as well.
Creates traffic for each transaction as the people must have had a traffic record when they checked out.
Merchants tended to be within a very small radius. I provide some variation to the dataset I scattered them out.
Initial traffic was too tightly bound. I added more variation.
What's next for Capital One IP Informer
I want to provide even greater isolation from the underlaying Django. Preferably in a separate package for install.
It was a tight race to get everything useable and working. I soloed this and it certainly kept the 24 hours busy.