With more than two dozen PRIs providing voice connectivity to contact centers and branch stores, we had no way of knowing when we needed more voice capacity to support an increase of inbound call volume. The manual way of querying voice gateways was not satisfactory. We also needed a way to automatically change call routing based on business conditions and to roll back to our normal call routing when appropriate
How it works
We ingest Cisco Voice gateway data into Splunk Cloud in near real time and monitor for PRI channel utilization using Splunk queries and macros. Alerts tell us when we are in warning or critical states and a custom Python script using the Splunk API and the Cisco AXL API allows us to change call routing on the fly.
Challenges I ran into
No one had ever done anything like this and no one knew specifically how to get the data out of the voice gateways. Cisco produces multiple event records per channel connection so we have to develop a primary key which allowed us to count distinct channels in use. There is poor documentation re: Python and the Cisco AXL API which caused development delays. We had only been a splunk customer for approximately 3 months before I started this project so there was a lot to learn.
Accomplishments that I'm proud of
Doing something that's never done before. Several Cisco experts told me what I wanted to do wasn't possible. Splunk helped me prove them wrong. Innovation is a AAA core belief and Splunk enabled me to be innovative in a way I couldn't image.
What I learned
Learning Splunk - how to use the query language and how to leverage the Splunk community to solve problems Learning the Splunk API Learning Python Learning the Cisco AXM API
What's next for PRI Capacity
Better integration with Cisco AXL API to automate advanced call routing Creating PRI Trunk groups in Splunk Adding support for non PRI voice service (SIP and POTS)