Mulexa-know your network's traffic better with ease

Mulexa gives snapshot of API traffic through anypoint platform's report to presents them on UI & is also accessible through Amazon Alexa. Core logic behind this cool product is anypoint platform.

Comment

Inspiration

Analysis of traffic to mule APIs is a key factor in developing great digital experience for an organization. Analytics section of API manager in anypoint platform provides a great setup to create custom dashboards and reports for analysis. They're great but needs someone who is familiar with report structure, dashboard format to use them to the fullest. Reports are CSVs with huge data and need manual analysis. They have a little limitation too - 20,000 events per request. I have myself ran into sufferings because of this limitation and spending enough time on manually analyzing the report by downloading them everytime from platform.

Mulexa started off as a way to give my APIs a voice through the Amazon Alexa services. I build a skill in alexa developer console and tried to hit mule APIs and made alexa to Say the response. This was a new experience as I had only seen mule running behind the curtains. So hearing mule response was possible but I thought of giving a visual appeal too using mule. I'm not a UI developer but I figured out ways to render HTML pages through mule. BOOM There I had an idea of making a medium to see and hear what mule had to offer.

What it does

Mulexa would find its place in any industry which uses mulesoft because it analyses traffic coming to mule APIs

Mulexa is a 2 way solution to analyze traffic to mule APIs.

  1. UI through HTML engine
  2. Through voice commands - Alexa

The traditional 3-layer diagram followed in its implementation is: image

Public portal URL

The 2 experience level APIs use this source of information and other tools to return response in the form of UI(HTML) or speech response (Alexa). It gives user friendly data (after transforming CSV from events report) in terms of numbers and charts which are easy to comprehend and analyze.

Other features included are as Follows:

  1. UI - Filtering results based on time interval
  2. UI - Downloading the Report in CSV format of specified duration
  3. UI - Downloading the full Report in CSV
  4. UI - Parameters like average response time, request count per API, request count per consumer, etc.
  5. UI - Creating a ticket in issue management platforms like Salesforce (Case) and ServiceNow (incident) (proposed)
  6. UI - SFDC Cases retrieval
  7. UI - Receive offline version of report PDF through email/WhatsApp
  8. UI - Uploading a CSV from local directory and see dashboard for data in it
  9. Alexa - Snapshot of traffic to APIs
  10. Get report on WhatsApp as PDF
  11. Count of cases in SFDC for different values of status

How I built it

Following platforms are used:

  1. Anypoint platform - trial account to deploy app, call analytics API
  2. SFDC trial account to create/get cases
  3. Twilio trial account to send WhatsApp message
  4. api2pdf trial account - to convert html to PDF used to send report through email/WhatsApp
  5. Googlecharts, quickchart to draw bar charts and pie chart in the project
  6. api.qrserver to generate QRs on the UI

Data Source

The CSV from events API is the core source of information to the project. At startup, report API is called for last 24 hours and saved in mule app's directory. In every 15 minutes, data for last 30 minutes is retrieved and unique entries from last file are identified and the file is updated with latest information (content of last file + traffic of last 15 minutes). Object store is used to store Authentication header (needed to call platform API) so that login API is not called everytime. Upon expiration of Bearer token, it is retrieved again.

image

UI

The font color has been made in sync with mulesoft's brand color to make it look familiar to muleys :) Hackathon logo is on every page with hyperlink to the dashboard page

Parse template module is used to parse HTML scripts to show UI bearing data from the report. Following screens show the implementation Home Page

image

Dashboard Page The dashboard is an amalgamation of data in tabular and graphical format derived from CSV and transformed accordingly. Duration query parameter is created for the dashboard page to filter results for desired duration Dashboard page

image

There are 2 QR codes to browse dashboard on device post scanning it and to download an offline version to device that scanned it respectively QR to see dashboard internally calls the same page URL with same parameters in mobile device

image

QR to download offline version calls: Download API

image

Download icons are use to divide actual CSV to user's device for specified duration and full CSV respectively. GET - /api/dashboard/report/export is called with optional parameter duration based on use-case Download API

The following image shows CSV being downloaded

image

Basic parameters and their vales in table giving an overview

image

Pie charts showing success vs failures and split between errors encountered

image Count of success and failures from the report through dataweave transformations and use of google charts and html chart resulted in above chart

Bar graph showing average response time of APIs being called is prepared by getting average response time from report grouped by method(verb) and URL(path) in dataweave to feed parameter for quickchart.io to create chart as an image

image

Tables to show top 3 consumers (Application) and top 3 APIs(in terms of request count) - again transformation of data from CSV and creating HTML table. HTML chart created for all consumers and traffic from them & Number of requests from all APIs

image

As mentioned earlier, the poll runs every 15 minutes, to refresh data before the next poll, mule logo as button can be clicked that calls GET: /api/dashboard/report/import: http://mule-hackathon-2k21.us-e2.cloudhub.io/api/dashboard/report/import

If the user has any concern, he/she can raise a SFDC case. Clicking on the Salesforce logo will bring pop up form to fill that calls POST:/api/dashboard/issues/sfdc Create Case API link

image

image

Table showing recent 5 cases sorted on last modified

image

Hyperlink to More SFDC cases takes user to page showing SFDC cases filtered by cases-GET:/api/dashboard/issues/sfdc?status={sfdc_status} http://mule-hackathon-2k21.us-e2.cloudhub.io/api/dashboard/issues/sfdc?status=Closed

image

Receive report section has icons of email and WhatsApp that call GET: /api/dashboard/notify/email?duration=24h and /api/dashboard/notify/whatsapp?duration=24h respectively. These APIs prepare an offline version of the dashboard in HTML, converts them to PDF through api2pdf platform and then sends as email (email connector) and WhatsApp (through Twilio REST API)respectively

Email - http://mule-hackathon-2k21.us-e2.cloudhub.io/api/dashboard/notify/email?duration=24h

Whatsapp - http://mule-hackathon-2k21.us-e2.cloudhub.io/api/dashboard/notify/whatsapp?duration=24h

Received email

image

Received WhatsApp message

image

Last feature on this page enables user to upload their own CSV in the same format as that of events API and hit GO. This calls POST: /api/dashboard/import with mime-type multipart/form-data and mule API reads CSV, prepares payload for HTML UI of similar format.

image

Alexa

Working with Alexa development kit was pretty interesting and different that mule. Skill, intents, slots have to be created for alexa to understand the commands in a skill.

image

Current features through alexa are: Getting snapshot of requests This calls GET:/api/alexa/snapshot?duration={} which in turn reads the CSV, parses it and returns basic detail like - number of requests, success percentage, average response time, etc. Alexa-snapshot API

Following screen shows the request and response of the intent:

image

Get latest cases count based on status The count is retrieved from response of previous call, mule response is saved in session attributes and is further used.

image

image

Challenges I ran into

  1. Working on UI was little tricky as I did not have any experience on UI side.
  2. Creating charts through various platforms with data from CSV and fitting them correctly in UI

Accomplishments that we're proud of

  1. Mulexa will certainly help me and my time in better analysis of our application network
  2. Designing UI from a user perspective and working on it in code
  3. I can now 'see' and 'hear' what mule can do (through UI and alexa)
  4. Finishing this up in the festival season ;)

What I learned

A lot!!! this section may have few points, but the learning has been immense

  1. My speed on dataweave scripts has certainly increased because of transformations done here
  2. New platforms like quickchart, api2pdf, twilio, alexa
  3. Thinking out of the box to get more functionalities in the product

What's next for Mulexa-know your network's traffic better with ease

I think it is just a beginning, the potential is unlimited. Things on top of my head that can still be done here are:

  1. Creating case from Alexa
  2. Report issue on other platforms too
  3. Use a cloud based file storage system so that mule runtime does not have a bulky CSV, coz it'll eventually slow things down :/
Share this project:

Updates