We are a team of test automation center of excellence, and it’s time for our craft to grow up. Today everyone is moving towards automation, but have you ever thought about the time spent on analyzing the execution reports? Yes, if the execution takes 1 hour, then after then analysis of reports takes 3-4 hours.
What it does
The proposed solution helps in creating a centralized live product functional health dashboard system, which helps the team to track the functional health of the product and make release decisions. It uses AI technology to auto-analyze the reports.
How we built it
ReportPortal is an OpenSource tool, that is built using microservice architecture. We have built various REST APIs corresponding to different modules of the ReportPortal, which is built using Client-Server-Agent architecture. We have built clients and agents, Client interacts with REST API based on the language binding and agent interacts with the framework binding like Java-Junit, Java-TestNG, Python-RobotFramework, Python-PyTest and many more. This Client-Server-Agent architecture of ReportPortal helps in displaying live execution reports of the test execution. ReportPortal uses AI, using NLP and Elasticsearch, for the auto-analysis of the execution reports.
Challenges we ran into
For Test Automation:
- No centralized live dashboard system.
- No Live Execution reports
- Test Execution creates different reports when launched from multiple VMs / docker containers / K. Clusters.
- Execution reports are analyzed manually and it's time-consuming.
Accomplishments that we're proud of
- Create a centralized live product functional health dashboard system.
- Easy to Integrate with any test framework
- Aggregate all results and info like test history, logs, attachments in one place.
- Auto Analyze the test failures using Artificial intelligence on collected historical data
- Receive analytics and visualize your statistics and metrics in a real-time centralized dashboard
- Access your test automation results on any device
- Integrate with JIRA systems
- Configurable with GIT active directory and LDAP authorization systems
- Creating and maintaining dashboard for Multiple projects
- Provide role-based access to all members
- Email notification and Alert configuration
- Track release readiness
- Winner of Bangalore city hackathon under “Best QA/Process/CSR Idea”
What we learned
I gained confidence, inspiration, and motivation. I learned about how to set MongoDB Database, how the microservices architecture works, how to interact with multiple different APIs using different technologies. It involved a lot of growth, a lot of struggle and truly a lot of learning.
What's next for NextGen Autocrats
Based on the proposed solution, we would train and onboard all the product teams to implement ReportPortal, which would help the teams to see the live execution reports. During any environment failures, they could raise JIRA instead of waiting for complete execution to finish. This would help the team to track the functional health and track the release readiness of the product. And this will reduce the manual test report analyses using AI.