S.W.A.T. - Scanning and Website Availability Tool
· Automate daily time intensive testing of website page statuses by navigating to webpages via browsers to simulate end-user interaction
· Provide daily confirmation that the site pages are up and running without page errors.
· Reads the URL’s list from URL List file
· Navigates to the URL
· Scrapes the HTML on the page then finds all HREF’s on the page.
· Navigates to all internal HREF’s and repeats scraping process until all internal HREF’s have been scraped.
· Determines whether a page has any server errors and if not then the page is passed.
· An additional scrape is done to list all form input elements on a page if it exists, to determine if all the form elements on the page are displaying correctly.
· Summary of the page status (Number of Pages Passed or Failed, Number of Input Form Elements and Number of Pages found).
· Elements per Page that include the element type and element name.
· Site Map of Webpages.
· Determine if a page requires login credentials and completes login process.
· Incorporate multi-tenant processing to improve efficiencies and run times.
· Incorporate more browser types beyond Chrome, including IE, Edge, Safari, and Firefox
· Identify additional website elements including document types (word, pdf, etc.), images, videos.
· Introduce a machine learning component that can perform analysis to determine correct CSS rendering.
Machine Learning Component (Future)
· Currently training a deep learning neural network using an unsupervised learning model to identify what a correctly displayed website looks like.
· Future steps will include deep analysis of the encoded representation learnt by the deep learning model. For instance, we can compress an image of a webpage to a representation understood by the model and use clustering algorithms to make a distinction not only between erroneous and normal webpages, but other features of a website such as text and images.
· This can be integrated into a custom UiPath activity that would query the AI to determine if a webpage rendered correctly as well as what level of certainty.