What impact will generative ML have on software testing?

Generative machine learning has become of increased interest among technologists, the media, academia as well as wider society in recent months. It has supplanted NFTs (non-fungible tokens), the metaverse and the blockchain in the headlines of newspapers and physical space on our social media feed. The reason for why is not hard to find: generative machine learning has incredible implications on content creation, and with it raises a number of interesting questions and dilemmas such as what is or isn't art, the rights of the original author(s) from which the machine learning model learned from, the impact of generative machine learning on creative jobs (which were previously thought of as being immune from the effect of artificial intelligence or AI), grading within the education system as well as misinformation particularly in the online space. These are all "sexy" subjects. However, generative machine learning also has the ability to impact areas not frequently as discussed. One such area is software testing. In this essay, we will look at how software testing will be impacted by generative machine learning. We will first begin by exploring what exactly is generative machine learning and how it works. We will then proceed to explaining what software testing is and why it is important. After this, we will explain how generative machine learning will impact software testing. Then, we move to focus on TestRigor, which is a platform that uses technologies to improve software testing and ensuring the seamlessness of the process from handing off code from software engineers to test engineers. We will then explain its advantages, updates to making specifications executable and how generative machine learning is critical to the process. Finally, we will then conclude. In our conclusion we will express prospects of generative machine learning for software testing and future avenues that the technology can take.

Before delving into generative machine learning, it is important to first distinguish between supervised and unsupervised training in the context of artificial intelligence. Supervised training is where a computer is given labelled data and output data, from which it then "learns" and when given a different piece of information, can then make an appropriate decision. Unsupervised training is when a computer is not given information at all and is then expected to learn from this. Generative machine learning falls into the latter, namely, unsupervised machine learning. Generative machine learning, or more accurately generative models, are given a set of information and are then expected to produce a completely new set of information. With generative machine learning, we can give a model a set (technically called a "corpus") of information and then produce an entirely new set of information: imagine new text from the entire written corpus of Shakespeare, or new paintings from the entire work of Leonardo da Vinci. This is how popular tools such as ChatGPT and DALL-E work and it is has ignited an extensive interest in the area.

Software testing is the process in which we test software to identify bugs, errors as well as ensure the overall quality and performance of the code at hand. Software testing is usually further subdivided into different categories: stress testing, unit testing, regression testing, performance testing, usability testing, to name just a few. Software is usually created by many thousands, if not tens of thousands (and in some cases, hundreds of thousands to upwards several million) of lines of code. In the industry this is known as source lines of code or SLOC. Software is also usually created by multiple teams of people. One problem with many software "pipelines" - pipelines describing the process of producing code and then shipping it off to testing and then publishing it to the wider world - is that as code becomes larger, different teams frequently fall "out of sync" with one another. That is to say, suppose we have version 1 of a particular source code for a given software. This version is then shipped to the testing team to verify and check the code. The team that develops the code will then progress to version 2. Depending on how long the testing period takes, it's entirely possible that the testing team may be stuck on version 1 whilst the development team progresses from version 1 to version 2 to version 3, and so on. This is problematic and defeats the purpose of software testing. As we will see later on in this essay, testRigor aims to deal with this issue.

It has frequently been the case that there has been a lot of focus on generative models and their applications to content creation such as artwork and literature. However, generative models also have applications when it comes to software testing. In the context of software testing, generative models are able to take human-readable input and phrases and take that as instructions to perform a particular task; in some respects, almost as if you are talking to a person. This works by a model being fed a corpus and being trained to carry out a sequence of tasks based on that information. Generative learning and its models will impact software testing in a number of ways ways. The first is that platforms that utilise this form of artificial intelligence can find out problems almost as if they have discovered it "on their own". The second is that software testing platforms that use generative learning can take human input and be able to understand the problem at hand, something that is not able to be done with a platform that doesn't use artificial intelligence. The third is that software testing platforms that use generative learning can use image classification to better understand images on websites, such as the shopping icon indicating that there is an e-commerce functionality behind it that needs to be tested. The fourth is that generative learning can better understand what the experience is of a a user using a particular website.

TestRigor is a platform that enables businesses and other organisations or entities to test their website or app through the use of human language. For example, instead of programming a test using a computer language or framework (for instance, Unit when using Java) a user need only use a human-level language with TestRigor to see how their website works and whether it works as intended. For example, with the platform, you can enter a phrase such as "Click Search" and the platform will be able to understand what this actually means. This is a good integration of artificial intelligence because it combines a field known as natural language processing (otherwise known as NLP) with the platform to understand human queries. One good thing about TestRigor is that it also shows screenshots of tests as they are done, which is helpful from a UI perspective. TestRigor also uses artificial intelligence to help inform a tester of the experience of what a user is from their own point of view, which helps clarify the broad journey to the tester. TestRigor also has facilities for specific forms of tests: for example, TestRigor has regression testing, which is a form of testing that specifically checks for whether a new function or facility breaks a given website. Another interesting feature of TestRigor is that it enables tests to be done irrespective of whether the website is or is not behind a password-protected area, provided that you as the tester provide the relevant details. This is useful because it enables you as a tester to perform tests on a website both as a logged-in user as well as logged-out user.

TestRigor also has other desirable features. It integrates well with Jira, the project tracking software. TestRigor also enables you to test native apps on your desktop.

With a "race" emerging in the field of artificial intelligence, and generative learning being one of the shining jewels, the question becomes where exactly does that put software testing for the future? In my view generative models will be increasingly used in software tests, and there will be a more in-depth knowledge of human queries, such as understanding idioms and slang words that as of present are not readily understood just yet (it is possible that a generative model could be trained on a corpus of Urban Dictionary but that is a story for a different day). Software testing is an integral part of the software development life cycle (SDLC) and generative learning is an interesting dimension to a part of software that end users do not typically see. TestRigor offers an interesting toolkit to help make the software testing phase more fun, productive and easy.

Built With

• n/a