Evolution of Software Testing towards AI/ML
- (1980 – 1990) Manual Testing in Waterfall and V model
- (1990 – 2000) Automation Tools (Ex: QTP)
- (2000 – 2010) More Robust Automation Tools + Open Source Frameworks – Agile Methodology (faster release cycles)
- (2010 – 2018) DevOps, Continuous Testing, Continuous Integration/Continuous Delivery
- (The FUTURE) Autonomous Testing, ML and AI that requires collaborative & smart testing
There is no doubt that the test automation space is undergoing a lot of transformations. Machine Learning (ML), Deep Learning, and Artificial Intelligence (AI) are being leveraged more and more as part of the testing activities.
While the space is still growing from a maturity standpoint, it is a great time for practitioners (developers and test engineers) to start understanding the key use cases and advantages of ML/AI on the overall test automation.
Organizations today are trying to leverage the concepts of Machine Learning and Artificial Intelligence to help solve and remediate issues around the following:
- Object identification and handling of dynamic object locators
- The heart of test automation is object recognition using the right set of properties – ID, XPath, Name or CSS. The test fails even for a minor tweak to the properties made by the developer. Manual identification of the objects takes more time and even the maintenance and constant changes become cumbersome. In recent times, test automation tools apply self-healing techniques using AI and ML algorithms to dynamically sense the property changes and the object repository is self healed.
- Based on available data, make quality-related decisions
- AI/ML can help us select the tests that have to be run for any new changes to the application thereby executing test cases based on high-risk segments. This helps in assisting us better with decision making.
- Not limited to functional testing
- Tools that support functional testing have limitations to test other non-functional requirements like usability, security, etc. AI allows the automation of visual validation of the application to test whether each UI element matches the design requirements of shape, color, size and position and is not overlapping with other UI elements. Penetration testing is another example where the information gathered by AI tests and the analysis make it easier for the testing team to perform an effective security testing.
Not all ML/AI solutions offer codeless test authoring, there are solutions like Testim.io that offer AI capabilities for test artifacts authoring, test execution and maintenance of automation scripts.
The following comparison table shows some differences in the workflow and skills that are required to use ML/AI codeless (in most cases) test automation tools and traditional test frameworks like Selenium, Appium, Cucumber, Espresso, etc.
As observed in the above table, there isn’t one solution that is better than the other, but rather a set of requirements a practitioner or a team needs to address prior to either adapting to a new model or holding onto the current one or combining both.
To highlight a few benefits for codeless testing we can look at the speed of test authoring, lower bar from a skillset perspective, more stable test scenarios due to self-healing code and object identification, etc.
To highlight benefits from the traditional test automation space, we can look at mature space, better integrations and open-source hooks, a higher skill set that enables greater coding flexibility, advanced testing types, and support for all kinds of apps.
Conclusion
There is no decision or bottom line other than becoming familiar and aware of market transformations, understanding the use cases that slow you down, and seeing which tools in the AI/ML space can be used to aid these pains.