When companies grow, they usually turn to automated testing as a way to save time and money. However, test automation isn’t a onetime effort. The initial task of creating automated tests is only part of the job. Test maintenance is an integral aspect of test automation. Furthermore, it’s important to consider that test maintenance will continuously take time, as there are always changes to code, framework, integration, and other components of your application. In this post, we’ll talk about test maintenance overall, including what it is, how to keep tests healthy, and how to save time on test maintenance.
First, let’s talk about what test maintenance actually is. Maintenance itself is the process of repairing something—in our case, we repair tests. And why do tests need to be repaired?
Well, in a world where companies are increasingly agile, changes come often. Sometimes, code changes are even deployed several times a day. As such, it’s important to keep your tests up to date with the code changes. It’s also important to update your automation framework infrastructure if there are any changes to the tools or third-party libraries that you use. Keeping abreast of these changes is a big part of test maintenance, particularly when you consider the need for regression testing.
If you add new features to your mobile apps, then you need regression testing in place to assure that adding new features didn’t break existing functionality Regression testing has become a crucial part of the software development life cycle. And usually regression tests get automated so you can save time on the day-to-day manual testing of the same functionality, such as the login form, registration, and so on.
The regression test suite usually contains a large amount of tests. It requires test maintenance as well. It’s important to spend as little time as possible on test maintenance and still have the same quality of tests. Tests need to find bugs, not just fail for false reasons.
In an agile world where teams push changes several times every day, most companies adopt continuous integration and continuous delivery. It’s important to include testing in your CI/CD pipeline. In fact, testing can take place after each commit. Running tests after every change can help you identify issues faster.
For QA to understand why tests failed, it’s crucial to integrate robust reporting with CI. For example, if you use Jenkins, you might integrate the Allure reporting plugin. Good reporting will help you identify more quickly if there’s a failure after the latest commit. Without reports, it will be difficult to understand why tests are actually failing.
When you have CI in place, it’s also important that your tests are always up to date. In this case, test maintenance is a priority, as builds won’t be completed if tests fail. Also, if tests are not maintained and updated, bugs might be leaked to other environments or even to production. Luckily, reporting can help you with test maintenance by helping QA engineers determine if there’s a broken test that the team needs to update.
Following best practices in keeping tests healthy will help you reduce maintenance time for mobile apps. It will also help you to keep code clean and readable. Below are some best practices you should follow to decrease the time and effort spent maintaining tests while increasing code quality.
Before you start to write test scripts, it’s important to understand what tests need to be written.
As we all know, it’s impossible to automate everything. That being said, when designing tests to be automated, you should consider which features are used the most, what different input values there are, how much time is spent on manual testing of a particular functionality, and so on.
That’s why proper planning and test design can play an important role in test maintenance.
Each app has several screens. And each of these screens contains different elements. These elements can be reused in different tests. When one element locator is changed, it needs to be updated in all the tests that use it. Maintenance time will take a while in this case because you need to manually go through every test that uses a locator and update it. Imagine the manual workload if you had thousands of tests!
By using a page object model, you update the locator in one place. Once done, tests will automatically use the updated locator. The same applies to methods. You can write one method and reuse it on different test classes. Using a page object model can save you a lot of time on daily test maintenance.
Another point of making tests healthy is to use variables in your code instead of hardcoding values. This can also save time during test creation because you create one method to cover different scenarios. As with a page object model, if a variable is changed, you can update it in one place instead of manually changing multiple classes where the method is used.
When talking about mobile apps, it’s important to have identifiers for the locators that you’ll use in your tests. For iOS apps, and sometimes for Android, XPath is used to identify the element instead of an ID. QA engineers can still use XPath in their tests; however, a recommended best practice is to have IDs such as a class name for the element.
Why is an ID preferred to XPath?
First of all, using XPath increases test runtime. When using XPath, the system goes through the entire DOM tree to find the element. But with an ID, the system finds the element much faster.
Second, tests will break more often if there’s any change to the DOM tree, even when the locator itself isn’t changed. Having IDs is a better way to locate the element. That’s why it’s important to collaborate with developers—so that they can provide IDs for elements in use. Doing so will help reduce precious time on test execution as well.
Another way to have tests without false failures is to wait on elements before doing any action. It’s very common to see false errors as a result of some timeout issue where the element didn’t appear when the action was being performed. With wait, this problem will go away. As a result, tests will be more stable.
Even following best practices, it’s still important to have code checked by another person. With the help of a reviewer, inconsistencies, missing scenarios, or simply suggestions to improve code can be addressed.
In today’s mobile world, there are thousands of types of devices people use every day. If you have a mobile app, it should work on any device and any screen resolution. Usually, tests run on multiple devices to cover large, standard, and small screens to support different resolutions. Some elements are visible on a large screen, but not on a small screen. It becomes challenging to get the necessary coverage in tests. However, it’s important to cover different sizes so that end users won’t face any issues while using the app.
With the help of cloud device tools, test coverage can easily be achieved. Cloud device tools spare you from having to set up the whole infrastructure in your organization. They also provide easy integration that you need to do only once, which will also save time. As such, you won’t need to maintain your device farm all the time to have devices up to date.
Test maintenance takes time, so it’s important to consider it when planning projects. If there are significant changes to the code, it might take most of your time just to maintain your tests. It’s better to plan for it in advance to avoid any surprises on the way.
21 can significantly reduce overall time spent on test maintenance. 21 has a smart locators algorithm that chooses the best match for the element locator regardless of its position on the screen. If locators are changed by developers, tests will still run without a need to change them. This feature is a huge time-saver. You do not need to manage locators or plug unique IDs.
Another advantage 21 provides is auto-maintenance of tests. With AI technology, 21 can analyze your tests, learn your application, and then adjust tests in case of changes. Even if the screen order is changed, 21 can detect it. You can reduce about 90% of your test maintenance time.
Finally 21 has built in page models: All tests point to a single inventory of screes so that you only need to update a screen once and it will get updated on all tests. For more information about how to update tests to changes in the application with 21 click here.
As you can see, only writing automated tests is not enough. Test maintenance is a mandatory task in the day-to-day responsibilities of a QA engineer. And the more tests you have, the more complex test maintenance can be. As a result, maintaining tests will take time from your workweek. When planning test automation for your project, it’s important to consider test maintenance.
By following the best practices we’ve shared here, you will be able to keep your tests healthy and reduce time spent on test maintenance. However, it’s not possible to forgo test maintenance entirely. But regardless of time spent on maintenance, having test automation in place will pay off. And if you want to further reduce the time you spend maintaining tests for your mobile apps, give 21 a try.
This post was written by Alona Tupchei. Alona has six years of experience in automation testing and the manual testing of web-based and mobile applications. She’s been working on different projects in the domains of e-commerce, real estate, and airlines. She executes her testing not only from a technical point of view but from the customer’s point of view and believes that usability of a product is as important as functionality.