Automation testing is an important part of the software development life cycle. To pick the right tool for your company, it’s crucial to consider the resources, time, and skills of your team at the beginning of the project. This post describes some mobile testing tools that are widely used nowadays.
Since every team’s needs will vary, we’ve sorted the tools alphabetically and by whether they require coding skills or are codeless. So, you can determine which tool will best suit your needs.
By the word “code,” it’s easy to guess that these tools will require actual coding. Identifying locators, creating test configurations, integrating with other tools, and creating tests all are things that are done programmatically. Here are some tools to consider.
Nowadays, Appium is among the most popular open-source mobile testing tools. It supports multiple languages, both iOS and Android operating systems, as well as native, hybrid, and React Native apps. Appium does not need access to source code. As a result, tests are independent from your development project. Another advantage of Appium is that it’s completely free.
Setting up Appium is complex for those who are new to automation. It might take a while to set everything up in order to start working and actually write tests and run them. But the community is huge, which can help you learn it faster and fix any issues on the go. Another disadvantage of Appium is that it’s flaky. This is because Appium is built on top of UIAutomator2 so it can interact with Android. Similarly, it’s built on top of WebDriverAgent to interact with iOS. That’s why it supports both platforms.
Maintaining Appium tests is a challenge. If locators change or the user flow changes, there’s no auto-maintenance or self-healing capabilities. That means that tests will fail and QA engineers will need to manually update locators each time there is a change. Also, QA engineers need to specify the locators manually, so they need to have knowledge about element location strategies.
Finally Appium’s support for cross-platform applications such as React is limited. These applications don’t expose details of the elements when the application is rendered on the mobile device; hence, many times there aren’t unique selectors to rely on to find the elements.
Previously, developers used Espresso for UI tests, but it became popular among automation testers, too. Espresso is an open-source framework developed by Google. So far, Espresso is embedded into Android Studio. So, it won’t take much time to set everything up.
The main advantages of using Espresso are speed and having reliable, stable tests.
On the other hand, the only programming languages Espresso supports are Java and Kotlin. Programming knowledge is required to work with Espresso. It supports only Android native apps. If you have iOS native apps on your project, you will have to have a separate framework in order to automate them.
Similar to Appium, there’s no auto-maintenance for locators. As a result, QA engineers will need to change the locators each time there is a change.
Being more popular nowadays, Espresso can be integrated with widely used CI tools such as Jenkins. You can use emulators built in Android Studio to run your tests. If you prefer to test on real devices, it’s also possible to integrate with popular cloud solutions such as BrowserStack, Sauce Labs, Perfecto, and others.
XCUITest is a native UI testing framework created by Apple and embedded into Xcode. Like Espresso, it has become more popular recently. It supports testing only for UI iOS native apps. XCUITest supports Swift and Objective-C programming languages. You might find it challenging to find staff with the appropriate skill set.
Like Espresso, XCUITest’s execution time is quick, and tests are reliable. As a result, you can save time by not having to investigate failed or flaky tests.
Integration with most CI tools and cloud device labs is available.
When we say “codeless,” you might think these tools don’t use any coding. However, a lot of coding has been done behind the scenes. With intuitive interfaces, codeless tools offer the ability to create tests without knowledge of programming languages. Here’s a list of tools that might serve your needs.
21 is an engineering intelligence platform. It uses Appium commands under the hood. The advantage of this tool is its sophisticated locator system, which is agnostic to the development framework. Unlike Appium, 21 updates locators automatically if a locator change occurs. You actually point to an element and 21 picks up the right locator, even if a unique one doesn’t exist.
21’s auto-maintenance capabilities reduce time significantly, as changes to a single test are autonomously applied to other tests. Users who have used 21 report a 90% reduction in maintenance. Moreover, 21 automatically covers different screen resolutions.
21 has built-in emulators or real devices, so the speed you get from uploading the app to creating and executing a test is measured in minutes. Speaking of authoring, 21 learns your application as you create tests: the functionality of the screens and the navigation. That allows 21 to pre-build tests with very little input. The accuracy those tests are pre-built at is pretty impressive.
Tests are run on real devices that are already integrated in the tool. It’s still possible to integrate 21 with cloud device labs such as Perfecto, Sauce Labs, and Kobiton. And, with the help of shell commands, you can easily integrate 21 with CI such as Jenkins. 21 also recently announced integration to bug reporting tools such as Jira.
There’s no need to know a programming language in order to use 21. The interface is intuitive, so everyone can create scenarios. The tool is robust and supports API calls, location, network conditions, fingerprint testing, and more.
A drawback of 21 can be in places where there’s a need to do something custom, although it does give you the ability to write custom code if you need more flexibility.
TestProject is an open-source tool. It supports web, mobile, and API testing. Fortunately, it is compatible with macOS, Windows, and Linux. Its SDK helps those without programming knowledge create tests. With the help of a test recorder, you can quickly and easily create tests in a few minutes. It has built-in actions for verification, interaction with elements, and more. TestProject does require you to specify locators, which takes time and requires maintenance.
TestProject has integration with BrowserStack and Sauce Labs. It doesn’t have built-in emulators, simulators, or real devices. In order to run tests, you’ll need to connect your real device. Alternatively, you can use an emulator from Android SDK or a simulator from Xcode. Connecting devices to TestProject does require some effort.
Using a plugin, TestProject integrates with Jenkins pretty easily.
However, when using TestProject, you might encounter some issues with methods such as swipe. If you create a test with one screen size with particular coordinates, it might not work on another screen size.
With the help of add-ons, it offers more features to testers. It’s possible to create more add-ons using TestProject SDK. But, while TestProject itself is codeless, programming skills in Java or C# are required in order to work with its SDK.
Ranorex is an “all-in-one” licensed tool used for web, desktop, and mobile testing. It’s worth mentioning that Ranorex is a Windows tool. Mac users might see that as a big disadvantage.
This tool is good for beginners and for those who don’t know scripting. Because it has a recorder, it can be used by manual testers. As a result, the speed of test creation is relatively fast. Additionally, Ranorex has a locators repository. Using Ranorex Spy, you can see organized items in a tree structure. Speaking of locators, Ranorex has smart locator identification. Even if there’s a change to the locator, Ranorex is smart enough to identify it automatically.
Ranorex has integration with widely used CI tools such as Jenkins, Bamboo, and Azure DevOps Server, among others. However, less integration is possible compared with other mobile testing tools. Additionally, Ranorex has integration with cloud providers such as BrowserStack and Sauce Labs. But, similar to its CI integration, the list of available integrations is smaller than other tools. Ranorex also offers integration with Jira and TestRail for those who use these tools for bug tracking and test management, respectively.
There’s no perfect tool that can suit every company or team. It’s important to take into consideration team members’ skill sets, needed platforms, time to automate and maintain tests, and many other factors.
Appium can be a good choice for those who have teams with the skills and resources to do scripting. The tool is cross-platform, which is also a great benefit.
Espresso and XCUITests are fast in test execution. If you have only one platform to support, these are appropriate frameworks that can help you. You can still use Espresso and XCUITest for cross-platform development if there are enough resources to maintain two frameworks instead of one.
21 is a good solution for companies that want to quickly get coverage. Also, it has amazing locator support that helps to reduce a lot of time on test maintenance and supports cross-platform applications.
TestProject and Ranorex can be good for companies that need end-to-end testing covering web, mobile, desktop, and API. There’s no need for programming skills. As a result, it’s easy to archive test coverage by manual QA engineers in a short time frame.
Alona Tupchei is the author of this post. Alona has six years of experience in automation testing and the manual testing of web-based and mobile applications. She’s been working on different projects in the domains of e-commerce, real estate, and airlines. She executes her testing not only from a technical point of view but from the customer’s point of view and believes that usability of a product is as important as functionality