We’re living in a mobile-driven world. It’s very rare to see someone without a mobile phone. Facing this reality, most companies that have web applications are also considering adding mobile app support to increase traffic and engagement. In fact, you can find an app for any domain nowadays, be it banking, networking, social media, cooking, fitness, or another industry.
Once a company implements an app—whether it’s web or mobile—there’s always a need for quality assurance (QA) to make sure your app functions properly and meets end users’ expectations. QA is an important phase in the software development life cycle (SDLC). It ensures that end users won’t see bugs within the app and will enjoy using the app, coming back to it again and again. In this post, we’ll talk about mobile QA and how to make it more efficient.
It’s obvious from its name that mobile QA is QA performed on mobile apps. These apps can be native, React Native, web apps, Xamarin, or hybrid apps.
Mobile testing happens at different stages of the SDLC. To achieve better quality in your mobile app, it’s important to perform different types of tests. Let’s look at the most important types of tests that you can use to help guarantee your app’s quality.
QA engineers perform functional testing in order to make sure the app functions properly. Additionally, QA performs functional testing against requirements (if they exist), which can help you make sure the app meets business criteria, too. An app can perform well on one device (or OS version) but not on another which is why testing the functionality on multiple devices is critical.
Mobile phones have different screen sizes. QA engineers should always make sure the app has the same UI elements on different screen sizes without any elements overlapping. All end users should have the same user experience (UX) and UI. If the UI is broken, the user may churn and permanently uninstall it.
Specifically with mobile apps, it’s important to assure that users can install and update the app.
In the case of a website, users simply enter the URL and are able to use the website from any browser. But a mobile app’s functionality is different in two aspects: First, just like desktop applications It’s dependent on the hardware of the device as well as the OS. Second, the end user needs to install the app from the app store before he can experience it.
If there’s something wrong with the mobile app, a user might not eve be able to install it on their device. You need to test correct installation because, in order to use the app, the end user will first need to install it from the app store.
Another important thing to test is whether you’re able to update the app correctly. Testing this is similar to testing installation. Sometimes, a user updating the application from the store causes the app to crash or behave incorrectly. This can happen, for example, because components of the new version are incompatible with components of the old version. Testing to make sure the app receives updates correctly can spare the end user from facing these issues.
Performance testing is key to understanding the user experience under different conditions. Usually, the QA team automates performance tests using different tools such as JMeter or LoadRunner. With the help of performance testing, you can make sure your mobile app can handle high user traffic, among other demands. It’s important to verify how your mobile app behaves on an older device with weaker hardware. Usually, an app performs more slowly and might crash on older device versions. Checking this can ensure a good UX for customers who are using older devices.
It’s always important to make sure new features or bug fixes didn’t break previously working functionalities. Regression testing is usually automated; however, manual regression is still taking place in the SDLC. Regardless of how you do your regression testing, it’s important to perform it after any bug fixes, configuration changes, deployment of new features, etc. As an example, you can retest the whole application before releasing to production or migrating data. You can also perform regression testing after fixing a small bug—this would include retesting the components that might have been affected by the fix. It’s important to perform regression testing on the app before it’s released to the store, as this can prevent surprises in production.
Another way to get good feedback from users regarding the app’s usability is to perform A/B testing. Taking this approach can help you understand what the end users want and if end users need features you are developing.
The idea of A/B testing is to experiment with different designs or functionalities. Using metrics, A/B testing enables you to see if the new designs or functionalities are enabling users to have better interactions with the app. These metrics can be set based on your tests. Depending on how many tests you want to perform, you divide users by groups and analyze each group. If metrics are better with group A, the QA team can consider that deployment to be a successful variant and implement it for all users.
Last but not least, having beta testing in place can help prevent bugs from making it into production. For iOS apps, beta versions of your mobile app can be distributed through TestFlight. For Android, you can beta test your app using the Google Play store. You can test the beta version of your app internally within your company, or test it with external users. The goal is to see if there are any bugs you didn’t find or if crashes occur. Additionally, beta testers can make sure the app meets their needs when using it.
Below is some additional mobile-specific guidance that you should consider in order for your mobile QA processes to successfully guarantee your product’s quality.
When testing mobile apps, it’s important to consider the variety of devices on the market. So, when planning mobile app testing, it’s a must to check as many devices as you can—and to check devices with different screen resolutions. Device farm or cloud device solutions can help you get test coverage for the most commonly used devices, as well as different screen resolutions and OS versions. Usually, you should try to cover the last two versions of iOS and the last five versions of Android. However, it’s up to your QA team to decide what to cover based on the users’ usage.
Having to get test coverage for multiple devices will incur costs to obtain those devices. As such, you should know when it’s necessary to test on a real device as opposed to a device simulator.
Xcode and Android Studio provide some virtual device simulators that you can use to test your app. You might think using virtual devices is a good way to not buy many real devices. While virtual devices are helpful to quickly check something or debug the app, when it comes to end user experience, it’s preferred to use real devices.
Depending on the app, there might be a lot of interaction with the phone’s hardware—for example, location, network connection, sound, video, and so on. That’s why it’s important to see how your app behaves with the hardware itself.
That being said, virtual devices are good to perform sanity with specific tests that have to do with hardware, being tests on real devices. To perform tests on various real devices, you can use cloud device providers such as BrowserStack, Perfecto, Sauce Labs, and others. Using these tools can save you the cost of buying real devices in the future.
Another important point is to cover all tests on all supported platforms. Usually, apps are developed for iOS and Android together. As such, it’s important to understand the platform-specific components. Some components are developed differently for these platforms.
Consistency is another point to consider when talking about cross-platform apps. You should check that all features are the same between platforms. Additionally, it’s good to check the position of elements to ensure the same experience for users. However, there might be a slight difference in the case of native apps due to the native components of these apps.
Every year, Apple and Google release multiple minor and major versions. Usually, a beta version of each comes earlier so that companies can adjust their apps to the new OS. By testing your app against an OS’s beta version, you’ll identify and fix bugs before the OS is released globally.
Unlike with web testing, when testing mobile apps, connection to a mobile network, absence of WiFi, and slow internet are important aspects to pay attention to.
The beauty of testing mobile apps is that you can actually analyze what end users are doing with your app. That’s where tracking comes into play. There are several tools that you can use for tracking, such as Firebase, Google Analytics, or AppsFlyer. Usually, developers will implement tracking. Accordingly, your QA team needs to test that the tracking is being implemented correctly. This is typically done by using the app and seeing that events are displayed correctly when interacting with the app.
Another mobile-specific functionality is push notifications. Many apps use them to interact with a user or notify a user about events related to their app. Regardless of what technology you use for push notifications, it’s still important to test their functionality. First of all, when testing push notifications, you should check if the notification was sent and appeared on the phone. Second, it’s important to check the content of the push notifications to avoid any grammatical errors or incorrect content in the push notification itself.
You can see that mobile apps have their own specifics to consider. Apart from the usual testing your QA engineers do—such as regression, performance, or functional testing—it’s important to consider the variety of devices you need to check to ensure all of your end users will use the app in the same way. UX and UI play a crucial role in mobile QA, as the limitations of screen space on a mobile device make it critical that all necessary functionalities are available and intuitive for users to interact with.
Additionally, global OS updates can break your app. Paying attention to OS updates and trying their beta versions with your app can reduce the risk of crashes or incorrect behavior due to incompatibility between the app and the updated OS.
On that note, it’s also always good to invite beta testers to find any other inconsistencies in your app. A/B testing can also help you understand your end users’ behavior and if they actually need the features you provide.
These practices can help you achieve exceptional user experience.
This post was written by Alona Tupchei. Alona has six years of experience in automation testing and the manual testing of web-based and mobile applications. She’s been working on different projects in the domains of e-commerce, real estate, and airlines. She executes her testing not only from a technical point of view but from the customer’s point of view and believes that usability of a product is as important as functionality.