QA engineers may face different challenges when it comes to testing different products. For websites, the challenges can arise from handling multiple browsers, external services, etc. For mobile, the challenges are more complex. That’s because, apart from the functionality of the application itself, you need to consider hardware features, the huge variety of devices to be tested, and other mobile-specific factors. So, in this post, we’ll be talking about the challenges for manual mobile testing.
One of the main challenges in mobile applications testing is to make sure the end user will have the same experience regardless of their device’s model, OS version, or screen size. According to an OpenSignal report, there were about 24,000 Android devices on the market in 2015. And with each year, that number is growing rapidly. There’s less fragmentation among iOS devices; however, it’s still important to consider testing on different types of iOS devices.
There are countless combinations of features you need to account for, including
Let’s look at a few of those aspects in more detail.
Another challenge for manual mobile testing is screen size, which always differs from device to device. Whereas your app might look and function perfectly on a larger screen, on smaller screens, not all elements of your mobile app might be visible, or they could be overlapped by other elements. That affects not only the appearance but also the usability of the mobile application. Considering screen size in testing is must.
Another challenge is making your app support different platforms. The most popular platforms nowadays are iOS and Android. Unlike with websites, where you can just paste the link into any browser, with mobile applications you actually need to install the application separately for iOS and Android in case of native or React Native apps. In addition to testing each platform separately, it’s important to verify that the app is consistent between different platforms. The end user should have the same experience, regardless of platform.
Having a different OS version gives complexity to testing itself, as apps can behave differently on different versions. With Android’s or iOS’s major releases, some components that were developed for previous versions might behave differently on the newest version and vice versa. That’s why QA engineers can’t simply test the application on one OS version—they must make sure it still works the same way on other versions. Usually, each company defines what versions of each platform to support.
When testing mobile applications, the main difference from testing websites is the hardware itself. Most mobile applications interact with the mobile phone’s hardware. It gives users the ability to use maps, receive messages, make calls, and interact with other features. In addition to testing the mobile app itself, QA engineers also have to test the hardware. Let’s take a look at some of the most common features you should test.
If your application uses location (for example, maps), then it’s mandatory to ensure it works as it should. The challenge comes when you need to test a particular location using GPS that is different from the present location. With the help of tools such as emulators, it’s possible to “fake” the location coordinates in order to test and verify how the application behaves. However, it’s always better to test real world scenarios that are the same as what users will do when using the application.
Push notifications are used to communicate with a user in case of changes or important updates to the mobile application. Usually, third-party services such as Firebase or AppsFlyer are used to send push notifications on mobile applications. In addition to that, it’s important that the user enables a permission on the device itself so that they receive the push notifications.
When the network is slow, it’s possible to see the application crash or have unexpected behavior. The challenge is to test how an application behaves under slow network coverage and different types of networks, such as 2G, 3G, 4G, Edge, etc. With the help of tools such as emulators, you can fake the network and make sure the application still works as expected.
Web application testing is limited to mouse clicks and keyboard strokes. Mobile offers a much deeper interaction with the device sensors: pitch, swipe, force touch, fingerprints, and so on. Testing gestures is challenging enough when manually testing the application. Automating that adds a whole new level of complexity. I remember that when fingerprint was just introduced, we literally had a robotic finger with a fingerprint mold that would be triggered to test that gesture. Today, you don’t need a robotic finger, but still, most gestures are complex to test, especially over different devices.
Nowadays, a lot of mobile applications use multi-language support. Also, it’s possible to change the language on the mobile device itself. Localization can be challenging because QA engineers need to make sure the application behaves the same way with different languages, regardless of whether the language was changed within the application itself or on device.
Apart from content, some languages have different rules of writing. For example, when a device’s language is switched to Arabic, everything needs to be written from right to left, and all components are displayed right to left as well.
Compared to the website, there is a challenge to place all mandatory information and functionality into the mobile application considering the screen size. If on a website we have plenty of space to place all our functionality, on mobile applications this space is limited. It is a challenge to place all needed features in the mobile screen and also make sure the user experience is the same.
Another tricky part is to make sure the mobile application does not consume a lot of CPU cycles in order to continue having the same performance of the application itself and device. Being on background application can also consume a lot of power that will lead to the battery being uncharged faster. With the help of external libraries like LeakCanary it is possible to identify when leakage is happening.
Mobile testing is more complex than testing websites. It has its own challenges. The biggest challenge here can be the variety of devices. There are thousands of devices that have different screen size, OS version, model and platform type. All that brings a big amount of possible combinations users might have. To test all these combinations is not physically possible in a decent period of time. That’s why it’s important to plan testing to make sure most of it can be covered. Possible with a help of device cloud solutions where you can choose the phone type, OS version, screen size etc.
Testing possible OS versions might increase testing time. Usually companies determine how many versions they want to support that makes it slightly easier for testers. They won’t need to test all existing versions in this case.
Device hardware is closely connected to the application itself so such features like maps, calls, push notifications need to be tested not only in the application itself but also on the device level.
Localization can become a challenge when testing mobile applications supporting different languages. It is time consuming to make sure a mobile application behaves the same way regardless of the languages selected.
This post was written by Alona Tupchei. Alona has six years of experience in automation testing and the manual testing of web-based and mobile applications. She’s been working on different projects in the domains of e-commerce, real estate, and airlines. She executes her testing not only from a technical point of view but from the customer’s point of view and believes that usability of a product is as important as functionality.