Embedding mobile compatibility testing into a long-term development roadmap

Mobile compatibility testing for a complex eCommerce platform

The client: A leading retailer
Industry: Retail
Technologies: AngularJS
Ten10 services: Compatibility testing

The client

A leading retailer that has a presence throughout the UK through supermarkets and via an online store. They have an increasingly mobile-focussed customer base and was looking to expand their digital services to keep pace with competitors.

The client has a complex eCommerce platform that  allows consumers to purchase a range of groceries and other goods. The application prior to our testing only included a traditional desktop view of the site, restricting the functionality and usability of the site when accessed from mobile devices.

The client had limited mobile compatibility testing experience and resource in-house. Their aim was to convert the core functionality on the web application to a responsive design with a staged rollout required over a number of releases.

Compatibility testing challenges

The client was upgrading their existing eCommerce product to an AngularJS site; converting the core functionality on the web application to be responsively designed as well as integrating the up and down stream systems.

The rollout of the updated site was taking place over a number of releases spanning several months, with the rollout following a logical order, for example, the product pages being made responsive before the basket and checkout pages.

Some mobile compatibility testing was being performed, but it tended to be ad-hoc and carried out on the personal devices owned by the development, product and ops teams. Whilst this found some defects, structured reporting of the discovered defects was limited and not consistent.

The client was becoming aware of a number of reports from users where their new AngularJS site, was not working well on some lesser known mobile devices. There was particular concern that more issues would come to light on pages essential for the primary revenue stream of the business.

Moreover - they had experienced issues before where an upgrade to a new version of the operating system (OS), would raise a number of reports from users that the system did not work well. Whilst only limited to a small number of users, there were concerns that in the event of a problem, users could be taking their business elsewhere.

The Ten10 compatibility test solution

Ten10 was engaged to perform mobile compatibility testing following these steps:

• Collaboration with DevOps engineers to understand usage.
• Identification of target devices for compatibility testing.
• Ten10 device lab to be used to avoid up-front device costs.
• Mobile user-friendly reporting with prioritised actionable recommendations.

Planning compatibility testing

The first step in the project was to carry out a discovery phase; this involved working with the DevOps engineers to understand the usage of the existing application in the live environment. From this the range of target devices, operating systems and browsers were able to be identified, then prioritised by the most used and the highest conversion rates for the client by type of device.

Using Ten10's Digital Test Lab the client could be confident that the necessary test coverage could be achieved without incurring additional expense to purchase the necessary devices.

The second phase was the design of test sessions. Here the appropriate list of combinations for devices and browsers were planned for. The test team also worked together with the developers, to ensure that the design and implementation of the application were well understood in order for tests to be accurately devised.

As well as the testing of existing devices, Ten10 took responsibility for insights and testing recommendations relating to all new devices, operating systems and browsers coming onto the market. This provided the client with peace of mind that the right devices were continuing to be identified and tested against on a continuing basis via Ten10’s cloud-based test environment.

Managing mobile compatibility testing

A structured feedback process was setup for the client, ensuring the information being provided was easily categorised and managed. Issues found during mobile compatibility testing were generally clustered against specific types of device, operating system or browser. This enabled the client to prioritise and action the application changes needed.

Solving a browser compatibility issue

During the testing Ten10 identified that a specific browser was exhibiting issues rendering form fields that resulted in the checkout functionality not working as expected. Not only was this issue reported to the client, but it also enabled Ten10 to adopt a risk-based test approach to the ongoing and future testing, prioritising this particular browser and allowing resources to be allocated most efficiently based on the design and compatibility issues which were being found.