Functional & mobile compatibility testing, delivered remotely & at pace

Functional & mobile compatibility testing, delivered remotely & at pace

The client: An international money transfer business serving both retail and business customers with more than 40 currency pairs and over 200 payment destinations.
Industry: Financial services
Technologies: HTML 5
Ten10 services: Test strategy, functional testing and mobile compatibility testing

The project:

• A new foreign exchange website. to allow customers to transfer funds overseas.
• A minimum viable product (MVP) for phase 1 launch.
• Initially targeted at a UK-based audience.
• No in-house testing capability
• External expertise sourced to assist with functional, mobile compatibility and [functional] security testing of the site.

The solution:

• Collaboration with client project manager and lead developer to understand site architecture and visitor journeys.
• Adoption of exploratory testing approach in order to identify issues more quickly.
• First class collaboration and communication critical to successful delivery of the project.

The testing challenge

The client was looking to develop a new foreign exchange website, initially to serve a UK-based audience but potentially with a view to futureserve a global audience.

As the website would be supporting a new service, the client were looking to launch a minimum viable product (MVP) initially to monitor demand, gain user feedback and inform future development.

As there is no in-house test capability at the client, it was recognised that they would need to source some external testing expertise to help with the project. There were three particular areas of interest for the client; functional, mobile compatibility and security of the site (from a functional point of view), with the main priority being the functional testing of the site.

A flexible test approach

Ten10 was initially engaged to carry out a short fixed-term piece of consultancy to advise on the best testing approach for the new site.

Discovery

During the initial discovery phase, the test team project manager spent one day with the client’s project manager and lead developer to learn about the website; building up a complete understanding of exactly who the audience would be and their objectives when visiting the site, as well as ascertaining how the site had been developed and understand the site’s architecture. This was essential preparatory work in order that an appropriate test approach could be determined.

Test planning

Ten10 provided the client with a recommended Test Approach document, which included a comprehensive set of best practice recommendations specific to the client’s website testing project as well as offering advice on performance testing and operational acceptance testing (OAT).

Due to the client resources and timescales available to deliver the project objectives, Ten10 recommended using an exploratory approach to testing, rather than a more traditional, scripted approach.

Using exploratory sessionbased testing would enable more issues to be flushed out and identified more quickly, and so would help to meet the timescale that the client needed to work towards.

“As the majority of the website development was complete at the point that Ten10 were engaged, it meant that the development team needed fast feedback on the quality of the website. The pressure on timescales coupled with the anticipated close working between the development and test teams meant that an exploratory testing approach was the most logical choice.”

Emma Hargreaves, Test Consultant, Ten10.

Also included in the Test Approach document was advice from Ten10 on the combination of mobile devices, operating systems and browsers that the foreign exchange website should be tested against for compatibility. This was based on UK market usage of Android and Apple devices to identify the most popular combinations and therefore the minimum to test against, but also included a secondary level of recommended, optional combinations.

Refining the test approach.

Ten10 worked together with the client’s project manager to review the Test Approach and identify which elements were of the highest priority for the business. From this, Ten10 created and defined a detailed statement of work (SOW) that balanced the testing needs for the project with the budget and resource available.

From the approved SOW, Ten10 was then able to build out a comprehensive test plan, documenting the exploratory sessions that would be performed for the agreed functional and mobile compatibility testing.

Delivering the testing

During the exploratory testing Ten10 identified 350 defects. Ten10 assigned an initial priority to each defect in terms of risk to go-live, ranging from urgent to high, medium or low. The client was provided with full access to all defect details so they were able to review the findings and, where relevant, change the priority assigned based on their own in-depth understanding of the planned website usage and the business’s attitude to risk.

In a testing project such as this one would expect to see the gap between the total number of defects found and the defect closure rate to shrink over time. However, in the early reporting provided to the client it was apparent that as defects were continuing to be identified, the defect closure rate was not gaining on the total number of defects found (see figure 1). Ten10 discussed this finding with the client and agreed to provide an additional report separating out the urgent and high priority defects to enable easier monitoring of the progress being made to resolve significant defects (see figure 2).

Effective communications for remote working

The successful delivery of the project was heavily reliant on the regular contact and communication between Ten10’s test team and the client’s in-house development team. Collaborative repositories for exploratory test sessions and defects, setup by Ten10 were central to the efficiency of the project management, as these enabled the client to have full and unrestricted access to up-to-date information at all times.

To effectively manage remote working by the test team, a stand-up was held each morning via Skype between the test team and the client’s development team. The stand-up gave both teams the opportunity to discuss any project risks or blockers, review a daily burndown of the sessions completed, as well as monitoring the defect closure rate against plan. This ensured there was complete transparency on the project’s progress but also, importantly, it enabled the project priorities to be reviewed and adjusted in light of the exploratory test findings.

Feedback from the client project manager was that the daily team stand-ups (via Skype) really minimised the effect of remote working across different locations and time zones. The collaborative repositories for test sessions and defects also facilitated open and up to date communications throughout the project.

The structure of the test sessions repository worked well and gave the client further confidence that all functional areas were being covered and allowed for prioritisation to ensure that the high priority areas were tackled first.

Testing delivered at pace

Working with a specialist testing consultancy provided confidence that testing had been successfully performed on the site, and gave the client the insight to identify and address the urgent and high priority defects ahead of launching site. An added benefit for the client was that Ten10 were able to provide some independent usability feedback as a result of their experience testing and using the site.

The client was able to deliver a first-class experience for their users, helping to develop trust and customer loyalty.