The client
The client is a global firm of professional intermediaries that plays a pivotal role in the world’s financial, energy and commodities markets. The 2,800-broker organisation manages a portfolio of businesses providing intermediary services, contextual insights and intelligence, trade execution, data and analytics. It enjoyed revenues in excess of £900 million in the first half of 2017.
Setting the project goals
The company uses numerous trading platforms across the newly-merged organisations to process the trading of a wide range of asset classes including rates, FX, commodities, fixed income, credit and equities.
The client had no system for unifying data delivery in order to produce the desired output: reports for the regulator, in the correct format, of transactions generated by the company. To attain compliance could have involved major changes to all its trading platforms. In addition, given that the regulation was to come into effect within a year of starting the project, there was a deadline to meet.
Determining a test approach for variable data and data delivery methods
The large number of widely disparate order and trade management and processing platforms in use by the company generates data in a range of formats – such as JSON and XML, for example. The data delivery methodology also varies by platform: some push data, others pull. In addition, the number of trading strategies is in excess of 3,000, which is too high for a manual testing approach. Consequently, it would have been unfeasible, given the scope and complexity of the work and the schedule involved, to make changes to each of these systems so they could directly produce the transparency information required by MiFID II in a unified format within the deadline.
This variety of order management and trade management systems and the variety of asset classes and instruments traded on those platforms led Ten10 to make two strategic recommendations.
First was the development of an automation framework to support automatic injection of resources into the company’s data lake, a process that would involve simulating the upstream systems. The second was the
automation of verification and validation of results and output – in effect, intercepting the data to be sent to the regulator – instead of relying on conventional, manual functional testing to perform these tests.
Developing an Agile test approach as part of a continuous integration solution
Deploying a small team of five individuals, we worked closely with the client’s developers to perform a test coverage that would have otherwise required a huge manual functional testing team.
We used a custom Java automation testing framework executed as part of a continuous integration solution – in an Agile approach to development. Regression testing capability was consistently built alongside functional testing without extra efforts by re-using the same test scripts in a continuous integration solution triggered on each new build.
The automation testing framework underwent several transformations throughout the project life-cycle to adapt to changing project needs. It ended up being also used to test areas of the data-lake in isolation, to test systems other than the data-lake solution itself, and even allowed early end-to-end testing while some external dependencies were not yet met by supporting reference data mocking-up. This helped to keep project progress on track.
The versatility of our automation framework and the skills of our resources meant new areas of testing which cropped up late into the project could be picked up by the existing testing team without requiring additional resources.
In addition, Ten10 has delivered extra value in the form of assistance with solution design, project management, solutions architecture and implementation creation.
Test deliverables at-a-glance
Jira management
Managed the ~2500 tickets for the entire 20-25 person GIRCA team.
Manual testing across multiple front and middle office systems including strategy testing
Order management systems: 9 different platforms
Trade management systems: 12 different platforms
Field validations: ~150 fields *~95 deal types *2 (orders & trades), (4-5 rounds of testing – more for reportable instruments)
Lifecycle testing: ~5 fields *95 deal types *15 scenarios (many steps each) *2(orders & trades)
(4-5 rounds of testing- more for reportable instruments)
Automation testing
Hundreds of scripts, thousands of assertions, run several times a day.
