12 December 2017
Testing Testing Testing
Anthony  Walton

Testing Testing Testing

Anthony Walton - Iliad Solutions

14Posts 70,824Views 6Comments

10 things a CEO should know about testing payments before it goes wrong

14 November 2016  |  4961 views  |  0

We were talking in the office recently about the pressure CEOs face when things go wrong. If a payment system falls over, and it hits the headlines, it can have a devastating impact on the business.

As a results, we thought it would be useful to list the top ten things that CEOs need to know about payment testing – and here they are:

 

Software testing should be a governance issue

High profile software failures have a significant impact on the share prices of those companies affected.  On average this equates to around 4% of the value of the business. If any other single element had this potential to damage your business value, you’d make it a governance issue.

Some of your people are ‘knocking together’ bits of code to test critical developments

Your organisation may spend hundreds of thousands, even millions on software solutions and then permit a contractor or software author to ‘knock together’ some code to test it.  To quote an old adage, you risk spoiling the ship for a ha’porth of tar. Project overruns in both time and cost put pressure on testing, and often force significantly shortened test cycles.  Without a properly engineered test environment you run the risk of your mistakes leading to full blown social media exposure.

The answer isn’t ‘throw more people at it’

The law of diminishing returns governs manual software testing in the same way it rules so many people-dependent functions.  F1 teams, faced with making cars faster, changed the shape, weight, balance and behaviour of their cars – they didn’t just add engines until they broke. Too often, adding people simply adds confusion, cost, inefficiency and management overheads, and rarely achieves the intended results.

You throw away a lot of the IP you develop

Manual testers come and go.  Either they leave, or you change your testing partner. They record their results in spreadsheets and written reports, which can’t be easily transferred, re-employed or automated. As testing evolves, the IP you develop is an important part of protecting your business. Not to capture this in an employable form is an oversight at best and can lead to added problems further down the line.

There is such a thing as human white space

Those queues you see at ATMs on a Friday lunchtime when traffic is busy are repeated in your test environment, while people wait for access to devices or schemes to complete their testing.  Limited equipment means literally hours wasted by testers waiting for availability – and that ignores the time taken to re-set the hardware and software environments when they do eventually become available.

Identifying a problem early saves a small fortune

Almost everyone in software development knows of the 1-10-100 rule. In essence, it costs a hundred times as much to resolve a software issue in production as it does in the design and development phases of a project.  In spite of this, testing remains an afterthought for too many organisations.

Your software is probably tested in pieces, and never in the way it works in the real world

It’s likely that if you take the simplest payment transaction in your business you have up to four test simulators, operated by people with specific skills, with separate results and different reporting formats. When, for instance, your ATM to VISA transaction fails its test, it will almost certainly require a handful of people to sit together in a room and work out where and why.  But in the real world a transaction flows. It doesn’t break itself into chunks that can be debated into an operational state. You should test as you run your business – from end to end; anything less than that is inviting failure.

Your test environment is almost certainly not auditable

Manual testing is inordinately difficult to repeat consistently.  Successful results obtained once may have depended on a unique set of manually created conditions that cannot be recreated.  It may be impossible to deliver proof that the test worked, or failed, when challenged weeks or months later, after a production outage for example.

Test to prove your software doesn’t work – not to prove it does!

Happy day testing is too tempting for developers.  If I’d been allowed to mark my own homework, I’d be a PhD. I’d have spent my time testing what I already knew, not what I had yet to learn.  Too many organisations focus on proving that software works, and yet the focus should be on trying to break it.  It’s too easy to be self-satisfied right up to the point that Twitter goes into meltdown because your clients can’t get at their money.

Your testing environment is probably not fit for purpose

Sadly, the test environments of most financial institutions are less than ideal. No matter how hard the staff work, their equipment is probably second rate at best and close to archaic at worst.  When I explained how testing works to a former senior product officer from a high street bank, he was shocked. He told me that if he’d known how his former company tested he would never have released a single product.

 

So what should you do?

You need to invest in the right testing solutions and the right test strategies. If your people don’t have the right strategy at their disposal, and the right environment within which to employ it, you are setting up both them and yourself for failure.

We’d suggest you get to know your test environment properly; learn what really happens and how exposed you and your organisation are. Testing shouldn’t be an afterthought or something you scrimp on – it should be central to ensuring that you reduce the risk of failure to an absolute minimum.

 

a member-uploaded image TagsPaymentsRisk & regulation

Comments: (0)

Comment on this story (membership required)

Latest posts from Anthony

Profitability and the future for US Banks

9 hours ago  |  1165 views  |  0 comments | recomends Recommends 0 TagsRetail bankingTransaction bankingGroupTrends in Financial Services

World Economics Forum, analysis of the outlook for Fintech

12 September 2017  |  9182 views  |  1 comments | recomends Recommends 1 TagsPaymentsInnovationGroupFintech

IT spend at EU banks set to increase

03 August 2017  |  7047 views  |  0 comments | recomends Recommends 0 TagsPaymentsInnovationOpen APIsGroupInnovation in Financial Services

THE REAL COST OF OUTMODED TESTING

24 July 2017  |  5128 views  |  0 comments | recomends Recommends 0 TagsPaymentsInnovationGroupDigital Banking Trends

The biggest threat to the future of your payments business

09 May 2017  |  6135 views  |  3 comments | recomends Recommends 0 TagsPaymentsEBAdayGroupTrends in Financial Services

Anthony 's profile

job title CEO
location Leeds
member since 2011
Summary profile See full profile »
Identifying ways to improve the testing of our clients mission critical systems.

Anthony 's expertise

Member since 2007
4 posts6 comments
What Anthony reads

Who's commenting on Anthony 's posts

André Casterman
Anthony Pickup
Ketharaman Swaminathan
Matt Scott