12 December 2017
Jes Breslaw

Data as a Service for FS

Jes Breslaw - Delphix

4Posts 12,592Views 1Comments

Sealed with a glitch: Why are technical errors sneaking into payments systems?

06 March 2015  |  4662 views  |  6

Valentines day was sealed with a less than loving glitch for retailers this year. Thousands British businesses were left helpless after a technical fault hit their card payments system, on Valentine’s Day weekend.

The outage from Global Payments is the latest in a long line of IT glitches from the banking and payments sector. We’re less two months into the new calendar year, and already there have been system failures at Tesco Bank, Natwest and Sainsbury's Bank. 

With regulators due to report back on their investigations into the resilience of key technology infrastructure in the coming weeks, the financial services industry needs to ask itself the question: how are these defects creeping into production? Are financial service providers not testing thoroughly, acting in haste or is their infrastructure simply unable to cope? 

The payments industry has to find a way to move faster but not risk its critical reputational capital. To make sure that IT systems are resilient and reliable, there needs to be more focus on reducing the risks in testing. For example in the fast moving payments industry, test data can be obsolete after only a couple of hours. Even worse subsets of data or even synthesised data may be used, introducing greater risk of errors. Moving to a Data as a Service model means those organisations could develop and test using near-live data, thus ruling out glitches caused by data errors. Test environments can be restored in minutes rather than hours, increasing testing cycles by as much as 1000x.

The regulator has already shown that it’s prepared to take action against financial service providers that leave consumers financially stranded and businesses are increasingly intolerant of outages, mishaps and mistakes. However, demand for the latest and greatest services have never been higher. As the old Facebook developer mantra read, ‘Move fast and break things’, the urge to innovate increases the risk of mistakes. But if banks and payment providers are going to be able to move at speed and maintain stability, they need to be able to embrace new technologies and models that enable this agility.

Businesses cannot afford to have technical glitches, especially when they hit them where it hurts, the cash register. For retailers and traders, Valentine’s Day only comes once a year and once it’s gone that influx of sales is gone with it. Technical errors on days that are critical to the financial success of the business are a recipe for customer churn. With a greater focus on risk-free innovation, businesses can ensure they are moving at pace to provide the best possible service, in the most stable manner.

TagsCardsPayments

Comments: (7)

Stanley Epstein
Stanley Epstein - Citadel Advantage Ltd - Modiin | 07 March, 2015, 10:39

You have hit the nail on the head when you suggest inadequate testing. I want to add a couple of extra points as well – bad design (usually because both the business and the technical requirements are not properly understood) exacerbated by the need for indecent haste in launching, demanded by the product owners within the institution. The need for speed is a very explosive factor that often turns sour.

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
James Tomaney
James Tomaney - RenovITe Technologies Inc - Edinburgh | 09 March, 2015, 03:23

I'd certainly agree that a lack of focus on testing is a root cause. I'd qualify that further as an over reliance on manual testing techniques from the 1990s. Modern payment systems are too complex and change too fast for old fashioned testing techniques to be successful.

1 thumb up! 1 thumb up! (Log in to thumb up)
A Finextra member
A Finextra member | 09 March, 2015, 08:46

Banks tend to treat developments as financial projects, so they do not plan for failure due to technical and social interaction in the build process.

On top of this the board is composed of narrowly financial people. And those are not alert to the type of disaster signals that technical projects show.

We call this disaster by optimism

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
Ketharaman Swaminathan
Ketharaman Swaminathan - GTM360 Marketing Solutions - Pune | 09 March, 2015, 09:30

Not sure whether FPS, TARGET2 and SEPA qualify as modern payment systems but, in my experience of running their implementations, business always asks why automated testing isn't being used and IT always pushes back saying modern payment systems are too complex, change too fast and are subject to too many adhoc moving parts driven by nebulous regulation for automated testing to work. If automated testing has now become flexible enough to handle these challenges, the industry should evangelize the market accordingly.

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
Jes Breslaw
Jes Breslaw - Delphix - London | 09 March, 2015, 09:53

Perhaps not so much a question of speed but more of Agility. For example we have customers using packaged solutions like Calypso which are great, but adding features/upgrading takes a long time. If companies can streamline dev/test by adopting agile or DevOps practises, then the delivery becomes faster and quality (if done correctly) goes up. Delphix takes care of the data refresh, rest and rewind, but you need the rest to achieve real speed.

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
Anthony  Walton
Anthony Walton - Iliad Solutions - Leeds | 09 March, 2015, 12:47

It’s almost the perfect storm for introducing bugs into Payments Systems.

 

A clash of old and new technology

The increasing adoption of “Services” , often external

Agile and DevOps redefining what a delivery is, and how long you have to test it

The need to prototype, on-board and deliver more and more quicker and quicker.

 

It’s great to automate but this requires vendors to provide genuine end-to-end coverage, and not in 5 tools that don’t allow you to make changes yourself.

 

1 thumb up! 1 thumb up! (Log in to thumb up)
James Tomaney
James Tomaney - RenovITe Technologies Inc - Edinburgh | 09 March, 2015, 12:51

@Anthony - I have to agree with you on the need for end-to-end coverage and ideally with an integrated test harness that validates end-to-end and not just at each end

1 thumb up! 1 thumb up! (Log in to thumb up)
Comment on this story (membership required)

Latest posts from Jes

Sealed with a glitch: Why are technical errors sneaking into payments systems?

06 March 2015  |  4662 views  |  6 comments | recomends Recommends 0 TagsCardsPayments

Innovate or fail!

26 February 2015  |  2275 views  |  0 comments | recomends Recommends 0

Disrupt or be disrupted

30 January 2015  |  3487 views  |  1 comments | recomends Recommends 0 TagsPaymentsRetail banking

Stress testing: withstanding the resource crisis

30 January 2015  |  2169 views  |  0 comments | recomends Recommends 0 TagsRisk & regulationRetail banking

Jes's profile

job title Director Strategy, EMEA
location London
member since 2015
Summary profile See full profile »
http://www.delphix.com

Jes's expertise

Member since 2014
4 posts1 comments
What Jes reads
Jes's blog archive
2015 (4)

Who's commenting on Jes's posts