Valentines day was sealed with a less than loving glitch for retailers this year. Thousands British businesses were left helpless after a technical fault hit their card payments system, on Valentine’s Day weekend.

The outage from Global Payments is the latest in a long line of IT glitches from the banking and payments sector. We’re less two months into the new calendar year, and already there have been system failures at Tesco Bank, Natwest and Sainsbury's Bank. 

With regulators due to report back on their investigations into the resilience of key technology infrastructure in the coming weeks, the financial services industry needs to ask itself the question: how are these defects creeping into production? Are financial service providers not testing thoroughly, acting in haste or is their infrastructure simply unable to cope? 

The payments industry has to find a way to move faster but not risk its critical reputational capital. To make sure that IT systems are resilient and reliable, there needs to be more focus on reducing the risks in testing. For example in the fast moving payments industry, test data can be obsolete after only a couple of hours. Even worse subsets of data or even synthesised data may be used, introducing greater risk of errors. Moving to a Data as a Service model means those organisations could develop and test using near-live data, thus ruling out glitches caused by data errors. Test environments can be restored in minutes rather than hours, increasing testing cycles by as much as 1000x.

The regulator has already shown that it’s prepared to take action against financial service providers that leave consumers financially stranded and businesses are increasingly intolerant of outages, mishaps and mistakes. However, demand for the latest and greatest services have never been higher. As the old Facebook developer mantra read, ‘Move fast and break things’, the urge to innovate increases the risk of mistakes. But if banks and payment providers are going to be able to move at speed and maintain stability, they need to be able to embrace new technologies and models that enable this agility.

Businesses cannot afford to have technical glitches, especially when they hit them where it hurts, the cash register. For retailers and traders, Valentine’s Day only comes once a year and once it’s gone that influx of sales is gone with it. Technical errors on days that are critical to the financial success of the business are a recipe for customer churn. With a greater focus on risk-free innovation, businesses can ensure they are moving at pace to provide the best possible service, in the most stable manner.