24 July 2016
Find out more

RBS meltdown blamed on Indian IT operative

27 June 2012  |  12225 views  |  5 keyboard

Royal Bank of Scotland is refusing to confirm or deny reports that an inexperienced IT operative in India was partially responsible for the computer meltdown that afflicted payments processing at the bank last week.

According to reports in UK tech rag The Register, the junior technician in India accidentally wiped out a queue of batched payments while backing out of a routine software upgrade.

Quoting a 'source familiar with the matter', the Register says that the situation spiralled out of control during an attempt to reverse a planned upgrade to the bank's mainframe-hosted CA-7 workload automation engine.

"When they did the back-out, a major error was made. An inexperienced person cleared the whole queue ... they erased all the scheduling," says the source. "That created a large backlog as all the wiped information had to be re-inputted to the system and reprocessed. A complicated legacy mainframe system at RBS and a team inexperienced in its quirks made the problem harder to fix."

Earlier this year the bank moved to recruit CA-7 batch support staff in Hyderabad on a salary of £9000-£11,000 per annum while slashing higher paid equivalents in the UK.

If substantiated, the revelations could prove damaging for the bank's embattled boss Stephen Hester, who on Tuesday denied claims that the breakdown was in any way connected to underinvestment in IT and the offshoring of jobs to India.

In an interview with Sky News that touched on the outsourcing rumours, Hester said: "Well I have no evidence of that. The IT centre - our main centre, we're standing outside here in Edinburgh, [is] nothing to do with overseas. Our UK backbone has seen substantial investment."

Analysts at Ovum point to the problems faced by large banks in running mission-critical processes on ageing legacy systems with a diminishing pool of skilled and experienced staff.

"With most banks under heavy cost pressures, this means relatively junior staff are often given responsibility for systems where they have little experience beyond the routine, particularly in a stress situation (as with RBS) where things go outside normal operations," states Ovum. "The challenge for banks is that an operation (e.g. the software update to CA-7) that would score relatively low on the risk register escalated rapidly into a critical error. It illustrates that conventional cost versus risk decisions can be problematic, particularly for mission-critical (and their supporting) systems."

Separately, the FT is reporting that RBS is considering legal action against Computer Associates, the supplier of the CA-7 software. The board of the bank spent much of Tuesday in crisis talks, says the FT, after which it decided to launch a full-scale review.

The FT quotes an insider at the meeting: "It was certainly an issue with this software. We will still have to establish if this was their fault or if it was our handling of the software."

These are choppy waters for the bailed-out bank group, with politicians also demanding answers. The chairman of the Treasury Select Committee, Andrew Tyrie MP, has written to RBS chief Hester, and FSA chairman, Lord Turne, calling for action. "We need to know exactly what went wrong and what will be done to give us confidence that similar mistakes are not repeated in the future," says Tyrie. "Basic bank functions are crucial in a modern economy. It is deeply concerning that they appear to have been put at risk again."

Inexperienced RBS operative's blunder led to bank meltdown - The Register
KeywordsOUTSOURCING

Comments: (5)

Alexander Peschkoff
Alexander Peschkoff - TEDIPAY - London | 27 June, 2012, 10:55

"The Royal Bank of Scotland (RBS) Group is an international banking and financial services company. From its headquarters in Edinburgh, the RBS Group operates in the United Kingdom, Europe, the Middle East, the Americas and Asia, serving over 30 million customers worldwide."

Having a single IT failure point for a global company of that standing (and, especially, if such a point is represented by "inexperienced (!) IT operative") is laughable (and doesn't sound convincingly plausible). If that is indeed the case, it's time to grind that axe.

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
A Finextra member
A Finextra member | 27 June, 2012, 13:11

Situations such as this should never be attributed to the failure of an individual - its a failure of the 'System' that should have been designed to prevent a series of unfortunate incidents excalating into a major disaster...  Sadly it does appear that RBS are suffering from a condition that affects people who skip servicing or who cannot afford the best oil, filters and spares for their cars.... breakdowns are an inevitable consequence..... Someone at RBS may have taken a view that reducing costs, in order to securing a better bonus last year was a risk worth taking..... its a form of casino banking where you can win... but only if you dont lose (big time)

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
A Finextra member
A Finextra member | 27 June, 2012, 16:11

Is this a case of:

Too many cowboys, not enough indians?

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
A Finextra member
A Finextra member | 28 June, 2012, 09:14

What nonsense:

<< The FT quotes an insider at the meeting: "It was certainly an issue with this software. We will still have to establish if this was their fault or if it was our handling of the software." >>

Backups shoudl have been made, and it should have been able to revert back.  Also I've read that the upgrade was not properly tested.

The fault is most probably not with the outsourced ITer who managed to delete schedules whilst backing out - the fault lies with RBS management trying to offshore too agressively without ensuring knowledge transfer and bulletproof procedures to deal with such a situation.

They were offlien for a week, and as Keith Appleyard reports here on a Finextra blog, there are still severe consequences for customers.   Has anything like this happened before?  A top nationwaide bank going offline (more or less) due to an IT incident?

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
Paul Nash
Paul Nash - CityLearning Ltd - Dublin | 19 April, 2016, 16:49

As Chad would put it: Wot, no backup?

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
Comment on this story (membership required)

Finextra news in your inbox

For Finextra's free daily newsletter, breaking news flashes and weekly jobs board: sign up now

Related stories

I'm a person, not a bank: tweeters mistakenly bombard @natwest

I'm a person, not a bank: tweeters mistakenly bombard @natwest

25 June 2012  |  12918 views  |  1 comments | 1 tweets
RBS and NatWest Web services knocked out; Oslo Bors hit by DDoS attack

RBS and NatWest Web services knocked out; Oslo Bors hit by DDoS attack

21 June 2012  |  12904 views  |  3 comments
&quot;Brutal&quot; jobs cuts at RBS spark debate on branch investment

"Brutal" jobs cuts at RBS spark debate on branch investment

19 June 2012  |  7435 views  |  0 comments
RBS closes Bristol site

RBS closes Bristol site

29 November 2011  |  5551 views  |  0 comments
RBS investment bank unit targets tech and market data in cutback plans

RBS investment bank unit targets tech and market data in cutback plans

17 October 2011  |  7968 views  |  0 comments
RBS and CBA suffer technical glitches

RBS and CBA suffer technical glitches

14 December 2010  |  10386 views  |  0 comments
Hester stresses importance of technology in RBS rebuilding plan

Hester stresses importance of technology in RBS rebuilding plan

25 October 2010  |  10452 views  |  0 comments
RBS axes 500 back office jobs at investment banking unit

RBS axes 500 back office jobs at investment banking unit

28 September 2010  |  8945 views  |  0 comments
RBS to axe 3500 tech and support jobs

RBS to axe 3500 tech and support jobs

02 September 2010  |  11060 views  |  1 comments
RBS to shed 2600 jobs; 500 roles to be shipped offshore

RBS to shed 2600 jobs; 500 roles to be shipped offshore

10 May 2010  |  9163 views  |  0 comments
RBS cuts over 200 Irish IT jobs; Lloyds axes another 585 roles

RBS cuts over 200 Irish IT jobs; Lloyds axes another 585 roles

21 January 2010  |  10057 views  |  2 comments
RBS cuts hundreds of IT jobs

RBS cuts hundreds of IT jobs

21 May 2009  |  8922 views  |  0 comments
RBS to shed 9000 more jobs in back office shake-up

RBS to shed 9000 more jobs in back office shake-up

07 April 2009  |  8012 views  |  0 comments
RBS to cut 2300 UK back office jobs

RBS to cut 2300 UK back office jobs

10 February 2009  |  10684 views  |  0 comments

Related company news

 

Related blogs

Create a blog about this story (membership required)
Visit VocaLink.comVisit www.abe-eba.euFind out more

Top topics

Most viewed Most shared
MasterCard agrees £700m VocaLink acquisitionMasterCard agrees £700m VocaLink acqu...
8354 views 14 comments | 32 tweets | 37 linkedin
satelliteContactless Bitcoin startup Plutus Tap &am...
8044 views comments | 6 tweets | 3 linkedin
hands typing furiouslyBanking on IoT: Security in the Internet o...
8002 views 3 | 19 tweets | 7 linkedin
hands typing furiouslyWhat Every FinTech CEO Should Know About R...
7958 views 0 | 14 tweets | 8 linkedin
Santander doubles down on fintech fundSantander doubles down on fintech fund
7662 views comments | 22 tweets | 28 linkedin

Featured job


Brussels (Belgium) or Paris (France)

Find your next job