20 December 2014

Chris Pickles

Chris Pickles - Consultant

92Posts 205,591Views 15Comments

Consolidated Audit Trail and really BIG Data

29 January 2013  |  3504 views  |  1

In New York the meeting was held today for the organisations that are interested in bidding to provide and run CAT NMS – the Consolidated Audit Trail system for the National Market System of the USA.

Keeping a full audit train of all securities operations across all market participants, being able to reconstruct what happened to every order, providing access to all market participants and to the many Self-Regulating Organisations (SROs) that monitor and regulate market activity, tracking instruments over years even when they change identifiers  – this is not just a “consolidated tape” project, but much, much more.

And the data volumes predicted are BIG.  The capability to handle 50 billion messages a day.  Storing all data for a 5 year period and enabling the market community and the regulating bodies to access all of that data.  All with a predicted growth rate in data volumes of 25% per year.

On top of the sheer scale there are the whole business operational issues of such a huge industry utility to address as well.  Customer Relationship Management capabilities to meet the needs of all regulated market participants and SROs.  Notification systems to advise market participants of issues, problems, faults, and all in a customer service-oriented timescale.

And then there’s the aspect of access: how will market participants reach this utility to interact with it not only in batch mode but in a real-time mode to deal with reporting, changes to reports, queries about transactions, etc.  And the question of standards – what messaging formats will be used to deliver data to and from and access data in this system that will be at the heart of the US securities market..

Today’s pre-RFP bidders conference saw the beginnings of what has to be one of the largest IT projects that has ever been undertaken in the financial markets sector anywhere in the world.

TagsTrade executionPost-trade & ops

Comments: (1)

Ketharaman Swaminathan - GTM360 Marketing Solutions - Pune | 01 February, 2013, 14:17

Phew... 50B messages per day is H*U*G*E. To put that in perspective, UK FPS is rated for a peak volume of "only" 300M messages / day and Twitter handles "only" 175M messages a day. Besides, we can be sure that a good number of the 50B messages processed by CAT NMS will be about more than what someone had for lunch or dinner on that day, thereby multiplying the processing complexity! Given these figures, CAT NMS might prove to T*H*E largest IT project in any sector ever, period. That said, I noticed that the 50B figure on the CAT NMS website refers to the daily volume of records, not messages. Not sure if each message will contain only one record.

Be the first to give this comment the thumbs up 0 thumb ups! (Log in to thumb up)
Comment on this story (membership required)
Log in to receive notifications when someone posts a comment

Latest posts from Chris

E-Identification is the next major crossroads

11 June 2014  |  1057 views  |  0  |  Recommends 0 TagsSecurityRisk & regulation

Customering - not Banking

10 June 2014  |  1259 views  |  1  |  Recommends 1 TagsMobile & onlineRetail banking

ESMA announces consultations on MiFID II reforms

22 May 2014  |  2532 views  |  0  |  Recommends 1 TagsTrade executionRisk & regulation

Boosting savings - how technology can help

22 May 2014  |  1724 views  |  0  |  Recommends 0 TagsMobile & onlineRetail banking

AIFMD deadline looms in July

20 May 2014  |  1574 views  |  0  |  Recommends 1 TagsRisk & regulationPost-trade & ops

Chris's profile

job title Consultant
location England
member since 2009
Summary profile See full profile »
I help organisations that work in the financial sector around the world to understand better how the sector works, how regulations impact the business operations of financial institutions, and how to...

Chris's expertise

What Chris reads
Chris writes about

Who is commenting on Chris's posts

Tony Wenzel