Blog article
See all stories »

Chicago - where the Big Data grows

Watching how data volumes coming out of the Chicago derivatives markets have been growing is what you do when you have to build IT infrastructure to support a modern bank or broker anywhere in the world.  Moore’s Law, where processors double in power every 18 months, becomes a practical bottleneck, because the world isn’t governed by Moore’s Law of processor power – it’s governed by Butters’ Law for networks where network capacity doubles every 9 months.  That means that you’ve got an exponentially-increasing difference between the volume of incoming data that you have to process and your firm’s capability to process that data.

Looking at recent statistics from the FIF Financial Information Forum ( we’re seeing the volume of data coming out of securities-type trading platforms hit the 100,000 messages per second level per trading platform.  But this is small compared to what’s coming out of Chicago, and you need to keep an eye on the data traffic forecast from OPRA (Options Price Reporting Authority) to see what that impact already is and is expected to be.

A dealing room today that wants to follow all of the Chicago derivatives markets in real-time needs to be capable of handling over 6 million messages per second.  And that’s happening already – and the volumes are continuing to increase significantly, just as they have done for the last 20+ years.

10 years ago, derivatives exchanges in Europe were balking at telling their members that they should plan to increase their network capacity from 4 Mbps to 10 Mbps.  Today a single firm needs over 1 Gbps just to receive all of the market data from Chicago without its network being a throttle that slows down its receipt of that data.

And as well all know, in financial markets today, microseconds mean money.


Comments: (4)

Ketharaman Swaminathan
Ketharaman Swaminathan - GTM360 Marketing Solutions - Pune 26 March, 2013, 13:11Be the first to give this comment the thumbs up 0 likes

Interesting figures. Thanks for sharing. Taking off from your last line, though, I'd have thought that the growing trend of colocation during the same period would've had the opposite effect on bandwidth. After all, if a bank's trading servers were placed on the exchange's LAN to cater to HFT / low latency trading, it shouldn't need as much WAN bandwidth between its premises and that of the exchange as it did in the pre-colocation era. Maybe your figures have already factored that but I was just wondering...

A Finextra member
A Finextra member 26 March, 2013, 15:49Be the first to give this comment the thumbs up 0 likes

Chicago has multiple exchanges, so being co-located in one exchange doesn't get you co-located in all of them and, on its own, doesn't get you lowest-latency access to all of the Chicago exchanges. OPRA is the market data feed - not the trading system for each exchange - and is a separate agency, so firms that want to trade have to take account of all of those factors when they develop their trading architecture (for investment firms) and/or their architecture for market data aggregation (because this also applies to data vendors around the world).  And today OPRA includes more than just Chicago.  Hope this clarifies.

Ketharaman Swaminathan
Ketharaman Swaminathan - GTM360 Marketing Solutions - Pune 26 March, 2013, 17:24Be the first to give this comment the thumbs up 0 likes

Yes, it clarifies. Thanks.

Allan Grody
Allan Grody - Financial InterGroup - New York 30 March, 2013, 11:57Be the first to give this comment the thumbs up 0 likes


It would seem that taken to its logical conclusion the “industry”
would continue to spend huge amounts of money speeding up its acquisition and access to market, order, trade and execution data. I wonder for what purpose?

Surely each firm’s increment of speed or data acquisition
advantage is met with another’s step up in spend rate to one-up and gain temporary advantage.  This speed and spend frenzy is overlaid on VERY important capital and contract market infrastructure that is the life blood of the global economy. The transactions that flow through its communication pipes and computers allow capital to seek its best uses while supporting the techniques of risk management that is supposed to mitigate the risk of each transaction and/or a portfolio of such transactions.

It seems to me that as an industry we are failing to understand
that it is mainly our customers that pay for the costs of maintaining the speed and spend highway. We will surely lose their trust when the whole thing goes “pop” AGAIN. Then where are we?

Let’s get real, get our “industry” leadership to get us to a better place before the regulators take us AGAIN to the woodshed, perhaps this time to permanent purgatory!