Blog article
See all stories »

Lifting The Fog – Finding The Optimal Market Data Architecture

Market Data Over The Next Decade is About Preparation

Managing market data has never been as exciting (or complex). Discovering the cloud’s potential to displace legacy market data environments, financial institutions have adopted a cloud-first mentality and are openly embracing the technology for its reported benefits. The decade-long partnerships between LSEG, CME Group and their respective cloud service providers (CSP) are flagship examples. The growing interest in cloud-based architecture is undeniable, however, it does not necessarily translate into the deprecation of legacy on-prem IT architectures and mainframes.

Mainframes are predicted to remain in service for another decade as they are (currently) sufficient for traditional tasks and, as previously highlighted, uprooting entire market data systems for the cloud is a resource-intensive process with the risk of failure. However, in light of the industry-wide accelerated adoption, the cloud cannot be ignored either. The balancing act to find the most optimised architecture suggests that hybrid cloud may become the mainstream over the next 10 years by retaining core functionalities in a tested environment, slowly phasing into the predicted distributed cloud landscape while technologies and methodologies mature for industry needs.

The progressive hybrid-cloud approach towards a wider cloud adoption strategy is also possibly the raison d'être for LSEG’s acquisition of MayStreet, a low-latency feed handler vendor that enables its subsidiary (Refinitiv) to modernise legacy components and hedge against their 10-year Microsoft Azure partnership.

Gaining A Competitive Cloud Advantage

Given the next decade may be a transitional era of hybrid cloud, the challenge for market data managers will be ensuring market data systems remain reliable and resilient throughout the metamorphic process with minimal disruptions, like system outages – especially considering the global regulatory focus and the associated penalties. Competitive advantages only materialise after architectures prove robust long term and are, according to Gartner, orders of magnitude higher value when observability initiatives are highly orchestrated and integrated.

Gartner also found that applied observability provides firms with unique opportunities to drive competitive advantages through shorter latency of decision-making and time to market, accuracy, and proactive customer centricity by analysing actual data resulting from enterprise operations to define future decisions. It aligns with Deloitte, which found predictive maintenance increases productivity by 25%, reduces breakdowns by 70% and lowers maintenance costs by 25%.

The future is not about predicting; it’s about preparing. The value proposition of applied observability involves a shift from reactive to proactive. IT leaders’ highly orchestrated use of actual stakeholder actions, rather than intent or predictions, drives competitive advantage.” (Gartner Research)

Delivering Measurable Value

It is increasingly evident that observability is key for long-term cloud computing success. Unfortunately, as noted in an earlier article, observability is usually fragmented as cloud solutions and legacy systems normally possess a proprietary monitoring solution. For example; Google Cloud's Operations Suite - formerly Stackdriver -  AWS’ CloudWatch, and Microsoft's Azure Monitor

According to a report, organisations typically rely on seven infrastructure monitoring solutions to manage multi-cloud environments. Data management teams must subsequently query data and maintain individual monitoring solutions manually, which restricts resources for value-generating activities and creates infrastructural blind spots. The IT leader-focused report also found:

  • 58% said infrastructure management is an increasing drain on resources as usage of cloud services increases.
  • 57% said multiple monitoring solutions across multi-cloud environments made it difficult to optimise infrastructure performance and resource consumption.
  • 61% said observability blind spots in their environments are becoming a greater risk to digital transformation.

To achieve true infrastructural observability, holistically, consolidating and centralising the respective monitoring solutions into a single interface is a logical first step. Data management teams can then easily extract value-add analytics to facilitate what Deloitte writes as predictive maintenance, “taking proactive measures based on advanced data analytics to predict and avoid machine failure.” As Splunk highlighted, IT spending must deliver measurable value and data management teams cannot deliver (or optimise) what they do not know. Getting data to tell you what you don’t know requires analytics.

"Predicting failures via advanced analytics can increase equipment uptime by up to 20%." (Deloitte)

Bridging The Skills Gap

With many firms looking to the cloud over the next decade, an industry-wide blended skills shortage has officially been recognised and has “reached a crisis level in many organisations” according to Leong on Gartner. To address the shortage, Leong adds that many organisations are attempting to train existing staff but may fail because the “DevOps emphasis of successful cloud-optimised or cloud-native adoption results in fundamentally different jobs” and the additional stress “often turns into a spiral of departures.” Leong’s solution? Contactors to fill the near-term gaps, including training and a gradual handover, but advises caution against those that “throw less-qualified bodies into the mix”:

Many organizations are successfully bridging the gap with consulting (professional services) and managed services (a Gartner survey showed about three-quarters of organizations use such services for at least a portion of their cloud IaaS+PaaS adoption). Many cloud managed services deals include explicit training and gradual handover to the customer’s personnel, allowing the customer to take over bit by bit as their team gets comfortable.

However, MSPs, SIs, and other outsourcers are also struggling to fulfil the demand, which leads to both project delays as well as throwing less-qualified bodies into the mix in order to try to meet contractual obligations and grow revenues.

Regular readers will recall that in November we discussed the longer-term cloud considerations and what was dubbed the “manifesto for the multi-cloud infrastructure partner” by IPC Systems’ Director of Network Product Strategy, Richard Balmer. It included (1) The provision of holistic visibility tools; (2) Public cloud integration expertise; (3) Multicast distribution complexity awareness; (4) Service-level-agreed (SLA) performance guarantees.

1647

Comments: (0)

Now hiring