Shifting trajectory of financial markets and trading in the AI age

Amid the evolving trading models and market platforms, the fundamental construct of financial markets—to conclude transactions between anonymous or known counterparties—has largely remained unchanged over the years. At the same time, swayed by advanced technology and data-driven innovations, financial markets have witnessed significant changes in trading methods in recent decades.

Wider adoption of quantitative investing approach in the later part of the 20th century, anchored on pioneering research in investment and portfolio theories, catalysed the transition away from traditional floor or voice-based trading to electronic or computer-based trading. Theories and models like Modern Portfolio Theory, Capital Asset Pricing Model, Efficient Market Hypothesis, Option Pricing Model, and the Three-Factor Model spurred a new wave of automated trading, including algorithmic or high-frequency trading.

Contours of sweeping changes reshaping Financial Markets

Intensifying macroeconomic uncertainties in the post-GFC years have led to a drastic shift in market microstructure accompanied by increased volatility and an exponential jump in global trade volumes. Further, the continuing surge of Artificial Intelligence (AI) in recent years has fundamentally reshaped the information paradigm in pre-trade and trading decisions, swaying buy-side motivations and adding competitive pressure on stock exchanges.

Exponential growth in trade volumes

The World Federation of Exchanges (WFE) data indicates that the number of cash equity trades through electronic order books on global exchanges jumped more than 3.5 times in the last decade (2014: 15.35 billion, 2024: 54.65 billion). The exchange-traded derivatives (ETD) data indicates a 13-times jump in the annual volume of equity derivatives – from (2014: 11.77 billion, 2024: 154.05 billion). Likewise, the annual volume of interest rates, commodities and currency derivatives has jumped 2.1, 2.2, and 1.5 times, respectively, in the last decade. Remarkably, with 8.54 billion contracts in 2024, ETF derivatives have shown more than 6 times growth in this period.

Shift in market microstructure

Innovations in trading models have led to significant changes in the market microstructure and price discovery mechanism in recent years. Balancing transparency and market impact, the proliferation of new dark pools offers enhanced price discovery and protection for large orders. Alongside traditional order types, new types like mid-point, dynamic mid-point extended life, sweep (dark or lit), plus (limit and iceberg), and block (conditional and binding) orders offer a flexible range of order conditions.

Within fixed income and forex trading platforms, enhanced constructs of dealer-to-dealer (D2D), dealer-to-client (D2C) and all-to-all trading models enable different execution modes, including RFQ, order books and streaming services. Recent developments such as 24-hour trading, T+1 settlement, and CCP interoperability significantly influence the shape of the emerging trading landscape.

Unfolding of new information paradigm

A recent study by the Stanford Graduate School of Business and Boston College – Carroll School of Management found that an alpha-seeking AI fund manager outperformed human portfolio managers in 93% of stock-picking cases over a 30-year period, with an average return of 600%. Without implying humans relinquish control over trading decisions, this underscores AI's growing role in autonomous trading decisions.

Amid proliferating AI waves, new methods of unravelling distinct insights from diverse datasets have fundamentally transformed the approach to trading decisions. The application of AI/ML and large language models (LLMs) adds newer abilities to swiftly interpret voluminous alternative data and correlate qualitative cues with real-time analysis of financial and market data, to derive trading signals and predictions.

The continuous rise in global spending on financial market data and analytics underscores the trend of greater use of data and AI in trading decisions. Burton-Taylor International Consulting reports that global spending on financial markets data and news touched $44.3 billion in 2024, with a CAGR of 7.5% between 2020 and 2024. Further, spending on alternative data by the investment management industry in 2024 is estimated to be in the range of $8-12 billion.

AI-led transformation of the trading landscape

Non-conventional nuances of pre-trade decision analytics

Until recently, algo trading strategies primarily relied on historical and time series analysis and market microstructure patterns. With Generative AI augmenting market research and pre-trade decision analytics, firms find newer ways to identify emerging themes by synthesizing macroeconomic, geopolitical, and market events, as well as investors and intermediaries’ sentiments, to uncover unique trading insights. 

Beyond statistical arbitrage relying on macro or micro-events and market microstructure patterns, firms seeking the AI-driven trading alpha are increasingly focusing on faster integration of unconventional research sources while enhancing real-time automation of middle and back-office information flows. While harnessing non-conventional insights, firms continue to assimilate traditional nuances of macroeconomic and financial metrics, regulatory developments, corporate disclosures, and market activity insights within trading strategies.

Adaptive (self-learning) trading strategies

In a market structure siloed on an asset class basis, the complex interplay of liquidity, impact and dealers’ intermediation dynamics widely influence execution quality, timing risk and transaction costs for investment firms. Moreover, differentiated trading methods for different asset classes require more deft orchestration of adaptive trading strategies and execution performance monitoring to minimize arrival slippages, shortfalls, costs and quality deviations. Beyond traditional metrics – i.e., volume, volatility, price, liquidity, costs, fill-rates – based predefined execution scenarios, self-adaptive execution plans exploit macroeconomic and geopolitical events and theme-driven signals to improve trading performance.

As advanced analytics propel the trading engines, the new generation of Algo Wheels expands the frontier of algo trading with enhanced speed, execution quality and transparency. These systems quickly adapt to real-time market events by swiftly adjusting order slicing, pricing and routing conditions, ensuring optimal execution in volatile markets. Further, liquidity plugging in the form of basis trading in correlated assets—whether cash and derivatives markets, indices and ETFs, or through triangular arbitrage in currency markets while looking through the lens of peer activity—promises considerable improvement in execution performance.

Real-time execution performance: Risk and transaction costs analytics

AI-driven systems enhance real-time execution performance monitoring by anticipating market microstructure patterns and enabling prompt interventions to optimize participation levels and in-flight execution. An AI-augmented dashboard enables real-time visualization and drilldown of execution quality, risks and costs through predictive heatmaps at both pre-trade and execution stages. Navigating volatility and adverse market conditions, these capabilities enable better alignment with execution quality and costs benchmarks, as well as support more fitting hedging and risk control thresholds. An indicative view of real-time execution performance analytics:

• Execution performance benchmarking including implementation shortfall, price trend cost, adverse selection, mark out and trading alpha insights.
• Transaction Cost Analytics (TCA) with predictive analysis of explicit costs such as exchange fees, brokerage commissions, clearing fees, and implicit costs, including impact costs, spread costs, and missed trade opportunity costs.
• Risk management analytics covering portfolio exposure and concentration, hedging strategies and sensitivities, counterparty exposure and margin forecasts, as well as risk parity insights.

Importantly, equity, forex and other liquid ETD derivatives with low-touch trading are better suited to AI-driven automated monitoring. However, the fixed income market, with its fragmented and complex trading models entailing high-touch specialists, is not yet prepared for widespread adoption of automated methods.

Trading infrastructure resiliency and adaptability

Amid evolving market dynamics, firms need to implement adaptive trading systems and processes to cope with shifts in market structure and ensure seamless connectivity to multiple data sources as well as new trading venues, interdealer broker platforms and counterparties. Given the criticality of high-performance trading platforms, AI-enabled tools offer considerable benefits for IT operations and performance monitoring. Augmenting human supervision capacities, AI enables enhanced observability with forecasting of IT infrastructure resource needs for provisioning and load balancing, and monitoring of operational performance and prediction of degradation and fail points for timely triage measures.

Harnessing insights from past pattern analysis and predictive evaluation of vulnerabilities, AI supports adopting a resilient operation paradigm with a preventive bias and better calibrated risk tolerance thresholds. With rising cyber incidents and third-party and fourth-party risks, AI enables recursive reassessments of risk scenarios by incorporating updated knowledge of risk intensity to anticipate, identify and respond to changing risk situations.

Need for enhanced guardrails in AI-driven trading construct

The spate of technological evolution will certainly have more profound impact on the functioning of financial markets. As the trajectory of AI adoption continues to rise, the landscape of AI advancements is expected to be shaped by more sophisticated technological capabilities -e.g., deep neural networks, computer vision, speech recognition, among others. While the emerging model of AI-driven trading enhances responsiveness and adaptability, it also introduces significant new risks. Swayed by noisy and less deterministic analysis and insights, the usage of identical AI models and tools induces herding behaviour and creates an aggravated risk of flash crashes in the case of directional and leverage trading strategies. It highlights the need for thorough testing of algorithms and market-wide stress tests, additional circuit breakers, and kill-switches to preserve market integrity and resilience.

Importantly, AI-led innovation in trading relies on establishing AI governance structure and practices, along with adherence to data privacy and protection standards, to safeguard against risks throughout the lifecycle of AI systems. Certainly, governance of AI systems cannot be treated as just another set of compliance tick boxes.