In an earlier article, I had explained why there is a need for financial institutions (FIs) to implement holistic trade surveillance. However, implementing it is easier said than done. FIs need to think strategically and take note of certain key ingredients in order to make their holistic trade surveillance implementation a success.

What is holistic trade surveillance?

Holistic trade surveillance enables an integrated method to trade surveillance – it encompasses both trade surveillance and communication surveillance. By surveilling both trade and the associated communication, and leveraging the concerned structured and unstructured data sources, holistic trade surveillance provide a thorough approach. For e.g., in holistic trade surveillance, an alert that suggests a potential insider trading incident would be analyzed alongside the communication between the trader and the insider who conveyed the price-sensitive info to the trader. Without such integrated trade and communication surveillance and analysis, it would be challenging for the surveillance team to decisively prove that the trade was executed based upon insider information.

Following are some of the key characteristic of holistic trade surveillance:

It brings together trade, market, voice and written communication, and other relevant data points, to enable an integrated picture.

It aggregates data from myriad disparate sources - including emails, voice, video, text messaging, IM chats, memos, user & access activity log, information security log and transactional data.

It involves holistic data collection and analysis of a) trade related data including those of client, trader, broker, etc., b) trade related communication between individuals/entities, c) info distributed in the market during the given period, d) social media info which can be connected with the trading behavior, e) behavioral and HR data of staff and traders.

It applies unique analytics for uncovering insights and understanding of connection between events, trades and communications.

Proactively identifies nascent trading behaviors and communications to assess the possible intent.

Is imperative for effectively meeting the regulatory requirements such as those of MiFID II which demands that all pre-, at-, and post-trade data (including associated voice and text communications) be automatically brought together for trade reconstruction within 72 hours of request. Similarly, holistic trade surveillance is crucial to fulfil the Dodd-Frank Act requirement of complete audit trail for enabling accurate and comprehensive trade reconstructions.

By automatically linking all of the associated data, it saves substantial time for the surveillance teams – allowing analysts to conduct investigation in minutes instead of days. It makes surveillance processes much more effective and efficient.

Holistic trade surveillance: Key ingredients

1) Strategy. FI should take a phased, iterative, exploratory, and collaborative approach to holistic trade surveillance implementation. Towards this, they should proactively work with the key stakeholders – including senior management, compliance, information security and HR functions – and gain their support. Cross-functional collaboration would help FIs move away from a fragmented surveillance program to a holistic one.

Defining robust strategy vis-à-vis governance and control, communication surveillance and monitoring, solution build-versus-buy decision (on-premise, cloud, hybrid), and leveraging of “golden data source” for value maximization is crucial. Further, FIs should list out all relevant use cases of holistic trade surveillance and explore cross-leveraging opportunities. For example, apart from trade surveillance, trade communication could also be leveraged for employee surveillance, conduct risk management, AML, anti-fraud and internal threat management.

Efforts should be made to strengthen the three lines of defense (LoDs) and empower the concerned staff to take a risk-based and context-driven approach to surveillance. 1LoD, comprising the front-line management and their designees, should provide forward-facing surveillance, take real-time approach to risk-mitigation, and actively collaborate with 2LoD (i.e. the compliance teams). 2LoD should focus on collaborating across the compliance function (including trade, communications, AML, fraud, transaction monitoring etc.). 3LoD, comprising internal auditors, should pay emphasis on data quality.

 2) Architecture. FIs’ trade surveillance system should be well integrated with the GRC and operational risk management systems so as to provide more quantifiable conduct related data about the traders and trades. In order to enable predictive and preventative surveillance, solution should be able to proactively, efficiently and effectively monitor across the entire trade lifecycle. Also, it should be capable of ingesting any type of structured or unstructured data - including orders, trades, positions, user activity, email, market data, chat logs, voice, KYC & CRM. Robust and consistent global coverage across concerned markets, regulatory jurisdictions, lines of business, communications, trading venues and asset classes and instruments should be enabled. 

Implementing strong case and workflow management capabilities are important. A web-based, centralized and fully integrated case management solution is needed to support streamlined and integrated workflow and reporting, investigative workbench, efficient alert management, ad-hoc investigation, robust audit trail, and to boost compliance analyst productivity. Solution should enable advanced market visualization capabilities – for e.g., by displaying the alerts along with market activity and transactions – so as to allow compliance analysts to visualize watch list networks, market data and news, trades, orders, positions, control rooms and chat rooms communication etc. and see the activities in relation to each other (for e.g. if trades are impacting the market). Enabling robust graphing and web-based dashboard capabilities are vital.  

Where appropriate, FIs should consider adopting open solution architecture – this would help reduce the implementation and maintenance cost. The solution’s scalability aspect should be especially focused upon – solution should be capable of handling trillions of events and data points each day, when needed. Also, solution should possess streaming capabilities so as to enable capture and storage of events in real-time. This would help in immediate evaluation and analysis of alerts.

3) Data. To enable truly holistic trade surveillance, seamless sourcing and contextualizing of both structured and unstructured data – including new and historic transactions, orders, trades, positions, market data, trade and communication alerts, employee HR data, texts, chat logs, IMs, emails, call transcripts, voice messages, social media, case history etc. - is crucial. However, blending the structured and unstructured data from myriad sources is challenging.

To overcome this challenge, FIs should focus on optimally automating the data capture, indexing and harmonizing process and leverage big data and data lake capabilities. Complex Event Processing (CEP) technologies such as Apache Spark can be leveraged to analyze big data in real time. By utilizing data lake, data loss would be minimized, data integrity ensured, data silos removed, and more varieties of data stored. Data lakes possess the capability of processing higher volumes of data at substantially lower cost. Advanced analytics, artificial intelligence (AI) and machine learning (ML) based surveillance processes could also be directly run within the data lake. Preferably, FIs should implement single data lake for their trade surveillance function.

To help support electronic communication analysis, FIs should leverage, as appropriate, robust semantic (NoSQL / NoREL) database management systems, data stores and highly scalable in-memory data grid such as Cassandra, Apache Hadoop & Accumulo, and SAP’s HANA.

Credit Suisse partnered with Palantir to track rogue traders – the solution leverages big data technologies. As another example, Neurensic’s SCORE platform combines high-speed, big data processing capabilities with self-adaptive pattern recognition technology – to provide firms with continuous assessment of compliance risk associated with complex trading behaviors.

4) Advanced analytics: FIs should leverage predictive analytics, sentiment analytics and other advanced analytics (such as those referred below) for anomalous trading behavior detection, enabling risk-based discovery, supporting cross-asset-class & cross-market surveillance, supporting efficient trade reconstruction, and more.

Native visual analytics: Leveraging common visual analytics platform which is native to the data lake is recommended. In native visual analytics BI platform, the visual analytics engines are installed directly on each of the nodes within the data lake. In combination with self-service and intuitive web interface, analysts can collaborate on the “original data” – which has not been aggregated/summarized or moved to separate visualization server. This provides more confidence to analysts about the integrity of analysis. Analysts can visually study data across the surveillance platforms and link the risk activity patterns. They can analyze all data types in one place, and see historical and real-time data analysis side by side.

Real-time streaming analytics for enabling streaming capabilities needed to capture and store the events whenever they happen; and supporting the surveillance story using granular, time-based filters. Streaming analytics allow firms to execute real-time analytics computations on the data streaming from social media, applications, videos, websites, unstructured text, devices etc. It enables speedy time-sensitive processing along with the language integration for intuitive comprehension. Through real-time streaming analytics, huge volumes of data in motion are processed and analyzed quickly. Real-time streaming analytics can help generate timely alerts about potential threats. Technologies such as Kafka, Spark and Flink are making real-time streaming analytics feasible.

Entity and network analytics for creating holistic view across all of the internal and external data. It helps establish and analyze hidden entity linkages/relationships and identify the right communication data set for alerts investigation. Network analytics help flag internal/external communications, and identify the interactions that don’t obey normal communication patterns. FIs can adopt advanced visualization-based linkage analysis tools.

Behavioral analytics to help uncover hidden behavioral and conduct related threats by individuals or entities (counterparty, trading desk, accounts). It monitors and analyzes a wide range of data – trades, communications, alerts, HR database, access log, and other structured and unstructured data – to identify the deviations from normal behavior, and provides behavioral risk scoring capabilities.

Based upon the data, it creates profiles of individual’s/entity’s normal behavior across multiple risk dimensions (e.g. trading outside normal hours, increases in order cancellations, P&L changes etc.).  Further, anomaly detection models are leveraged to identify departures from the normal behavior. Individuals/entities are assigned cumulative risk score based upon their deviation from the normal behavior corresponding to each risk factor. Dashboard highlighting the individuals/entities with high risk scores enable analysts to timely identify the rogue elements.  Also, behavioral analytics data gets automatically correlated with traditional alerts within the case management tool, thereby providing surveillance analyst with additional context for the alert.

Recently, Firstrade Securities, a leading online investment firm, started using Trillium’s (a proprietary trading & trading technology firm) Surveyor platform for trade surveillance. Surveyor is a post trade analytics & market-surveillance tool.

5) Artificial Intelligence (AI) & Machine Learning (ML). FIs should leverage AI capabilities – including ML, robotic process automation (RPA), natural language processing (NLP) – for holistic trade surveillance. Nasdaq for example has explored ML for scoring and ranking of alerts that its trade surveillance solutions brings to customers’ attention. Leveraging AI/ML capabilities would help FIs in:

- Intelligent adjustment and fine-tuning of alert parameters to reduce number of false positives

- Enhancement of efficiency and relevance of alerts (through clustering of clients having common characteristics (e.g. long-only asset managers, market makers) into sub-sets)

- Enabling cross-asset linked alerts & real-time alerting

- Predictive evaluation and ranking/scoring of alerts as per their likely relevance and criticality

- Dynamically developing profiles of the trader activities (by automatically clustering various trader types depending upon their style of trading (e.g. HFT, block trading etc.)) to ascertain outlier behavior

- Using NLP to understand the context and instantly find suspect trade related conversations by automatically analyzing all communications and trades

- RPA capabilities to help automate trade surveillance tasks and workflow (e.g. automatic transfer of alert, as needed, to more sophisticated investigation teams)

- Automatic trade reconstruction, order book reconstruction, and market replay

Hong Kong Exchanges & Clearing Limited (HKEX) has successfully implemented, across its equity market, Nasdaq SMARTS Market Surveillance’s ML solution & participant-relationship discovery technology. Similarly, India’s National Stock Exchange (NSE) is leveraging AI solution to boost its surveillance operations.

While embedding ML capabilities in their holistic trade surveillance solution, FIs should first investigate the quality of historical data used for training the system and ensure it is optimal.  

Conclusion

The advantages of holistic trade surveillance implementation are far too many to ignore. FIs that dither in implementing holistic trade surveillance solution risk losing competitive advantage to those FIs that are proactive in implementing it. However, to ensure effective implementation, FIs need to take note of and utilize the key ingredients.