Following a survey we did back in 2014, I posted on Finextra about how machine learning technologies are progressing from academia, robotics and medical engineering into financial services. At that time, there seemed to be some hesitancy with only 12% of 80 quant-savvy finance professionals saying they used machine learning in their workflows.

Has Use of Machine Learning Changed? The 2016 Survey

To provide some answers, we decided to survey attendees at our 2016 finance conference. Our sample was mainly made up of numerically- and model-led quant roles and risk management roles and therefore those most likely to use machine learning. The respondents also spanned across the industry including buy-side, sell-side, insurance, supervision, consultancies, vendors and academia.

A headline finding is that machine learning use has quadrupled compared to 2014; yet 60% of respondents are still not currently using machine learning, although nearly a half of those plan to do so in the next year.

Of the methods used, supervised learning featured prominently (50%), but also unsupervised learning (17.5%), reinforcement learning (7.5%) and deep learning (5%).

What, Why, and Where?

First, a quick warning that machine learning buzzword bingo will follow. If you don’t follow the terms, enjoy the jargon for now, and seek guidance later. Avoid “textperts”, but seek those who, with clarity and credibility, can explain the terms, applications, opportunities and risks.

The buy-side – those who take risks to generate returns - have a head start. Portfolio and risk factor universes are machine learning-friendly. Asset managers can use the techniques to differentiate on innovation, especially when implemented quickly and well. One asset manager we know well applies machine learning techniques to determine correlation and predictive trends across macroeconomic, credit, liquidity, risk and money flow factors. They use supervised methods (support vector machines, classification and regression trees [CART], neural networks, adaptive boosting), unsupervised methods (self-organizing maps), and semi-supervised methods (counterpropagation neural networks).

Machine learning is not used in isolation. Buy-siders like our afore-mentioned asset manager also apply econo-physics methods such as change point analysis, fractal market hypotheses, such as wavelets, and of course that often-forgotten step-child of data science, optimisation, the numerical work-horse of risk parity and smart beta.  

The sell-side has a more nuanced “risk management-first” view, oriented around task, skills and, importantly, regulation. Data science and big data teams, sometimes born from Quant departments and other times IT, focus on “test” problems, such as fraud detection, money laundering investigation, or modelling customer activity.

In established functions such as risk management, the highly regulated transparency/validation-dependent model preferences have historically precluded the take up of “black box” machine learning.

However, banks are collaborating with supervisors constructively to reconsider the issue in some cases, for example on credit scorecard modelling, where well-documented CART methods are becoming mainstream. A potential outcome is more reliable models with longer shelf-lives, better for all.

In the front office and like their peers on the buy-side, traders are excited about multi-layered deep learning, but beware of fixing on trends that represent correlation at a point in time rather than causality across time. Deep learning is not a home-run. ‘Kkastner’, one commenter on an extensive Reddit thread on the topic, noted “You need signal – LSTMs [Long Short Term Memory] (and neural networks generally) are very good at automation aka making decisions from complex inputs with low noise. The stock market is absolutely not one of these things.” In addition, well-executed “simple” linear regression would have placed you in the top 10 of 264 in the famous Kaggle 2011-2012 algorithmic trading competition.

Natural language processing, e.g. internet-scraped text classification, is another prospective deep-learning application arousing trader interest.

Don’t forget the importance of other data science methods outside machine learning. One conference speaker presented a fabulous graph theory-inspired network representation of Panama Paper participants.

The financial services industry’s use of technology can be contradictory, fast-paced yet also conservative, sliced across segments, skills and tasks. Our small survey offers compelling glimpses into change, that the financial services industry is increasingly leveraging machine learning, as part of a broad tool-set, to address challenging problems and build exciting new use cases.