Machine learning in finance: Why and how?

The answer is simple: the promise of higher risk-adjusted returns. In general, industry practitioners have found that ML-derived alpha models outperform more traditional, linear models in predicting cross-sectional equity returns.

In this report, the authors explore the interaction between ML and other known quantitative techniques. They examine non-linear effects which are a key factor behind the superior performance of ML. The interaction effect between accounting red flags and equity returns is an interesting illustration of this.

Another notable feature of ML algorithms is their so called ‘knowledge discovery’ abilities. Given that they’re designed to look for relationships, an ML algorithm can decipher the link between inputs and outputs without specifying a hypothesis.

It’s worth mentioning that financial markets differ from other areas of modern life where ML has made tremendous strides. Financial markets’ inherent complexity throws up notable challenges. For instance, the Robeco research highlights the low signal-to-noise ratio of financial data. This means for a given security, any one metric is generally not a huge determinant of how that security will perform.

Another challenge relates to amount of data, a significant driver of an ML algorithm’s power. While it may feel like there is a vast amount of data in finance, it’s actually relatively small compared with other areas where ML has thrived.

Finally, financial markets are adaptive. This means they ‘learn’ over time, enabling investors to change their approach as required. However, because ML tends to perform well with static systems, this throws up potential difficulties.

Due to the short sample data history, overfitting and spurious correlations can occur. While common techniques to overcome this may not work as well in finance, human intuition and economic domain knowledge can help significantly. In the full Robeco paper, you’ll see the authors’ illustration of this using the SHapley Additive exPlanation (SHAP) value.

Robeco suggests that ML investors build a robust data and code infrastructure to tackle potential replicability challenges. This would include a robust system for code version control and documentation of all tested iterations and hypotheses.

Quant investors will be familiar with lookahead bias and the more general problem of data leakage. This phenomenon occurs when data used in the training set contains information that can be used to infer the prediction, information that would otherwise not be available to the ML model in live production. The report outlines various techniques to counteract this potential pitfall.

Machine learning models can be difficult to understand and explain, especially via performance attribution. The full Robeco report offers the fundamental approach to recent explainable machine learning work, including helpful diagrams.

The Robeco report analyses practical applications of ML to financial investing. It takes a comprehensive dive into each area of study:

1. Predicting cross-sectional stock returns
   The ML algorithms used here compare the relative, rather than absolute, returns of securities to predict whether a security’s price will rise or fall. The report delves into five consistent results that practitioner and academic studies have found, starting with the outperformance of ML algorithm prediction versus the traditional linear approach.

2. Predicting stock crashes
   This application differs slightly in focusing on the worst-performing stocks only, offering potential benefits for conservative strategies which can then exclude securities most likely to crash. The performance of this ML algorithm is shown to be greater than that of traditional approaches, with notable implications for stock selection when looking on a sectoral basis.

3. Predicting fundamental variables
   Company fundamentals have a major influence on stock price and performance. Fundamentals, being more stable, may be easier to predict than stock returns. ML models give more accurate earnings forecasts. One key conclusion is that ensemble models, traditional or machine learned, were more accurate than individual models alone.

4. Natural Language Processing (NLP) in multiple languages
   In general, a global portfolio may invest in 20-30 different countries, while a typical investor may understand only two or three languages, if that. NLP can be used to overcome this barrier. Taking Chinese as an example, NLP can be used to detect investment terms in both standard Chinese and slang. This can aid understanding of foreign language investment blogs for instance.