11-05-2023 · Visie

Machine learning in finance: Why and how?

With the rise in popularity of machine learning (ML), it’s no surprise that investment practitioners are attracted to this potential industry game changer.

That’s why this new research report by Robeco is an important contribution to the conversation. It provides detailed analysis of the huge upside of this technological advancement, but also expresses a much-needed note of caution for the finance and investment industry.


  • Mike Chen - Head of Alternative Alpha Research

    Mike Chen

    Head of Alternative Alpha Research

  • Weili Zhou - Head of Quant Equity Research and Head of Quant Investing & Research

    Weili Zhou

    Head of Quant Equity Research and Head of Quant Investing & Research

So, why ML in investing?

The answer is simple: the promise of higher risk-adjusted returns. In general, industry practitioners have found that ML-derived alpha models outperform more traditional, linear models in predicting cross-sectional equity returns.

In this report, the authors explore the interaction between ML and other known quantitative techniques. They examine non-linear effects which are a key factor behind the superior performance of ML. The interaction effect between accounting red flags and equity returns is an interesting illustration of this.

Another notable feature of ML algorithms is their so called ‘knowledge discovery’ abilities. Given that they’re designed to look for relationships, an ML algorithm can decipher the link between inputs and outputs without specifying a hypothesis.

Investor beware…

It’s worth mentioning that financial markets differ from other areas of modern life where ML has made tremendous strides. Financial markets’ inherent complexity throws up notable challenges. For instance, the Robeco research highlights the low signal-to-noise ratio of financial data. This means for a given security, any one metric is generally not a huge determinant of how that security will perform.

Another challenge relates to amount of data, a significant driver of an ML algorithm’s power. While it may feel like there is a vast amount of data in finance, it’s actually relatively small compared with other areas where ML has thrived.

Finally, financial markets are adaptive. This means they ‘learn’ over time, enabling investors to change their approach as required. However, because ML tends to perform well with static systems, this throws up potential difficulties.

Overcoming challenges in application

Due to the short sample data history, overfitting and spurious correlations can occur. While common techniques to overcome this may not work as well in finance, human intuition and economic domain knowledge can help significantly. In the full Robeco paper, you’ll see the authors’ illustration of this using the SHapley Additive exPlanation (SHAP) value.

Robeco suggests that ML investors build a robust data and code infrastructure to tackle potential replicability challenges. This would include a robust system for code version control and documentation of all tested iterations and hypotheses.

Quant investors will be familiar with lookahead bias and the more general problem of data leakage. This phenomenon occurs when data used in the training set contains information that can be used to infer the prediction, information that would otherwise not be available to the ML model in live production. The report outlines various techniques to counteract this potential pitfall.

Machine learning models can be difficult to understand and explain, especially via performance attribution. The full Robeco report offers the fundamental approach to recent explainable machine learning work, including helpful diagrams.

Nieuwe generatie quantoplossingen

De technologische vooruitgang levert ook nieuwe mogelijkheden op voor kwantitatieve beleggers. Het gebruik van meer data en de inzet van geavanceerde modelleringstechnieken stellen ons in staat om tot betere inzichten te komen en onze besluitvorming te verbeteren.

Sample ML applications in finance

The Robeco report analyses practical applications of ML to financial investing. It takes a comprehensive dive into each area of study:

  1. Predicting cross-sectional stock returns
    The ML algorithms used here compare the relative, rather than absolute, returns of securities to predict whether a security’s price will rise or fall. The report delves into five consistent results that practitioner and academic studies have found, starting with the outperformance of ML algorithm prediction versus the traditional linear approach.

  2. Predicting stock crashes
    This application differs slightly in focusing on the worst-performing stocks only, offering potential benefits for conservative strategies which can then exclude securities most likely to crash. The performance of this ML algorithm is shown to be greater than that of traditional approaches, with notable implications for stock selection when looking on a sectoral basis.

  3. Predicting fundamental variables
    Company fundamentals have a major influence on stock price and performance. Fundamentals, being more stable, may be easier to predict than stock returns. ML models give more accurate earnings forecasts. One key conclusion is that ensemble models, traditional or machine learned, were more accurate than individual models alone.

  4. Natural Language Processing (NLP) in multiple languages
    In general, a global portfolio may invest in 20-30 different countries, while a typical investor may understand only two or three languages, if that. NLP can be used to overcome this barrier. Taking Chinese as an example, NLP can be used to detect investment terms in both standard Chinese and slang. This can aid understanding of foreign language investment blogs for instance.