Using artificial intelligence for selecting small-cap stocks

Small-cap stocks are publicly traded companies with market capitalizations typically ranging between USD 250 million and USD 10 billion.¹ Small caps are an integral part of the global equity market portfolio, and many consider explicit allocation to them. “This is especially the case in today’s markets, where mega-cap tech stocks dominate large-cap indices, which also trade at elevated valuations,” says Hanauer. In contrast, small-cap indices comprise a much wider and more diversified mix of stocks, trading at a discount of approximately 20% compared to large caps.

The small-cap universe presents attractive alpha opportunities for investors who know where to find them, but also sizable challenges due to its broad and heterogeneous nature. Human analysts can only cover a limited number of names, and the more niche nature of small-cap firms makes specialization more difficult. ²

For example, on average, each small-cap stock is currently covered by about eight analysts, compared to 19 for large-cap stocks.³ Although fundamental managers might struggle to cover the investable small-cap universe, it fits naturally into a quantitative approach, as the cost of developing a quantitative model does not increase with the investable universe’s size. Conversely, as Roersma points out, quantitative strategies actually benefit from the universe’s breadth, from which they can systematically exploit a broad set of opportunities.

The small-cap universe’s size, diversity and complexity make it difficult for analysts and fundamental investors to cover it thoroughly. Its resulting market inefficiency and mispricing, however, can create larger opportunities to generate alpha. This greater degree of mispricing can be observed empirically, for example, by analyzing hypothetical top-minus-bottom portfolios based on well-known stock market factors such as value, momentum, and quality. Figure 1 illustrates that the annualized factor premiums and their corresponding volatilities are higher in the small-cap space than in the large-cap space.

To model such stock market dynamics, we turn to machine learning (ML), a subcategory of AI. In this specific case, says Hoogteijling, we are interested in modeling which stocks are likely to yield high returns (and thus are potential buy candidates) and vice versa. One key advantage of ML techniques is their flexibility in capturing data patterns. In a simpler, traditional approach, one would often assume that relationships are linear. However, not all real-life relationships can be linearly approximated. ML algorithms such as random forests and neural networks⁴ do not impose such a strict functional form, but instead can be more freely shaped depending on the data given to the model. ⁵

To assess the performance of these ML models – in other words, whether they can accurately predict whether stocks will outperform or underperform – we conduct a historical simulation study classifying the top and bottom half of stocks’ predicted returns. Then, we investigate to what extent the various models can correctly predict these two halves. If a model had no predictive power, about half of the stocks would be correctly predicted.

As illustrated in Figure 2, we find that both a generic factor model and an ML model outperform the ‘no predictability’ benchmark in terms of model accuracy. Furthermore, the ML model outperforms the generic value-quality-momentum factor model. Although stock-specific predictability of around 51-52% might seem low, small but consistent advantages can translate into long-term success. These stock-specific hit ratios translate into portfolio hit ratios of 68% and 78% for long-short quintile portfolios based on monthly returns, when sorted by generic and ML model predictions, respectively. These numbers highlight how even a slight improvement in predictive accuracy can lead to substantially better portfolio outcomes. ⁶

In recent years, AI has shifted from hype to practical application, with machine learning in particular showing real promise for quantitative investing. Small-cap equities, with their breadth, inefficiencies, and limited coverage, offer especially fertile ground for these techniques.

Yet, as Hoogteijling explains, AI is not a silver bullet: success depends on careful model selection, interpretation, and explainability. When used with skill and discipline, however, machine learning can be a powerful addition to the quant investor’s toolkit, and a source of enduring edge in small-cap markets.

This article is an excerpt of a special topic in our five-year outlook.

Footnotes

¹ As of the end of April 2025, based on the market capitalization numbers for the constituents of the MSCI World Small Cap Index, winsorized at the top and bottom 2.5%.
² As of the end of April 2025, based on four bottom-up-calculated multiples (price-to-book, forward price-to-earnings, price-to-earnings, and price-to-dividend) for the MSCI World and MSCI World Small Cap indices.
³ Source: IBES.
⁴ A neural network is a type of ML model that learns to recognize patterns, inspired by how the human brain processes information. A random forest is a group of simple decision-making models (called decision trees) that work together to make more accurate choices than just one tree on its own.
⁵ For a broader view of the different applications of ML in asset management, we refer to Blitz et al. (2023) and Chen and Zhou (2023).
⁶ The performance shown is hypothetical, based on theoretical models and assumptions, and is no guarantee for future results.