Limitations of the Approach

Why is it so difficult to forecast the S&P 500 Index?

The price of the S&P 500 is the result of the portfolio flows of a constantly changing set of market participants. Each is characterized by an investment objective, risk tolerance, holding period, and a level of leverage. Each of these elements is unknown and variable. At any given time, one class of market participants may dominate the price formation process. Furthermore, the decisions of market participants are contextual. The importance, the significance of the set of predictors used to form trading decisions changes and there is a feedback loop. Finally, the nature of the S&P 500 Index changes over time as its components vary. This makes any approach based on statistical learning difficult.

Can artificial intelligence ("AI") methods predict future S&P 500 returns based on historical daily prices and volumes?

No. I understand that some people use the term Artificial Intelligence for marketing purposes to attract investors, but in my opinion, to date, the answer is clearly no: there is simply not enough predictive information in the historical daily prices of the S&P 500. That does not mean the same is true for intraday timeframes. That does not mean that one can't extract a useful signal from past S&P 500 daily prices and combine it with the signals of other asset classes or equity indices to produce a very good investment portfolio (that is what CTAs do).

Is Merlin intelligent?

No. For its predictive part and for feature engineering, Merlin relies on advanced machine learning algorithms invented by very smart people. Their names are Cortes, Guillon, Breiman, Schapire, Schmidhuber, Tibshirani, Vapnik, to name a few.

Directional trading with a single asset.

Unlike a diversified market-neutral strategy, Merlin does not have the advantage of being able to choose the opportunities with the highest probability of success from a set of candidates. It has access to only one asset: the S&P 500. It can hold it or stay in cash. It must therefore settle for a lower level of probability of success. In this sense, its strategy is inferior and the compensation for accepting a less attractive risk/return profile is that its performance does not depend on identifying market anomalies or dealing with the risk of similar market-neutral funds being forced to sell.

A slightly frustrating return pattern

Merlin can go for weeks without proposing a single trade if the odds are not in its favor. Be warned, this can be infuriating at times, especially when the market takes off. It is all about compounding returns and avoiding consecutive mistakes.

A lesson of modesty

The US Economy is complex and continually evolving. Investors' risk appetite is constantly changing. Often, time series are noisy and contain very little predictive information. Predicting the direction of the S&P 500 is a fun but difficult task. Many believe it is impossible. They may be right. Merlin does not aim to provide certainty, just probabilities.