One of the toughest challenges facing smart beta investors today is in determining how these strategies will perform over changing market environments. A new research paper from Scientific Beta offers investors guidance on what to look for when assessing ETFs and indices based on these strategies.

Index provider Scientific Beta has outlined a framework for analysing smart beta strategies.

Proponents of smart beta strategies believe that their outperformance comes as either a reward for weathering factor risks or by exploiting the behavioural inefficiencies present in the market. Opponents often criticise providers of such products as marketeers of investment strategies based on cherry-picked historical data. The truth may be somewhere in between and as such it is imperative that investors perform in-depth analysis of the long-term performance of these strategies.

According to Scientific Beta, their publication “highlights the importance of a limited choice of factors with simple definitions to avoid the temptations of factor mining or factor fishing, which are among the main causes of the lack of relative out-of-sample robustness of smart beta strategies that are based on factor exposures. It also underlines the importance of allocating between smart factors that have decorrelated excess returns with respect to cap-weighted indices in order to favour the absolute robustness of the smart beta strategies implemented.”

Data mining

In order to choose strategies with true economic and academic grounding and avoid those which are just fitted to historical data, investors assessing smart beta products must be mindful of strategies which cherry-pick factors (factor fishing) or decide how to implement strategies on an ad-hoc basis (model mining).

Investors need to choose factor exposures that they believe have a clear economic rationale. Of the hundreds of factors which have been identified in academic papers, most practitioners believe size, low-risk, value, momentum and quality to be the most significant. When choosing a strategy based on these factors Scientific Beta recommends that a tried and tested method be used to select components. For example, when looking at the value premium, they recommend the use of book-to-market value and earnings-to-price when ranking the stock universe.

Index construction methodologies which are chosen purely on what has worked historically (model mining) may not produce the desired results in future years. For example, in 1999 a strategy attempting to gain exposure to the value factor by selecting components based on earnings would have returned 10.8% more than a strategy based on dividends. This highlights the wide variation in performance that can witnessed between strategies which are, at face value, attempting to capture the same risk premium, by changing the variable used as a proxy for the risk factor.

According to Scientific Beta, in order to avoid instances of data mining, the best practice is to use a proven stock selection technique and a standardised, rational index methodology, as opposed to deciding on a methodology after witnessing which has worked best in backtesting.

Non-robust weighting schemes

The method of diversification that strategies employ to implement portfolio weights can have a significant effect on the outcome of a smart beta strategy. Non-robust weighting schemes can lead to unrewarded risks such as increased sector exposures. For example, value strategies tend to have a pronounced exposure to the financial sector, during the financial crisis this would have lead to severe underperformance when compared to a strategy which used a sector-neutral diversification technique to avoid unintended risks.

There are also model-specific risks that can manifest themselves when a strategy relies too heavily on estimates such as expected returns, volatilities and correlation when diversifying portfolios for maximum risk-adjusted returns. In practice, strategies which employ techniques with fewer estimation requirements can often lead to better outcomes as they are less susceptible to error.

To improve the efficacy of smart beta strategies, Scientific Beta recommends that model risk be counteracted by the use of multiple diversification techniques which are then equally weighted. The use of sensible estimation techniques, such as those which exclude extreme data points, will also work to reduce unrewarded risks.

Analysing smart beta strategies

Analysing a strategy’s largest period of underperformance versus a comparable market capitalisation-weighted index can provide useful insight into whether the strategy is performing as intended. Robust strategies may experience periods of relative underperformance, but these should be consistent with market conditions and in-line with the index construction methodology.

De-composing strategy performance to determine the contribution coming from each factor (factor attribution) is useful in gauging how effective the strategy is at capturing its desired premium. A momentum strategy, for example, should be generating the majority of its relative performance from the momentum factor.

Understanding the probability of underperformance in a given year is a useful tool in analysing strategies which have similar long-term performance. Single factor strategies tend to exhibit a small number of large relative gains combined with more frequent periods of smaller losses. In comparison, strategies which combine factors have been seen to outperform with a higher frequency but to a smaller extent in each case.

Multi-factor portfolios

Investors must be aware of the dangers of relying too heavily on one factor to generate relative performance. Factors such as size, value and momentum perform differently over time, and can often endure long periods of underperformance versus a market cap-weighted strategy.

Factor performance varies over time and is dependent on the market environment. (Graphic © Scientific Beta)

Scientific Beta recommend diversification across factors to offset the time varying nature of performance.  As the variation of performance across factors is not in-sync, combining them should smooth outperformance.

When looking at the effectiveness of multi-factor strategies the investor is looking for relative outperformance across different market conditions. Analysing how a multi-factor strategy has performed relative to a broad market cap-weighted index during periods when markets made their highest returns and their lowest returns should provide this insight.

European investors looking for smart beta ETFs which follow multi-factor indices developed by Scientific Beta could look at the Morgan Stanley Scientific Beta Global Equity Factors UCITS ETF (GEF LN) and the Amundi ETF Global Equity Multi Smart Allocation Scientific Beta UCITS ETF-A EUR (SMRT FP) which provide exposure to the Scientific Beta Developed Multi-Beta Multi-Strategy Equal Weight Index.

Investors looking for single factor exposures are served by ETFs from iShares and Lyxor and Deutsche Asset & Wealth Management.