Upside and Downside Capture Ratio Statistics: Caveats and Cautions

Widespread investment wisdom dictates that there are two “irrational” tendencies of human behavior that harm investment performance when acted on excessively: greed and fear. In a simple sense, “greedy” investors want maximum participation in rising markets and “fearful” investors want minimum participation in falling markets. Upside and downside capture statistics correspond almost perfectly to these emotional responses to price movement, and it may be tempting for investors to base investment decisions on these statistics. However, there are many issues with these statistics that should be considered before basing any decisions on these often misleading numerical summaries. In fact, many funds attempting strategies based on these statistics have failed spectacularly in recent years.

Upside and downside capture statistics are generally calculated as ratios of fund to benchmark compounded returns, taken only during up- or down-markets. Also frequently reported is the ratio of upside to downside capture, with larger up/down ratios considered better for performance. While the motivation for these statistics makes sense from the fear/greed perspective, the following list of problems with respect to calculation and interpretation of these statistics should lead thoughtful fund managers to mostly disregard them:

1. Upside and downside returns themselves are not well defined, and can vary extremely based on the return period, i.e. daily, weekly, etc.. A close examination of price movement shows that fluctuations occur at many time scales, and both upward and downward price movement can be observed within smaller and smaller time periods. To demonstrate the difficulty in assigning “up market” and “down market” periods, consider the accompanying figure (below), in which four panels show different period representations of the same index, the S&P 500 over five calendar years, January 2012 to December 2016. The quarterly representation, in the bottom panel, has only four down market periods, marked in red, with a total downside return of -11%. The daily representation has many up and down periods, with a total downside return of -96.75%.
2. In an extreme example, it is possible for a fund to go up when the benchmark goes down, leading to a negative capture ratio statistic. A negative up/down ratio could signify the fund going up during both downmarkets and upmarkets, or quite the opposite, the fund going down during both. This example underlines why a closer examination of return is necessary and why the up/down ratio is meaningless. In fact, it is always better to thoughtfully examine the entire return history than to blindly trust any single summary of the data.
3. Because of the geometric nature of how returns compound, the downside returns always look less than the upside returns. How much less differs greatly according to the measurement period. Consider the first panel in the figure, the daily returns. A -96.75% return corresponds to approximately a 3077% upside return. These two are reciprocals and would cancel each other out, and the fund with these two returns consecutively would have a total 0% return. The observed 5298% upside return means that the fund had a positive return, but it is difficult to intuit the scale of the upside total return since (1+52.98)*(1-.9675) = 1.7545. Thus the total return over the five year period was about 75%.
4. Evaluating the upside/downside capture ratio over a longer period would certainly lead to a different ratio because of the difference in interpretation of negative returns, which bottom out at -100%, and positive ones, which can be arbitrarily large. Statistically speaking, the ratio does not converge as more data is used for the calculation. This non-convergence leads to huge difficulties in interpreting the statistic and making anything but the crudest of comparisons among funds.
5. Equal total performance can be attained with many different pairs of upside and downside capture statistics, and equivalent pairs under total performance may have very different ratios of upside to downside capture.
6. Many funds are too new to get any reliable information for both up and down markets. If the market has been mostly up or mostly down in every month of the evaluation period and the fund is relatively new, there may be little or no information about how the new fund performs in opposite market scenarios.
7. Reliance on these statistics is likely to motivate strategies which attempt to forecast up or down markets and increase or reduce fund participation based on these forecasts. There is actually very little leading information about the direction or timing of price movement. Leading indicators are normally priced rapidly into the valuation of any security, and remaining statistical anomalies can usually be attributed to market inefficiency rather than a true causal relation between a leading indicator and the sign of market returns. These anomalies generally have only a slight edge over random prediction and are not useful for accurately pinpointing the timing of future market reversals. Because we have no crystal balls, attempts to reduce downside participation will generally also reduce upside participation, and many funds attempting to separate upside and downside participation as a strategy have crashed and burned within recent memory. Still, the marketing appeal of avoiding market downturns is strong and these types of strategies persist despite poor track records.
8. Capture statistics are based on benchmarks, and rely on the appropriateness of the benchmark to the fund strategy. Many funds are not constructed around a benchmark and do not specifically aim to capture benchmark performance. Upside and Downside capture statistics are particularly poorly suited to such funds since their performance is untethered to any benchmark. Although investors always hope for the best possible returns over an investment period, there is no reason gains should be confined to periods for which another fund or index has positive returns.

Because of the difficulties and ambiguities associated with upside/downside capture statistics, we prefer other measures of fund performance which offer more stable calculations and intuitive interpretations. At the top of the list of meaningful performance summaries are total Risk and Return, which should be what investors ultimately care about. These do not depend on a choice of benchmark, and they measure what investors should most care about: the reliable attainment of increase in total portfolio value. Calculated risk and return statistics also approximate characteristics that rational investors should care about, rather than pandering to fear and greed.

Investors would do well to remember that capital markets have always and continue to provide the most reliable returns for investors with well-diversified positions in broad markets over the long term. The ride may at times be bumpy and unpredictable, but over time the rising tide lifts all boats.

Figure 1: S&P500 Returns over five years, sampled at four different frequencies.