Lesson 4.2: Performance metrics: Sharpe, drawdown and Sortino
Learn to measure a strategy: Sharpe, Sortino and Calmar ratios, drawdown, how long a track record must be, and how crash risk hides inside the Sharpe ratio.
By the end of this lesson you will be able to compute and interpret a strategy's return, volatility, Sharpe ratio, Sortino ratio, maximum drawdown and Calmar ratio from a series of returns, say how much the risk-free rate and the annualisation method change the answer, work out how long a track record must be before a Sharpe ratio means something, and explain how a strategy that sells insurance can show a great Sharpe ratio for two years and then collapse. You will show that a twelve-month series has a Sharpe ratio of 0.84 if the risk-free rate is ignored and 0.11 if it is 6 per cent, that a Sharpe ratio of 1.0 measured over five years has a 95 per cent range from about −0.1 to 2.1, that it takes about 6 years of data to tell a true Sharpe ratio of 1.0 from zero and 18 years for 0.5, and that autocorrelated monthly returns of 0.3 inflate an annualised Sharpe ratio by a third. You should have read Lessons 1.3 and 3.4. The mathematics is averages, standard deviations and square roots. This course is education about how markets and research work, not investment advice.
A backtest produces a column of profits and losses. By itself it is hard to judge. Is a 12 per cent annual return good? That depends on the risk taken to earn it, the period, the costs and how likely it is to repeat. Performance metrics reduce the column to a few numbers that answer these questions. Each metric captures one view and hides others, so a quant reports several and understands the blind spot of each. This lesson covers the standard set, and then shows where they mislead.
Return and risk, the starting point
Suppose a strategy has monthly returns, in per cent, of 2.1, −1.3, 3.0, 0.8, −4.2, 1.9, 2.5, −0.6, 1.2, −2.8, 3.4 and 0.9. They are an invented year. We use them as the running example.
Total return. Compounding the twelve months, 1.021 × 0.987 × ... gives a final value of 1.0679, a total return of 6.79 per cent. For exactly one year that is also the compound annual growth rate, CAGR. For a longer series, CAGR = (final value)^(1/years) − 1.
Mean return. The arithmetic mean is 0.575 per cent a month, or 6.9 per cent a year when multiplied by 12. It is a little above the compound return because of the volatility drag of Lesson 1.3.
Volatility. The standard deviation of the monthly returns is 2.36 per cent. Annualised with the square-root rule it is 2.36 × √12 = 8.18 per cent.
The Sharpe ratio
The Sharpe ratio, introduced by William Sharpe in 1966, measures return per unit of risk:
Sharpe ratio = (average return − risk-free return) ÷ standard deviation of returns
The numerator is the excess return, what you earned over holding cash, since a risk-free return could have been had without taking the risk. To annualise from a period with N periods in a year, multiply the per-period ratio by √N: monthly by √12, daily by √252.
Worked example 1. The monthly mean is 0.575 per cent and the standard deviation 2.36 per cent. Ignoring the risk-free rate, the monthly Sharpe ratio is 0.575 ÷ 2.36 = 0.244, and annualised it is 0.244 × 3.464 = 0.84. Now suppose the risk-free rate is 6 per cent a year, about 0.5 per cent a month. This is an illustrative figure: use the rate that applies to your currency and period. The excess return is 0.075 per cent a month and the Sharpe ratio falls to (0.075 ÷ 2.36) × 3.464 = 0.11. The same returns give 0.84 or 0.11 depending on one assumption. A quoted Sharpe ratio is meaningless unless it says what was subtracted. For a market-neutral strategy, whose capital earns interest on the side, using zero for the risk-free rate is common, but say so.
How good is a given Sharpe ratio? As rough guides for a diversified, liquid strategy after costs: below 0.5 is weak, around 1 is respectable, 2 is excellent and above 3 is very rare and usually a sign of a short history, a high-frequency strategy with a lot of capacity limits, or a bug. The numbers depend heavily on the type of strategy and the data.
Sharpe ratios are noisy
A Sharpe ratio is an estimate, with a standard error, as Lessons 1.3 and 3.4 showed. For independent returns, a good approximation of the standard error of an annual Sharpe ratio is
SE = √((1 + SR² ÷ 2) ÷ T)
where T is the number of years of data. A strategy with a measured Sharpe ratio of 1.0 over five years has SE = √(1.5 ÷ 5) = 0.55. The 95 per cent range is 1.0 ± 1.96 × 0.55, from −0.07 to 2.07. It could be a fine strategy or an ordinary one. The t-ratio is 1.0 ÷ 0.55 = 1.83, short of the usual cut-off of 2. How long must a track record be before a Sharpe ratio is distinguishable from zero? Setting the t-ratio to 2 gives T = 4 × (1 + SR² ÷ 2) ÷ SR²:
True Sharpe ratio · Years of data needed for a t-ratio of 20.5 · 18.0
1.0 · 6.0
1.5 · 3.8
2.0 · 3.0
A strategy with a true Sharpe ratio of 0.5, which would be a respectable long-only investment, needs about 18 years of history to be told apart from nothing. This is the reason that so much of investing rests on theory and economics rather than on track records alone. It is also why you must discount a short-track-record Sharpe ratio, and a best-of-many one even more, as Lesson 3.4 showed.
Put in your own numbers below.
Annualising when returns are autocorrelated
The square-root rule needs returns that are independent from one period to the next. When they are positively autocorrelated, as happens with illiquid assets valued at stale prices, with smoothed marks or with strategies that hold positions for a long time, the true annual volatility is larger than the square-root rule says, and the annualised Sharpe ratio is overstated. Andrew Lo showed in 2002 how to correct it. With monthly returns that have an autocorrelation of 0.3 at lag one and decaying geometrically, the proper annualisation factor is not √12 = 3.46 but 2.62, so the naive method overstates the Sharpe ratio by about a third (a factor of 1.32). At an autocorrelation of 0.5 the overstatement is 63 per cent. Check the autocorrelation of your returns (Lesson 2.4) before you quote an annual Sharpe ratio, especially for anything that is not marked to market daily.
Downside risk: the Sortino ratio
The Sharpe ratio divides by the standard deviation, which counts a large gain as risk as much as a large loss. Investors do not feel that way. The Sortino ratio, proposed by Frank Sortino and Lee Price in 1994, replaces the standard deviation with the downside deviation: the root mean square of the returns below a target, with returns above the target counted as zero. Sortino ratio = (average return − target) ÷ downside deviation.
Worked example 2. With a target of zero, the four losing months of the example have returns of −1.3, −4.2, −0.6 and −2.8 per cent, and the other eight months count as zero. The mean of the squared shortfalls over all twelve months is (1.3² + 4.2² + 0.6² + 2.8²) ÷ 12 = (1.69 + 17.64 + 0.36 + 7.84) ÷ 12 = 2.29, so the monthly downside deviation is 1.51 per cent, or 5.25 per cent annualised. The Sortino ratio is 6.9 ÷ 5.25 = 1.32, higher than the Sharpe ratio of 0.84 because the strategy's volatility comes partly from its gains. The Sortino ratio rewards a strategy whose upside is more volatile than its downside, and it punishes the opposite.
Drawdown, time under water and the Calmar ratio
Volatility measures the daily bumpiness. A drawdown measures what an investor actually lives through: the fall from a previous peak to a later low, as in Lesson 1.3. The maximum drawdown is the largest such fall, and the time under water is the length of time spent below a previous peak.
In the example the value is 1.0463 at the end of month 4 and 1.0023 at the end of month 5, a fall of 4.2 per cent, the maximum drawdown. It does not regain the previous peak until month 7. Other falls are smaller: 2.4 per cent in month 6 (a continuation), 0.6 per cent in month 8 and 2.8 per cent in month 10.
The Calmar ratio divides the compound annual return by the size of the maximum drawdown. Here, 6.79 ÷ 4.2 = 1.62. It is popular with managed-futures funds because it speaks to an investor's main worry, which is how bad it can get. Its weakness is that the maximum drawdown is a single observation: the longer the history, the deeper the worst fall is likely to be, so the ratio is not comparable across series of different lengths, and one extreme event dominates it. A cautious reader assumes the next maximum drawdown will be larger than the one in the backtest.
Other numbers worth reading
Measure · What it tells you · Watch out forWin rate · Share of periods or trades with a profit: 8 of 12 months, 67%, in the example · Meaningless without the size of wins and losses, as Lesson 2.1 showed
Profit factor · Total gains divided by total losses: (2.1 + 3.0 + 0.8 + 1.9 + 2.5 + 1.2 + 3.4 + 0.9) ÷ (1.3 + 4.2 + 0.6 + 2.8) = 15.8 ÷ 8.9 = 1.78 · Sensitive to a few large trades
Skewness and kurtosis · Shape of the returns: the example has a skewness of −0.76, a longer left tail · A strategy with negative skew can have a great Sharpe ratio, as the next section shows
Turnover · How much is traded: needed to judge costs (Lesson 4.4) · A strategy with high turnover is more sensitive to cost assumptions
Exposure · The share of time and capital at risk · A low-exposure strategy may have a high Sharpe ratio but little capacity
Beta and correlation to a benchmark · How much of the return is just market · A strategy that is a levered market bet is not an alpha
Information ratio · Excess return over a benchmark divided by tracking risk · Depends on the benchmark chosen
Worst month, worst day · The tail that actually happened · Short histories miss the worst
## Where the Sharpe ratio misleads: selling insurance
The Sharpe ratio sees only the mean and the standard deviation. It is blind to the shape of the returns, and a strategy can exploit that, deliberately or not. Consider a strategy that sells protection against rare events, for example by selling deep out-of-the-money options. Most months it collects a small premium, and once in a while it suffers a large loss.
Worked example 3. Suppose that each month the strategy earns about 1.0 per cent, with a small noise of 0.5 per cent, in 96 months out of 100, and loses about 12 per cent in the other 4. The long-run average is 0.96 × 1.0 + 0.04 × (−12) = 0.48 per cent a month, and the long-run standard deviation is 2.60 per cent a month, so the true Sharpe ratio is 0.48 ÷ 2.60 × √12 = 0.64, a mediocre figure. But the chance that a given month is a calm one is 96 per cent, and the chance that a whole 24-month history contains no crash at all is 0.96²⁴ = 38 per cent. In those crash-free histories, the standard deviation of the returns is just the small noise of 0.5, and the measured Sharpe ratio is about 7.2. We simulated 20,000 two-year histories and found that 37 per cent of them showed a Sharpe ratio above 2.
Anyone who looked at a two-year track record of this strategy would see a Sharpe ratio of 7, a maximum drawdown of nothing and a steady monthly profit. It is exactly the pattern of many failed funds. The lesson is not that the Sharpe ratio is useless, but that it must be read together with skewness, the worst day, the tail risk and an understanding of where the return comes from. A return that has a story, such as compensation for bearing a clear risk, is more believable than one that is simply smooth. Be most wary of a very smooth profit curve.
A reporting standard
When you present a backtest, report at least:
1. Annualised return and volatility, and how you annualised.
2. The risk-free rate used, and the Sharpe ratio with it stated.
3. The Sharpe ratio, its standard error or range, and the number of independent observations.
4. Maximum drawdown, its date and the time under water.
5. Skewness and the worst period, with the Sortino ratio.
6. Turnover and the costs assumed.
7. The number of variants tried to reach the result (Lesson 3.4).
8. Performance by sub-period, so that one good year does not carry the whole.
The code below computes the example's metrics, the track-record length, the autocorrelation adjustment and the crash simulation.
python
import numpy as np
r = np.array([2.1, -1.3, 3.0, 0.8, -4.2, 1.9, 2.5, -0.6, 1.2, -2.8, 3.4, 0.9]) / 100
equity = np.concatenate([[1.0], np.cumprod(1 + r)])
peak = np.maximum.accumulate(equity)
drawdown = equity / peak - 1
mean, sd = r.mean(), r.std(ddof=1)
print("total return", round(equity[-1] - 1, 4), "annual vol", round(sd np.sqrt(12), 4))
print("Sharpe, no risk-free rate", round(mean / sd np.sqrt(12), 2),
"| with 6% risk-free", round((mean 12 - 0.06) / (sd np.sqrt(12)), 2))
downside = np.sqrt((np.minimum(r, 0) 2).mean())
print("Sortino", round(mean / downside np.sqrt(12), 2),
"| max drawdown", round(drawdown.min(), 3), "| Calmar", round((equity[-1] - 1) / abs(drawdown.min()), 2))
print("win rate", round((r > 0).mean(), 2), "profit factor", round(r[r > 0].sum() / abs(r[r < 0].sum()), 2))
uncertainty of a Sharpe ratio and the years needed
for sr, years in ((1.0, 5), (0.5, 5), (1.0, 10)):
se = np.sqrt((1 + sr 2 / 2) / years)
print("SR", sr, "years", years, "SE", round(se, 2), "range", round(sr - 1.96 se, 2), "to", round(sr + 1.96 se, 2))
for sr in (0.5, 1.0, 1.5, 2.0):
print("true SR", sr, "years for t = 2:", round(4 (1 + sr 2 / 2) / sr 2, 1))
annualising monthly returns that are autocorrelated (Lo, 2002)
q = 12
for rho in (0.0, 0.3, 0.5):
factor = q / np.sqrt(q + 2 sum((q - k) rho k for k in range(1, q)))
print("monthly autocorrelation", rho, "annualisation factor", round(factor, 2),
"naive overstatement", round(np.sqrt(q) / factor, 2))
short volatility: a strategy that sells insurance
p, win, lose, noise = 0.04, 1.0, -12.0, 0.5
m = (1 - p) win + p lose
v = (1 - p) (win 2 + noise 2) + p (lose 2 + noise 2) - m 2
print("true Sharpe", round(m / np.sqrt(v) np.sqrt(12), 2), "| chance of no crash in 24 months", round((1 - p) 24, 2))
rng = np.random.defaultrng(4)
srs = []
for in range(20000):
crash = rng.random(24) < p
x = np.where(crash, rng.normal(lose, noise, 24), rng.normal(win, noise, 24))
srs.append(x.mean() / x.std(ddof=1) np.sqrt(12))
print("share of 24-month histories with a Sharpe above 2:", round((np.array(srs) > 2).mean(), 2))
Common mistakes
Quoting a Sharpe ratio without the risk-free rate. The same returns can give 0.84 or 0.11.
Trusting a Sharpe ratio from a short history. A Sharpe ratio of 1.0 over five years has a range of about −0.1 to 2.1.
Annualising smoothed returns with the square-root rule. Positive autocorrelation inflates the result, by a third at 0.3.
Reading a smooth curve as safety. Selling insurance gives a beautiful curve until the loss.
Using the maximum drawdown as if it were a limit. It is the worst so far, and the next one is likely to be deeper.
Comparing Calmar ratios across different lengths of history. The longer the history, the deeper the worst fall.
Reporting a win rate alone. It says nothing about the size of wins and losses.
Choosing the metric after the result. If you report the one that looks best, you are snooping again.
Forgetting costs and turnover. All of these metrics are only as good as the net returns they are built on.
Key takeaways
The twelve-month example returned 6.79 per cent with an annualised volatility of 8.18 per cent. Its Sharpe ratio is 0.84 with no risk-free rate and 0.11 with a 6 per cent rate. Always state which.
A Sharpe ratio has a standard error of about √((1 + SR² ÷ 2) ÷ T): a Sharpe of 1.0 over five years ranges from −0.07 to 2.07, and a true 0.5 needs about 18 years to reach a t-ratio of 2.
Autocorrelated returns inflate the square-root annualisation. At a monthly autocorrelation of 0.3 the overstatement is about a third.
The Sortino ratio uses only the downside deviation: 1.32 for the example. The Calmar ratio is the compound return divided by the maximum drawdown: 6.79 ÷ 4.2 = 1.62.
Drawdown, time under water, skewness and the worst period describe what the investor lives through. The maximum drawdown only grows with the length of history.
A strategy that sells insurance has a true Sharpe ratio of 0.64 in our example, but 38 per cent of its two-year histories show a crash-free Sharpe ratio near 7.2. A smooth return needs an explanation.
Report returns, risk, the risk-free rate, the Sharpe ratio with its uncertainty, drawdown and its duration, skewness, turnover, costs, the number of variants tried and sub-period results.
Check yourself
1. A strategy returns 9 per cent a year with a volatility of 6 per cent, and cash earns 5 per cent. What is its Sharpe ratio?
(9 − 5) ÷ 6 = 0.67.
2. A fund has a daily Sharpe ratio of 0.08. What is the annualised figure, and is the answer reliable after one year of data?
0.08 × √252 = 1.27. After one year the standard error is about √((1 + 0.8) ÷ 1) = 1.3, so the range runs from about −1.4 to 3.9: it is not reliable.
3. A strategy rises from 100 to 140, falls to 105, rises to 150 and falls to 120. What is the maximum drawdown?
The falls are (105 − 140) ÷ 140 = −25.0 per cent and (120 − 150) ÷ 150 = −20.0 per cent. The maximum drawdown is −25.0 per cent.
4. A private fund reports monthly returns with a lag-one autocorrelation of 0.5. Its annualised Sharpe ratio from the square-root rule is 1.6. What is a better estimate?
The naive method overstates by about 63 per cent, so 1.6 ÷ 1.63 = 0.98.
5. Why can a strategy with a great Sharpe ratio and a flawless two-year record still be dangerous?
If it earns a small steady premium for taking a rare large loss, the Sharpe ratio only reflects the calm part of the history. Look at the skewness, the source of the return and how often large losses occurred in longer data.
Try this
Set aside forty-five minutes. Use daily or monthly returns from one of your own backtests, or from a fund's published monthly returns.
1. Compute the total and annualised return, the volatility and the Sharpe ratio with and without a risk-free rate. Say which you would report and why.
2. Compute the standard error of the Sharpe ratio and its 95 per cent range with the simulator, and the number of years needed for a t-ratio of 2.
3. Compute the autocorrelation of the returns at lag one. If it is above 0.1, apply the correction from the code and compare.
4. Compute the Sortino ratio, the maximum drawdown with its start and end dates, the time under water and the Calmar ratio.
5. Compute the skewness and the worst month, and describe the shape of the distribution.
6. Split the history into halves, or into calendar years, and recompute the Sharpe ratio for each part. Say how stable it is.
7. Write a one-paragraph report following the standard of this lesson, and name the one number in it you trust least.
A good result is a one-page table of the metrics with the uncertainty of the Sharpe ratio next to it and a clear sentence on which numbers would change if the history were longer. The next lesson looks at the question that all of these metrics beg: how do you know the future will look like the past? It introduces walk-forward testing.
Read more
William Sharpe's 1994 article "The Sharpe Ratio" in the Journal of Portfolio Management gives the ratio's definition and uses, and Andrew Lo's 2002 article "The Statistics of Sharpe Ratios" in the Financial Analysts Journal derives the standard error and the correction for autocorrelation. Frank Sortino and Lee Price's 1994 article "Performance Measurement in a Downside Risk Framework" introduces the Sortino ratio. David Bailey and Marcos López de Prado's 2012 paper "The Sharpe Ratio Efficient Frontier" in the Journal of Risk adds the effects of skewness and kurtosis. For an accessible account of how smooth returns can hide risk, see Nassim Taleb's Fooled by Randomness.
Tags: Quant Trader Sharpe ratio Sortino ratio Drawdown Performance metrics