Lesson 5.3: Statistical arbitrage and factor investing
Learn statistical arbitrage and factor investing: factor models and residuals, market-neutral portfolios, residual reversal, factor premia and crowding risk.
By the end of this lesson you will be able to split a stock's return into the part explained by common factors and a residual, build a portfolio that is neutral to the market, explain how statistical arbitrage trades the residuals of hundreds of stocks as many small bets, say what factor investing earns and why factor premia are small and unstable, and work out how combining factors with low correlation raises the Sharpe ratio. You will show that a stock that rose 3.1 per cent on a day when the market rose 2 per cent had 2.4 points explained by its beta of 1.2, that a long and short position of equal rupee value in stocks with betas of 1.4 and 0.8 still carries ₹60 lakh of market exposure per ₹1 crore, that a residual-reversal portfolio of 60 simulated stocks had a Sharpe ratio of 1.10 before costs, 0.98 when the market was not removed and 0.55 after 1 basis point per unit traded, and that two factors with Sharpe ratios of 0.4 each combine to 0.68 when their correlation is −0.3. You should have read Lessons 2.3, 3.3, 4.4, 5.1 and 5.2. The mathematics is regression, correlation and ratios. This course is education about how markets and research work, not investment advice.
Trend following and pairs trading each look at a few assets. This lesson widens the lens to the whole cross-section of stocks, hundreds or thousands at a time. Two families of strategies live here. Statistical arbitrage makes many small bets on the idiosyncratic behaviour of individual stocks, after the market and other common forces have been removed. Factor investing does the opposite: it deliberately takes exposure to the common characteristics, such as value or momentum, that have historically been rewarded. Both rest on the same device, the factor model, so we start there.
The factor model: splitting a return into parts
A factor model says that a stock's return is made of a part that comes from common forces, which affect many stocks at once, and a part that is specific to the stock:
return of stock i = α + β₁ × factor 1 + β₂ × factor 2 + ... + ε
Each β, the loading or exposure, says how much the stock moves with that factor. The α is the average return not explained by the factors. The residual ε is what is left: the stock's own surprise. Common factors in equities include:
Factor · What it capturesMarket · The return of the stock market as a whole
Size · Small companies against large ones
Value · Cheap stocks, by earnings or book value, against expensive ones
Momentum · Recent winners against recent losers
Quality · Profitable, stable companies against weak ones
Low volatility · Calm stocks against volatile ones
Sector or industry · The movement of a group, such as banks or software
Some factors are defined from economic characteristics, as above. Others are found by statistics, for example by principal components analysis, which extracts from the matrix of stock returns the few combinations that explain the most variation. The first usually looks like the market, the next ones like sectors or styles.
Worked example 1. On one day, a stock rises 3.1 per cent while the market rises 2.0 per cent. The stock has a market beta of 1.2, so the market explains 1.2 × 2.0 = 2.4 points. Suppose its value exposure is 0.3 and the value factor returns +0.5 per cent that day, adding 0.15 points. The unexplained residual is 3.1 − 2.4 − 0.15 = 0.55 points. Only that last piece is news about this particular stock. A statistical arbitrageur trades such residuals and a factor investor holds the exposures.
Neutral portfolios
To bet on residuals without also betting on the market, you build a portfolio that is neutral to the factors you do not want.
Dollar-neutral. The rupee value of the longs equals that of the shorts. It is a start but not enough, as the next example shows.
Beta-neutral. The weighted sum of betas is zero, so the portfolio does not move with the market.
Sector-neutral and style-neutral. The same for each sector or factor, so the portfolio holds no view about them.
Worked example 2. You buy ₹1 crore of stock A with a beta of 1.4 and sell ₹1 crore of stock B with a beta of 0.8. The position is dollar-neutral. But its market exposure is 1.4 × 1 crore − 0.8 × 1 crore = ₹0.6 crore, ₹60 lakh net long the market. If the market falls 5 per cent the portfolio loses about ₹3 lakh from that alone, whatever A and B do relative to each other. To be beta-neutral, you could sell ₹1.75 crore of B (1.4 ÷ 0.8 × 1 crore) so that 0.8 × 1.75 = 1.4, or sell ₹60 lakh of index futures against the pair.
Statistical arbitrage
Statistical arbitrage, or stat arb, is a family of strategies that trade a large number of stocks, long and short, expecting that the residual part of each stock's return will mean-revert, or that stocks that are alike will converge. The word arbitrage is generous: there is no riskless profit, only a statistical edge applied over very many small bets. A typical process:
1. Define a universe of hundreds or thousands of liquid stocks.
2. Estimate a factor model on past data: market, sector and perhaps statistical factors. Marco Avellaneda and Jeong-Hyun Lee's 2010 paper describes one built on sector funds and principal components.
3. Compute the residual of each stock, and a signal from it, such as the negative of its recent residual return or the z-score of its accumulated residual.
4. Form weights that are larger for stronger signals, neutral to the market and other factors, with limits on size and exposure.
5. Trade often and hold briefly, for days, with careful attention to costs.
6. Run it with leverage, since each bet has a small expected profit and a low volatility.
The strategy relies on breadth, the fundamental law of Lesson 3.3. With an information coefficient of only 0.02 but 500 independent stocks and bets every few days, the information ratio, IC × √N, can be large. The price is turnover, and so costs, which Lesson 4.4 showed are decisive.
Worked example 3. We built a simulated market of 60 stocks and ten years. Each stock's return is its beta times the market return, with betas drawn between 0.6 and 1.4, plus an idiosyncratic part. In that part, a price gap mean-reverts with a daily coefficient of 0.95, and a white noise of 1 per cent a day is added. Each day, the rule estimates each stock's beta from the previous 250 days, removes the market part from its last five days of returns to get the residual, and goes long the stocks whose residual fell most and short those that rose most, with weights summing to zero and gross exposure of 1. All inputs use only past data. We averaged ten simulated markets.
Version · Sharpe ratioReversal on residual returns, before costs · 1.10
Reversal on raw returns, with the market not removed · 0.98
Residual reversal after costs of 1 basis point per unit traded · 0.55
The portfolio's beta to the market was 0.00, as designed. Removing the market before forming the signal raised the Sharpe ratio from 0.98 to 1.10, because the raw returns include a beta times the market move, which adds noise unrelated to the stock's own reversal. At a gross exposure of 1 the strategy earned about 3.2 per cent a year with a volatility of 2.9 per cent, and it traded 0.64 times its gross exposure each day, about 162 times a year. At 1 basis point per unit traded, the cost is 162 × 1 basis point = 1.6 per cent a year, which cuts the 3.2 per cent return in half and the Sharpe ratio to 0.55. A real stat arb firm would lever such a book several times to reach a target volatility, which magnifies both the profit and the cost of financing, and would work hard to keep trading costs below 1 basis point per unit.
Factor investing
Where stat arb tries to remove the common factors, factor investing wants them. The idea is that exposure to certain characteristics has been rewarded over long periods, and that an investor can earn those factor premia systematically, at a lower cost and with more transparency than by paying a manager to pick stocks. The best known factors are:
Value: stocks that are cheap relative to earnings, book value or sales have earned more than expensive ones.
Momentum: recent winners have continued to beat recent losers, the cross-sectional momentum of Lesson 5.1.
Quality: profitable, stable firms with low debt have earned more.
Low volatility: calm stocks have earned about as much as volatile ones, so they have had better risk-adjusted returns.
Size: small companies have, in some periods and markets, outperformed large ones.
There are two broad explanations. One is risk: the factor earns a premium because it is risky in a way that matters to investors, for instance value stocks may do badly in bad times. The other is behaviour and constraints: investors extrapolate too much, or institutions cannot hold certain stocks, so mispricings persist. Neither is proven, which is why factors can stop working.
The honest facts about factor premia are these. They are small: a long-short factor typically has a Sharpe ratio well below 1, often in the range of a few tenths over long periods. They are volatile and cyclical: each factor goes through long stretches of poor performance, sometimes years, and timing them is hard. They decay as they become known and crowded, as Lesson 4.3's evidence on published predictors showed. And they cost money to harvest: the long-short versions need shorting and turnover, and the long-only versions give up part of the premium.
Combining factors
The best defence against a factor's bad years is to hold several that are not correlated. Suppose two factors each have a Sharpe ratio of 0.4, and you hold them in equal amounts of risk. The combined Sharpe ratio is
SR combined = (SR₁ + SR₂) ÷ √(2 × (1 + ρ))
where ρ is the correlation of their returns. Worked example 4. With ρ = −0.3 the combination is 0.8 ÷ √(2 × 0.7) = 0.8 ÷ 1.18 = 0.68, far above the 0.4 of either factor. With ρ = 0.5 it is 0.8 ÷ √3 = 0.46, barely better than one factor. Value and momentum have historically been negatively correlated, since cheap stocks tend to be past losers, which is one reason that combining them has been popular. The benefit comes from low correlation, as in the diversification of Lesson 5.1.
Risks: crowding, crashes and the model itself
Crowding and forced selling. If many funds hold similar positions, a shock that forces one to sell pushes prices against all of them, which forces more selling. In August 2007, several quantitative equity funds suffered large losses in a few days, with no news about the stocks involved. Amir Khandani and Andrew Lo studied the episode, and Lesson 9.4 returns to it. High leverage and similar signals made the funds vulnerable.
Factor crashes. A factor can lose a great deal quickly: momentum in a sharp rebound, value in a long growth-led rally.
Leverage. Because the returns per unit of gross exposure are low, stat arb is run with leverage, and leverage multiplies losses and the cost of financing.
Model risk. The factor model may be misspecified: the loadings are estimated with error, and they change. A residual that is really an unrecognised factor is not a residual at all.
Decay and crowding of signals. Short-term reversal and similar signals have been traded for decades, and the profit per trade has shrunk as costs have fallen and more capital has arrived.
Shorting constraints. Borrowing may be unavailable or costly, and in India, short positions held beyond a day are usually taken through single-stock futures, which exist only for a limited list of shares.
Data and survivorship. The universe must be as it was on each date, as in Lesson 3.1.
Building such a strategy: a checklist
1. Define the universe point-in-time, with liquidity filters.
2. Estimate the factor model on past data only, and check that the residuals are close to uncorrelated.
3. Build the signal from residuals or from factor characteristics, and test it as in Lesson 3.3.
4. Construct the portfolio with neutrality constraints, size limits and a cap on participation.
5. Include realistic costs and borrow and test with costs doubled.
6. Walk forward and keep a holdout (Lessons 3.4 and 4.3).
7. Monitor exposures live: market beta, sector, style, and the correlation of your book with common factors.
8. Plan for the bad day: know your leverage, your liquidity and the positions you would sell first.
The code below builds the simulated market, runs the residual and raw reversal strategies and prints the portfolio beta, together with the factor decomposition, the hedge and the factor combination arithmetic.
python
import numpy as np
from math import sqrt
def sharpe(x):
return x.mean() / x.std(ddof=1) sqrt(252)
def makepanel(seed, nstocks=60, n=2520):
"""Stock returns = beta market + an idiosyncratic part. The idiosyncratic price gap mean-reverts."""
rng = np.random.defaultrng(seed)
market = rng.normal(0.0003, 0.01, n)
beta = rng.uniform(0.6, 1.4, nstocks)
gap = np.zeros((nstocks, n))
shock = rng.normal(0, 0.0045, (nstocks, n))
for t in range(1, n):
gap[:, t] = 0.95 gap[:, t - 1] + shock[:, t]
idio = np.diff(gap, axis=1, prepend=0) + rng.normal(0, 0.01, (nstocks, n))
return beta[:, None] market[None, :] + idio, market
def betas(returns, market):
m = market - market.mean()
r = returns - returns.mean(axis=1, keepdims=True)
return (r m).sum(axis=1) / (m m).sum()
def run(returns, market, costbp=0.0, residual=True, lookback=5, window=250):
nstocks, n = returns.shape
pnl, prev = np.zeros(n), np.zeros(nstocks)
for t in range(window + 10, n):
b = betas(returns[:, t - window:t], market[t - window:t]) # beta from past data only
past = returns[:, t - lookback:t]
if residual:
past = past - b[:, None] market[t - lookback:t][None, :] # remove the market part
signal = -past.sum(axis=1) # reversal: buy recent losers
signal -= signal.mean() # dollar neutral
w = signal / np.abs(signal).sum() # gross exposure of 1
pnl[t] = w @ returns[:, t] - np.abs(w - prev).sum() costbp / 1e4
prev = w
return pnl[window + 10:]
rows = []
for seed in range(10):
returns, market = makepanel(seed)
gross, raw, net = run(returns, market), run(returns, market, residual=False), run(returns, market, costbp=1.0)
m = market[260:]
betaport = (gross (m - m.mean())).sum() / ((m - m.mean()) 2).sum()
rows.append([sharpe(gross), sharpe(raw), sharpe(net), betaport])
print("residual reversal Sharpe, raw reversal Sharpe, residual reversal net of 1 bp, portfolio beta")
print(np.round(np.mean(rows, axis=0), 2))
return decomposition (worked example 1)
marketpart, valuepart = 1.2 2.0, 0.3 0.5
print("market part", marketpart, "value part", valuepart, "residual", round(3.1 - marketpart - valuepart, 2))
dollar-neutral is not beta-neutral (worked example 2), amounts in lakh
longamt, longbeta, shortamt, shortbeta = 100, 1.4, 100, 0.8
netbetaexposure = longamt longbeta - shortamt shortbeta
print("net market exposure", netbetaexposure, "lakh | loss if the market falls 5%:", netbetaexposure -0.05, "lakh")
print("short of B needed for beta neutrality:", longamt longbeta / shortbeta, "lakh")
combining factors (worked example 4)
for rho in (-0.3, 0.5):
print("correlation", rho, "combined Sharpe", round((0.4 + 0.4) / sqrt(2 (1 + rho)), 2))
Common mistakes
Calling a dollar-neutral book market-neutral. Equal rupees long and short can still carry a large net beta.
Estimating betas and residuals with the whole sample. Use past data only.
Leaving costs out of a high-turnover strategy. In our simulation, 1 basis point halved the Sharpe ratio.
Treating a residual as pure noise or pure signal. If the factor model is missing a factor, the residual contains it.
Too few stocks. Breadth is the source of the Sharpe ratio, and a few names give a few bets.
Chasing a factor after a good run. Premia are cyclical and decay as they become known.
Holding several factors that are really one. Check the correlations, and count the effective number of independent factors.
Running high leverage without a plan for a crowded unwind. The loss comes when everyone does the same trade.
Ignoring shorting constraints and borrow costs. They can remove much of the profit of the short side.
Key takeaways
A factor model splits a return into factor parts and a residual. A stock up 3.1 per cent with a beta of 1.2 on a 2 per cent market day had 2.4 points from the market, 0.15 from value and 0.55 of residual.
Dollar-neutral is not beta-neutral. ₹1 crore long with a beta of 1.4 against ₹1 crore short with a beta of 0.8 carries ₹60 lakh of market exposure, so sell index futures or resize the short.
Statistical arbitrage trades the residuals of many stocks as a large number of small, neutral bets and relies on breadth, low costs and leverage.
In our simulated 60-stock market, residual reversal had a Sharpe ratio of 1.10 before costs, against 0.98 for raw reversal, and 0.55 after 1 basis point per unit traded, with a market beta of zero. It traded about 162 times its exposure a year.
Factor investing takes the exposures that have been rewarded: value, momentum, quality, low volatility and size. The premia are small, cyclical and decay as they become known.
Combining factors helps when correlation is low: two factors with a Sharpe ratio of 0.4 give 0.68 at a correlation of −0.3 and 0.46 at 0.5.
The main risks are crowding and forced unwinds, factor crashes, leverage, model error and shorting constraints.
Check yourself
1. A stock has a beta of 0.9. On a day when the market returns −2.0 per cent the stock returns −2.5 per cent. What is its residual, ignoring other factors?
The market explains 0.9 × (−2.0) = −1.8 per cent. The residual is −2.5 − (−1.8) = −0.7 per cent.
2. You hold ₹2 crore long in a stock with a beta of 1.1 and ₹1.5 crore short in one with a beta of 1.0. What is the net market exposure, and how much index should you sell to neutralise it?
Net = 2 × 1.1 − 1.5 × 1.0 = 2.2 − 1.5 = ₹0.7 crore net long. Sell ₹0.7 crore of index futures.
3. Two factors have Sharpe ratios of 0.5 and 0.3 and a correlation of 0.1. What is the equal-risk combination?
(0.5 + 0.3) ÷ √(2 × 1.1) = 0.8 ÷ 1.48 = 0.54.
4. A stat arb book earns 3 per cent a year gross and trades 150 times its exposure a year. At what cost per unit traded does the net return fall to zero?
Annual cost = 150 × cost = 3 per cent, so cost = 0.02 per cent = 2 basis points per unit traded.
5. Why might a factor that has worked for 30 years stop working?
It may have been a reward for a risk that has changed, or a mispricing that was competed away once it was published and crowded, and its history may include luck. Factors also have long cyclical stretches of poor performance.
Try this
Set aside forty-five minutes. You need daily returns for at least 50 liquid Indian stocks and an index, for at least five years.
1. For each stock, regress its daily returns on the index returns over a rolling 250-day window using past data, and record the betas.
2. Compute five-day residual returns for each stock, by subtracting beta times the index return.
3. Each day, rank the stocks by their last five-day residual, go long the bottom quintile and short the top quintile with equal weights, and record the profit and loss. Compare with a version that uses raw returns.
4. Compute the portfolio's beta to the index, its Sharpe ratio, its turnover and the effect of a cost of 1, 2 and 5 basis points.
5. Use the hedge simulator to check that a dollar-neutral version of one day's portfolio is or is not beta-neutral.
6. Build two simple factors, such as 12-month momentum and a value ratio. Compute their long-short returns and their correlation, and use the factor simulator to see the combination.
7. Write a short note: the universe, the signal, the Sharpe ratio before and after costs, the beta, and three risks that you would monitor.
A good result is a market beta near zero, a Sharpe ratio before and after costs with the turnover next to it, and a clear statement of how much leverage would be needed to run the book. The last lesson of this module goes to the firm on the other side of every trade in this course: the market maker, and the art of executing well.
Read more
Marco Avellaneda and Jeong-Hyun Lee's 2010 paper "Statistical Arbitrage in the US Equities Market" in Quantitative Finance is the standard practical reference. Eugene Fama and Kenneth French's 1993 paper "Common Risk Factors in the Returns on Stocks and Bonds" in the Journal of Financial Economics introduced the three-factor model, and Clifford Asness, Tobias Moskowitz and Lasse Pedersen's 2013 paper "Value and Momentum Everywhere" in the Journal of Finance studies the two factors across markets. Amir Khandani and Andrew Lo's 2011 paper "What Happened to the Quants in August 2007?" in the Journal of Financial Markets analyses the crowding episode. Andrew Ang's Asset Management: A Systematic Approach to Factor Investing is a thorough text.
Tags: Quant Trader Statistical arbitrage Factor investing Market neutral Residual returns