Lesson 3.3: Features and signals: turning data into forecasts
Learn to turn data into features and signals: z-scores, ranks, the information coefficient, the fundamental law, and combining signals without double counting.
By the end of this lesson you will be able to say the difference between a feature and a signal, turn a raw measure into a cross-sectional score with z-scores, ranks and winsorising, convert scores into market-neutral weights, measure a signal's predictive power with the information coefficient and a quintile test, use the fundamental law of active management to see how skill and breadth combine, and count how many independent signals you really have when you combine several. You will show that five stocks with momentum of 30, 12, 5, −8 and −20 per cent get weights of +37, +12, +2, −17 and −33 per cent that sum to zero, that an information coefficient of 0.03 over 400 independent bets gives an information ratio of 0.6, and that five signals with an average correlation of 0.4 carry the information of only 1.9 independent ones. You should have read Lessons 3.1 and 3.2 on data and on the pipeline. The mathematics is averages, standard deviations, correlation and square roots. This course is education about how markets and research work, not investment advice.
By now you have clean data and a pipeline to run it through. The next question is what to compute from the data. Prices and volumes on their own are not forecasts. A quant turns them into variables that might carry information about what happens next, tests whether they do, and then turns the survivors into positions. This lesson covers that middle part of the pipeline: the stages that Lesson 3.2 called features and signals.
Features and signals
A feature is a variable computed from data at a point in time: the return over the past twelve months, the ratio of a share's price to its earnings, the volatility of the last 20 days, the volume relative to its average. It describes something about the security. A signal, or alpha, is a forecast built from features: a score for each security at each date that says how much better or worse than average its return is expected to be. A signal is a feature, or a combination of features, that has been shown to be useful and has been scaled to be comparable across securities.
The families of features are well known, and most professional signals come from a handful of ideas:
Family · Typical feature · The idea behind itMomentum · Return over the past 6 to 12 months, skipping the latest month · Winners tend to keep winning for a while, because information is absorbed slowly or investors herd
Short-term reversal · Return over the past week or month · Over very short spans, price pressure from large orders tends to reverse
Value · Earnings, book value or sales relative to price · Cheap assets tend to earn more, as compensation for risk or from over-reaction
Quality · Profitability, low debt, stable earnings · Healthy firms are undervalued relative to their strength
Carry · Yield or roll return from holding the asset · Assets that pay more tend to earn more unless the price falls by as much
Volatility and risk · Recent volatility, beta · Low-risk assets have, in many studies, earned more than their risk would suggest
Volume and flow · Unusual volume, order imbalance · Informed trading leaves a footprint
Seasonality · Day of the week, month, event calendars · Regular patterns in demand and in information
A feature is worth testing only if you can say why it should work, and who is on the other side of the trade who is making a mistake or being paid to take a risk. A feature found by trying a thousand transformations and keeping the one that fits has no story, and as Lesson 3.4 shows, it is very likely a fluke.
Two practical rules apply to every feature. First, it must use only information available at the time, which the pipeline enforces with the lag of Lesson 3.2. Second, it should be stationary in the sense of Lesson 2.4: use returns, ratios and changes rather than price levels, so that its meaning does not drift over time.
Turning a feature into a score
Raw features have different units and scales. A momentum of 30 per cent and a price-to-earnings ratio of 18 cannot be compared or added. And what matters for a long-short portfolio is how a security compares with the others on the same day. The standard step is to convert the feature into a cross-sectional score: for each date, rescale the feature across all securities so that it has a mean of zero and a standard deviation of one.
Worked example 1. Five stocks have a trailing momentum, in per cent, of 30, 12, 5, −8 and −20. The mean is 3.8 and the standard deviation is 19.1. The z-scores, each stock's distance from the mean in standard deviations, are (30 − 3.8) ÷ 19.1 = 1.37, then 0.43, 0.06, −0.62 and −1.25. To turn scores into portfolio weights that are dollar-neutral, with the longs and shorts balancing, divide each score by the sum of the absolute scores, 3.73. The weights are +0.368, +0.115, +0.017, −0.166 and −0.334. They sum to zero, so the portfolio does not depend on the market going up, and the absolute weights sum to one, so the gross exposure is 100 per cent. The strongest stock gets the largest long, the weakest gets the largest short.
Ranks and outliers
The z-score has a weakness: one extreme value distorts everything. Suppose a sixth stock has a momentum of 250 per cent, perhaps from a takeover bid. The mean and standard deviation jump, and the z-scores of the other five collapse to between −0.15 and −0.64. They are now almost indistinguishable, while the outlier alone scores +2.01. Three defences are common.
Rank. Replace each value by its rank, from the lowest to the highest, and then centre the ranks. In the five-stock example the ranks give weights of +0.333, +0.167, 0, −0.167 and −0.333. A rank ignores how big the differences are, and so it cannot be thrown off by an outlier.
Winsorise. Clip values beyond a chosen limit, for example the 1st and 99th percentiles or ±3 standard deviations, before scoring.
Take logs or use a robust scale, such as the median and the MAD of Lesson 3.1, in place of the mean and the standard deviation.
Other common steps are to neutralise the score, removing its dependence on the market, a sector or the stock's size, so that the signal does not just say "own the cheapest sector", and to scale by volatility, so that a position in a wild stock is smaller than the same score in a calm one, as Lesson 2.2 suggested.
Does the signal predict anything? The information coefficient
A score is worth something only if it lines up with what happens next. The standard measure is the information coefficient, or IC: the correlation, across securities on one date, between the score and the next period's return. It runs from −1 to +1. A rank version, using the rank correlation, is common because it is robust to outliers.
Worked example 2. Eight stocks have scores of 1.5, 1.0, 0.5, 0.2, −0.2, −0.6, −1.0 and −1.4. Their returns over the next month, in per cent, turn out to be 2.0, 0.5, 1.5, −0.3, 0.4, −1.0, −0.2 and −1.8. The correlation between the two lists is 0.86, and the rank correlation is also 0.86. Treat that number with suspicion: it is an invented, tidy example. In real markets the IC of a good signal is small: typically between 0.02 and 0.05 for a monthly forecast of stock returns, and a signal with an IC above 0.10 is rare enough that one should first look for a bug. The signal is weak on any one date. Its value comes from being applied again and again, and across many securities.
The IC is measured at each date, and the sequence of ICs is what you study. Suppose the monthly IC averages 0.03 with a standard deviation of 0.10 over 60 months. The ratio of the two, 0.3, is the IC information ratio, and the t-ratio of the mean IC is 0.3 × √60 = 2.3. That is better than chance, but not overwhelming, and a researcher who has tried many signals must raise the bar, as Lesson 3.4 shows.
The quintile test
A second view is to sort the securities by score on each date into five equal groups, quintiles, and compute the average return of each group over the next period. A good signal shows returns rising across the groups, from the lowest quintile to the highest, and the difference between the top and bottom is the return of a long-short strategy. Look at the whole pattern, not only the spread. If the top group earns a lot and the others are flat, the signal is concentrated in a few names. If the middle group beats both ends, the relationship is not monotonic and the score may be measuring something else.
The fundamental law of active management
How good a strategy can you build from a signal with a small IC? The answer, given by Richard Grinold in 1989, is the fundamental law of active management:
IR ≈ IC × √N
where IR is the information ratio, the strategy's excess return divided by its tracking risk, and N is the breadth, the number of independent bets you make per year. The law says that your ratio of reward to risk grows with the skill of each bet and, more slowly, with the number of independent bets. It is a rule of thumb, with assumptions such as independent bets of equal skill, and it overstates what is achievable in practice, but it is useful for seeing how the pieces trade off.
IC · Independent bets a year · IR = IC × √N0.03 · 400 · 0.60
0.05 · 100 · 0.50
0.02 · 1,000 · 0.63
0.10 · 25 · 0.50
A small edge applied to many different bets can match a big edge on a few. This is why quant funds trade thousands of securities, and why a signal with an IC of 0.02 can be worth having. Skill is hard to find, and breadth is easier to add. And the IR connects to income: with a target tracking risk of 5 per cent a year, an IR of 0.6 implies an expected excess return of about 0.6 × 5 = 3 per cent a year.
The catch is the word independent. Holding 400 stocks is not 400 bets if they all move with the market, or if one theme drives them. Breadth is the number of independent bets, and it is usually much lower than the number of positions.
Combining signals: how many are really independent?
A single signal is fragile, so quants combine many. The simplest combination averages the scores. The benefit depends on how correlated the signals are with each other. Five signals that all say the same thing add little. The number of truly independent signals, the effective number, is
Neff = N ÷ (1 + (N − 1) × ρ)
where N is the number of signals and ρ is their average pairwise correlation. If the signals are independent, ρ = 0 and Neff = N. If they are perfectly correlated, ρ = 1 and Neff = 1.
Worked example 3. Five signals with an average correlation of 0.4 have Neff = 5 ÷ (1 + 4 × 0.4) = 5 ÷ 2.6 = 1.9. You paid for five signals and have the information of fewer than two. Ten signals with a correlation of 0.5 give 10 ÷ 5.5 = 1.8, almost no more than five at 0.4. Five signals with a correlation of only 0.1 give 5 ÷ 1.4 = 3.6. It is much better to add one genuinely different signal, with a low correlation to the rest, than five variants of the same idea. This is why researchers look for new sources of information, not new versions of the same one.
Signals meet costs: speed and decay
A signal has a horizon: how long its information lasts. A momentum score may stay useful for months, a short-term reversal for days, and an order-flow imbalance for minutes. As you trade on a signal, its predictive power decays, and you must decide how often to trade. Trading fast captures more of the signal and pays more in costs, trading slow does the opposite.
The test of whether a signal is worth trading is arithmetic from Lesson 1.4. Suppose a signal predicts an average gain of 8 basis points per trade, and the round-trip cost, the spread, fees and slippage together, is 6 basis points. The net edge is 2 basis points, a quarter of the gross, and a small error in the cost estimate would erase it. Always judge a signal after costs, and expect the cost estimate to be the weakest part. Slower trading lowers the cost per unit of signal, and often gives a higher net result, even though the gross is lower.
Practical checks for a new signal
1. Write the story first. State why it should work and who loses.
2. Check the timing. Run the truncation test of Lesson 3.2 on the signal.
3. Look at the IC over time, not only its average, and at its t-ratio.
4. Look at the quintiles. Is the pattern monotonic, and is it driven by a few names or a few dates?
5. Neutralise what you did not mean to bet on, such as the market, sectors and size, and test again.
6. Check it after costs at a realistic trading speed.
7. Test it in different periods and markets. A signal that worked in one decade and vanished in the next is a warning.
8. Check its correlation with your existing signals. A new signal that duplicates an old one adds little.
9. Count the signals you tried to find it, for Lesson 3.4.
The code below standardises a feature, makes dollar-neutral weights, computes a rank IC and applies the fundamental law and the effective-number formula.
python
import numpy as np
from scipy import stats
1. z-scores and market-neutral weights for five stocks
mom = np.array([30, 12, 5, -8, -20.0])
z = (mom - mom.mean()) / mom.std(ddof=1)
w = z / np.abs(z).sum()
print("z", z.round(2), "weights", w.round(3), "sum", round(w.sum(), 6), "gross", round(np.abs(w).sum(), 3))
rank weights are robust to an outlier
rank = stats.rankdata(mom)
dr = rank - rank.mean()
print("rank weights", (dr / np.abs(dr).sum()).round(3))
withoutlier = np.array([30, 12, 5, -8, -20, 250.0])
zo = (withoutlier - withoutlier.mean()) / withoutlier.std(ddof=1)
print("z with an outlier", zo.round(2), "clipped", np.clip(zo, -3, 3).round(2))
2. information coefficient
score = np.array([1.5, 1.0, 0.5, 0.2, -0.2, -0.6, -1.0, -1.4])
nxt = np.array([2.0, 0.5, 1.5, -0.3, 0.4, -1.0, -0.2, -1.8])
print("IC", round(np.corrcoef(score, nxt)[0, 1], 2), "rank IC", round(stats.spearmanr(score, nxt)[0], 2))
IC over time: mean 0.03, sd 0.10, 60 months
icmean, icsd, months = 0.03, 0.10, 60
print("IC information ratio", icmean / icsd, "t-ratio", round(icmean / icsd np.sqrt(months), 2))
3. fundamental law of active management
for ic, n in ((0.03, 400), (0.05, 100), (0.02, 1000), (0.10, 25)):
print("IC", ic, "bets", n, "IR", round(ic np.sqrt(n), 2), "excess return at 5% tracking risk", round(ic np.sqrt(n) 5, 1), "%")
4. effective number of signals
def neff(n, rho):
return n / (1 + (n - 1) rho)
for n, rho in ((5, 0.4), (10, 0.5), (5, 0.1)):
print(n, "signals, correlation", rho, "-> effective", round(neff(n, rho), 2))
5. net edge after costs
gross, cost = 8, 6
print("net edge (bp)", gross - cost)
Common mistakes
Mixing units. A raw momentum and a raw ratio cannot be added. Convert each to a score first.
Scoring with the whole sample. Compute means and standard deviations date by date, from data available then.
Letting one outlier set the scale. Use ranks, clipping or robust scaling.
Expecting a large IC. An IC of 0.03 is useful. An IC of 0.3 usually means a leak.
Judging a signal from the top quintile only. Look at the whole pattern and at the number of names that drive it.
Counting positions as bets. Four hundred stocks that all move with the market are a few bets.
Adding variants of the same idea. Correlated signals share information, and the effective number is far smaller than the count.
Ignoring decay and costs. A signal that works on paper at a daily horizon may not survive costs.
Starting from the data, not the story. A feature without a reason is most likely a fluke.
Forgetting to neutralise. A signal that is secretly a bet on one sector or on the market is not what you think it is.
Key takeaways
A feature is a variable computed from data at a point in time. A signal is a forecast built from features, scaled to compare securities. Most signals come from momentum, reversal, value, quality, carry, risk, volume or seasonality.
Turn a feature into a cross-sectional z-score, a rank or a winsorised score, then divide by the sum of absolute scores for dollar-neutral weights. The five-stock example gave weights of +0.368, +0.115, +0.017, −0.166 and −0.334.
One outlier can crush other stocks' z-scores into a narrow band. Ranks and clipping defend against it.
The information coefficient is the correlation between the score and the next return. Good signals have an IC of 0.02 to 0.05, and an IC above 0.10 deserves a search for bugs. A monthly IC of 0.03 with a standard deviation of 0.10 over 60 months has a t-ratio of 2.3.
The quintile test shows the average return of each group: it should rise from the lowest to the highest and not be driven by a few names.
The fundamental law says IR ≈ IC × √N. An IC of 0.03 over 400 independent bets gives 0.6, and an IC of 0.02 over 1,000 gives 0.63. Breadth means independent bets, not positions.
Combining signals helps only if they differ: Neff = N ÷ (1 + (N − 1)ρ), so five signals with ρ = 0.4 are 1.9 independent ones. Judge every signal after costs: 8 basis points of gain with 6 of cost is 2.
Check yourself
1. Four stocks have trailing returns of 20, 10, 0 and −10 per cent. What are their z-scores?
The mean is 5 and the sample standard deviation is 12.9. The z-scores are 1.16, 0.39, −0.39 and −1.16.
2. A signal has an IC of 0.04 and you make 250 independent bets a year. What is the approximate information ratio, and the excess return at 6 per cent tracking risk?
IR = 0.04 × √250 = 0.04 × 15.8 = 0.63. Excess return ≈ 0.63 × 6 = 3.8 per cent a year, before costs.
3. You hold 100 stocks, but they all fall and rise with the market. How many independent bets do you have, roughly?
Close to one, plus whatever independent information the stock-specific parts carry. The number of positions is not the breadth.
4. You have six signals with an average correlation of 0.5. What is the effective number?
6 ÷ (1 + 5 × 0.5) = 6 ÷ 3.5 = 1.7. Adding more copies of the same idea does very little.
5. A new signal shows an IC of 0.35 on its first test. What should you do before you celebrate?
Treat it as a probable bug. Run the truncation test for look-ahead, check the data for leaks, and test it on a different period. Real signals have an IC of a few hundredths.
6. A signal predicts 5 basis points a trade and costs 4 to trade. A friend says to trade it twice as often. Why might that be a bad idea?
Trading twice as often doubles the cost per period while the extra trades capture less of a decaying signal, so the net edge may fall below zero. Judge it after costs at each speed.
Try this
Set aside forty-five minutes. You need monthly returns for at least 30 liquid Indian shares over at least five years.
1. Compute a 12-month return skipping the latest month for each share at each month end. This is your momentum feature.
2. Convert the feature to a cross-sectional z-score each month, after clipping at ±3. Compare with a rank score.
3. Compute the next month's return, and the rank IC for each month. Plot the series, and compute its mean, standard deviation and t-ratio.
4. Sort into quintiles each month, compute the average next-month return of each, and chart them. Is the pattern monotonic?
5. Build dollar-neutral weights from the scores and compute the long-short return series. Compare its Sharpe ratio with its standard error from Lesson 1.3.
6. Use the first simulator with your IC and an honest count of independent bets, and compare the implied IR with your measured one.
7. Write a short note: the story for the signal, the IC and its t-ratio, the quintile pattern, how many signals you tried, and whether you would trade it after costs.
A good result is a chart of monthly ICs, a bar chart of the quintile returns and a statement of whether the signal is distinguishable from noise, with its uncertainty. The last lesson of this module asks the hardest question of all: after all the signals you tried, how likely is it that the best one is just luck?
Read more
Richard Grinold's 1989 article "The Fundamental Law of Active Management" in the Journal of Portfolio Management is the source of the law, and Grinold and Ronald Kahn's book Active Portfolio Management develops it. Narasimhan Jegadeesh and Sheridan Titman's 1993 paper "Returns to Buying Winners and Selling Losers" is the classic study of momentum, and Eugene Fama and Kenneth French's 1992 paper "The Cross-Section of Expected Stock Returns" is a foundation for value and size. For the practice of building and combining signals, see Zura Kakushadze and Juan Serur's 151 Trading Strategies and Marcos López de Prado's Advances in Financial Machine Learning.
Tags: Quant Trader Features Signals Information coefficient Fundamental law