Three agents invent equity signals, one harness judges them all, and a statistics layer decides whether any of it is real. So far nothing farmed has cleared the bar — which is the interesting part.
Published
August 22, 2026
The question
A language model has read an enormous amount of finance. Working papers, textbooks, factor zoo surveys, decades of published anomalies. So here is a question worth asking directly:
Can you farm tradable equity signals out of that knowledge?
Not “can it explain momentum” — obviously it can. The question is whether it can propose a new cross-sectional predictor, one that survives a ruthless backtest it was not allowed to influence.
The honest way to answer that is to build the ruthless backtest first, and only then let the model start guessing. That inversion is the whole project. It is easy to build a pipeline where the model proposes ideas and you eyeball the Sharpe ratios and get excited. It is harder, and much more useful, to build one where the model’s ideas are judged by machinery that cannot be talked into anything.
This post is about that machinery: what it does, why each piece is there, and how you would write a signal for it yourself.
What “farming” means here
The setup is a division of labour between three agents and one harness.
Code
%%{init: {'theme': 'dark', 'themeVariables': {'primaryColor': '#0f3460', 'primaryTextColor': '#ffffff', 'primaryBorderColor': '#e94560', 'lineColor': '#a0a8c0', 'secondaryColor': '#1a1a2e', 'tertiaryColor': '#16213e', 'clusterBkg': '#16213e', 'clusterBorder': '#e94560', 'edgeLabelBackground': '#1a1a2e', 'fontFamily': 'inherit'}}}%%flowchart TD FRONTIER[("Research frontier\nwhat the library holds\nwhich themes are thin")] subgraph AGENTS["The creative half — costs money, can be wrong"] IDEA["Idea agent · Opus 5\nhypothesis + mechanism\nformula, sign, references"] DATA["Data agent · Sonnet 5\nmap concepts to real columns\ngate on coverage"] CODER["Coder agent · Sonnet 5\nwrite the Signal module\nself-correct on failure"] IDEA --> DATA --> CODER end subgraph HARNESS["The judging half — deterministic, identical for everyone"] VAL["No-look-ahead validator"] BT["Decile long/short backtest"] ROB["Robustness battery"] STAT["Factor attribution\nDeflated Sharpe · BH-FDR"] VAL --> BT --> ROB --> STAT end LEDGER[("Ledger\nevery attempt, including\nthe ones that died")] FRONTIER --> IDEA CODER --> VAL STAT --> LEDGER LEDGER -.->|raises the bar| STAT STAT --> ARTICLE["Published results"]
%%{init: {'theme': 'dark', 'themeVariables': {'primaryColor': '#0f3460', 'primaryTextColor': '#ffffff', 'primaryBorderColor': '#e94560', 'lineColor': '#a0a8c0', 'secondaryColor': '#1a1a2e', 'tertiaryColor': '#16213e', 'clusterBkg': '#16213e', 'clusterBorder': '#e94560', 'edgeLabelBackground': '#1a1a2e', 'fontFamily': 'inherit'}}}%%
flowchart TD
FRONTIER[("Research frontier\nwhat the library holds\nwhich themes are thin")]
subgraph AGENTS["The creative half — costs money, can be wrong"]
IDEA["Idea agent · Opus 5\nhypothesis + mechanism\nformula, sign, references"]
DATA["Data agent · Sonnet 5\nmap concepts to real columns\ngate on coverage"]
CODER["Coder agent · Sonnet 5\nwrite the Signal module\nself-correct on failure"]
IDEA --> DATA --> CODER
end
subgraph HARNESS["The judging half — deterministic, identical for everyone"]
VAL["No-look-ahead validator"]
BT["Decile long/short backtest"]
ROB["Robustness battery"]
STAT["Factor attribution\nDeflated Sharpe · BH-FDR"]
VAL --> BT --> ROB --> STAT
end
LEDGER[("Ledger\nevery attempt, including\nthe ones that died")]
FRONTIER --> IDEA
CODER --> VAL
STAT --> LEDGER
LEDGER -.->|raises the bar| STAT
STAT --> ARTICLE["Published results"]
The idea agent runs on the most capable model available, because generating a genuinely novel economic hypothesis is the one step where capability plausibly matters. It gets a research frontier — what is already in the library, which categories and data sources are thin — and is told to go somewhere new.
The data agent runs on a cheaper model and does something unglamorous but essential: map each requested concept onto a column that actually exists, and kill the idea if it doesn’t. On the first live run this gate caught the idea agent inventing a column called macro_cpi_chg12m_scale out of thin air. No such column. The hypothesis died before a line of code was written, at a cost of about two cents.
The coder agent writes the actual module, then self-corrects against the validator’s complaints for up to three attempts. Its output is treated as untrusted: a static check rejects file, network and process access before the module is ever imported.
Then everything hits the same harness. That part has no model in it at all.
What a signal actually is
This is the concept everything else rests on, and it is simpler than it sounds.
A cross-sectional signal is one number per stock per month. That’s it. Not a buy/sell decision, not a price target — just a score.
The crucial property: only the ranking matters. The absolute level is meaningless. A signal that outputs book-to-market ratios between 0.1 and 8, and a signal that outputs their logarithms, are the same signal. All the harness ever does with your number is sort by it.
Each month, mechanically:
Take every eligible stock — liquid, above $5, common stock, real exchange.
Compute the score for each one.
Sort. Long the top 10%, short the bottom 10%, equal-weighted.
Hold for one month. Earn the spread.
Repeat, 280 times, from 2000 to 2024.
If the ranking carries information about next month’s returns, the spread is positive on average and compounds. If it doesn’t, you get a random walk with transaction costs bolted on. That’s the entire test.
Orientation, and why the interface has an expected_sign
There’s a design wrinkle worth explaining because it shapes how you write a signal.
Some predictors work the “right” way up: high gross profitability predicts high returns. Others are inverted: high asset growth predicts low returns, and so does high volatility. If every signal had to be written so that “high = good”, you’d be constantly writing -volatility or 1/asset_growth, and guarding the division, and getting the sign wrong at 2am.
So the interface separates the two concerns. You write the natural formula, and declare which way it points:
class LowVolatility(BaseSignal): name ="low_volatility" category ="low_risk" expected_sign =-1# LOW vol predicts HIGH return description ="Low-volatility anomaly: trailing 1y realized vol (low = long)."def compute(self, panel):return panel["vol_252"] # the natural quantity, not its negation
The harness multiplies by expected_sign before ranking, so after orientation “high score = go long” always holds. You get to write vol_252 and mean it.
There is a second archetype too. A strategy_type = "macro_timing" signal returns one number per date rather than per stock — a desired market exposure. The harness standardizes it using only past data, maps it to a position in [−1, 1], and earns that fraction of the market. Same interface, different shape.
A worked example: one month, one signal
The description above is abstract enough that it is worth watching a single month go through the machine. Take inventory_growth_gap, one of the farmed signals. The hypothesis: if a company’s warehouse is filling up faster than its sales are growing, it over-forecast demand, and write-downs and discounts are coming. The formula is a gap between two growth rates:
expected_sign = -1, because it is the large gap that is the bad case.
Now one month-end. Twenty eligible names, which is small enough to print — the real cross-section runs 1,500 to 3,000 — sorted by the oriented score:
Ticker
raw (inv − sales growth)
oriented alpha
decile
weight
TTT
−0.55
0.55
9
+0.50
SSS
−0.31
0.31
9
+0.50
RRR
−0.24
0.24
8
0
QQQ
−0.19
0.19
8
0
…
…
…
…
0
DDD
0.25
−0.25
1
0
CCC
0.33
−0.33
1
0
BBB
0.41
−0.41
0
−0.50
AAA
0.62
−0.62
0
−0.50
Four things happened in that table, and they are the entire trading rule.
Orientation flipped the sign. AAA has the worst inventory bloat — stock piling up 62 percentage points faster than sales — and the multiplication by expected_sign = -1 sends it to the bottom of the sort. TTT, running the leanest inventory, goes to the top. After this step the harness never has to know what the signal measures; it only knows that high means long.
The sort became buckets. Ranks are converted to deciles, using average ranks so that tied scores stay together. Ranking by first-occurrence instead would have broken ties by the panel’s ticker order, i.e. alphabetically, which is a genuinely strange thing to trade on. The scores are also winsorized at 1% first — that clips outlier values so a company whose inventory tripled cannot distort anything downstream, while leaving its rank, and therefore its decile, untouched.
Buckets became weights. Equal weight within each leg: two longs at +50%, two shorts at −50%. In the real cross-section that is roughly 200 names a side at ±0.5% each. The book is 100% long and 100% short — dollar neutral, though nothing here forces the two legs to have matching market exposure. The middle eight deciles, 80% of the market, are simply not traded.
Weights met next month’s returns. Say the two longs return +3% and +6% and the two shorts return −4% and +1%. The long leg earns the average, +4.5%. The bottom decile’s own return averages −1.5%, and since it was held short, subtracting it adds to the spread: +6.0% gross for the month. Then turnover is charged. If the entire book had to be rebuilt from scratch, the traded notional is 2.0 and the cost at 10 bps is 20 bps for that month — which is why a slow, fundamentals-driven signal like this one and a fast monthly-reversal signal are not remotely competing on equal terms.
Repeat 280 times and that stream of monthly numbers is what every statistic in the rest of this post is computed from.
The part that matters: point-in-time correctness
Every backtest is a claim about what you would have known at the time. Almost all the ways to get that wrong are subtle.
Fundamentals join on the filing date, never the fiscal period. A company’s Q3 figures describe September but become public in November. Joining on the fiscal date hands your backtest two months of clairvoyance and produces beautiful, worthless results.
The universe keeps delisted names. This one bites in a way that’s easy to miss. When a company is acquired or goes bankrupt it stops appearing in the data — so its final month has no “next month” to compute a return from, and the naive fix is to drop the row. But dropping it silently removes the worst outcomes from whichever decile was holding them. In this dataset, 12,754 of roughly 19,800 tickers end before the sample does. That is not a rounding error; it is a systematic upward bias aimed squarely at the cheap, distressed names value signals love. The harness books an explicit delisting return instead of letting those rows vanish.
And every signal has to prove it isn’t cheating. Before anything is backtested, the validator recomputes it on a time-truncated panel and compares:
Recomputing a signal with the last 40% of history deleted must not change any earlier value — or even its missing/not-missing status.
That single check catches the entire family of look-ahead bugs. A full-sample z-score fails it. A .fillna() that propagates backwards fails it. A negative shift fails it. And notice that the NaN-status clause matters as much as the values: a signal that quietly becomes computable earlier when it can see the future is leaking, even if the numbers it produces look plausible.
The part nobody builds: deciding whether the result is real
Here is the failure mode this whole project is designed around.
Suppose you generate a hundred signals, backtest them all, and report the best one. Its Sharpe ratio is not an estimate of its quality. It’s an estimate of how many things you tried. Run enough random strategies against the same 280 months and one of them will look excellent, guaranteed, with no alpha anywhere in sight.
So the leaderboard is deliberately not sorted by Sharpe. Three things sit on top of it.
Factor attribution. Every signal’s returns are regressed on market, size, value and momentum — factors built from the same panel, so the comparison is like for like — with Newey-West standard errors. What matters is the intercept: the return the known premia cannot explain. This is brutally clarifying. The library’s momentum_12_1 signal has a factor R² of 0.92 and an alpha t-stat of −0.10. It isn’t a discovery. It is the momentum factor, rediscovered, and it should be paid for exactly once.
A search hurdle. Given how dispersed the Sharpes in the library are, you can compute the Sharpe you’d expect from the best of N random strategies. That’s the bar luck alone clears. It is not a constant: it is recomputed on every run from the ledger’s attempt count and the spread of Sharpes across the library, so it climbs as the search grows. Anything below it is not evidence of anything. The results page carries its current value.
Deflated Sharpe and Benjamini-Hochberg. The deflated Sharpe is the probability a signal is real given the size of the search that found it. The BH q-value controls the false-discovery rate across the whole library at once. The denominator for both comes from the ledger, which records every attempt — including the ones that died at the data gate or failed to compile. Counting only survivors would understate the search and flatter every number on the page.
This machinery has a property I’ve come to appreciate: it gets harder to satisfy the more you use it. Every farming run adds trials, which raises the hurdle for everything already in the library. A search large enough will push its own earlier discoveries back below significance. That is not a bug — it is the honest accounting finally showing up on the balance sheet.
What it found
There is deliberately no scoreboard in this post. The campaign runs daily and the library grows most nights, so any count printed here would be stale within a week. Worse, a count is the wrong thing to fixate on: in this setup the number of attempts is a cost, not an achievement, because every trial raises the bar for everything already found. The results page regenerates itself on every run and carries the current figures, hurdle included.
What is worth writing down is the shapes that keep recurring, because those have been stable since the first campaign day.
Nothing farmed has cleared the search hurdle. Not once, not yet. The only signal in the library that clears it is gross_profitability — Novy-Marx’s, published in 2013, and sitting in the library before any agent ran. Even that one is labelled a candidate rather than a survivor, because clearing the hurdle and surviving multiple-testing correction with a high deflated Sharpe are two different tests, and it passes one of them.
Everything else works, which is the point. Every farmed signal compiled. Every one passed the no-look-ahead validator. Every one arrived with a coherent economic story attached — inventory growth gaps, stock-comp dilution, oil-beta interactions, turn-of-year tax-loss reversals. The pipeline is not failing. The hypotheses are just wrong, in the ordinary way that most hypotheses are wrong.
The strongest t-statistics point the wrong way. This one I did not expect. Several farmed signals are comfortably significant under Benjamini-Hochberg — and negative. attention_spike_return_reversal posts a t-stat of −5.3 and filing_staleness_information_neglect −4.4, both with Sharpes well below zero. The agent found real cross-sectional structure and then declared the sign backwards. It is tempting to read that as a discovery in disguise: flip expected_sign and you have a strong signal. Resist it. Choosing the sign after seeing the result is just another trial, and it is the cheapest way there is to fool yourself. If the mechanism argument only works once you know the answer, it was never a mechanism argument.
The search keeps rediscovering itself. Left alone, the idea agent converges. The library holds a cluster of same-month seasonality variants that are recognisably one anomaly wearing different names, and pairs like rate_duration_mispricing and hidden_rate_duration_mismatch whose return streams are nearly identical — a novelty score of 0.02 between them. This is worse than wasted spend. Near-duplicates manufacture trials, trials raise the hurdle, and the library ends up penalising its own genuine entries for the sin of being asked the same question twice. Deduplication is not housekeeping here; it is part of the statistics.
I want to be precise about what this does and doesn’t show. It is a fixed dataset, a fixed universe, and one family of ideas. It is not proof that the approach can’t work. But it is a real result, and it lines up with the thing quantitative researchers already know and outsiders consistently underestimate: plausible is cheap. The hard part of finding alpha was never generating hypotheses. A language model generates a dozen excellent-sounding ones for under a dollar. The hard part is that almost all hypotheses are wrong, and the only way to know which is to test them in a way that can tell you no.
One entry is still worth watching. market_breadth_high_ratio_timing posts a Sharpe of just 0.12 — nothing — but an alpha t-stat of 2.64. Real explanatory power the factor model misses, wrapped in a construction too weak to trade. That is the shape a genuine discovery would first appear in, and it is exactly the shape a Sharpe-sorted leaderboard would throw away.
How you’d implement a signal
Everything above exists so this part can be short. Here is the whole contract:
from __future__ import annotationsimport pandas as pdfrom llm_alpha.signals.base import BaseSignalclass GrossMarginTrend(BaseSignal): name ="gross_margin_trend" category ="quality" expected_sign =1 description ="Year-over-year change in gross margin: improving pricing power."def compute(self, panel: pd.DataFrame) -> pd.Series: gm = panel["grossmargin"] gm_prior = gm.groupby(level="ticker").shift(12)return gm - gm_prior
That’s a complete, working signal. Four class attributes and one method. Walking through what each part is doing:
The panel is a DataFrame indexed by (date, ticker) — one row per stock per month-end, about 1.9 million of them. It carries price-derived features (ret_12_1, vol_252, beta_252, high_ratio), daily valuation ratios (pe, pb, marketcap), and point-in-time quarterly fundamentals (revenue, assets, gp, ncfo, grossmargin, …) already joined as of their filing date. You never load data yourself; it’s all there.
compute returns a Series indexed exactly like the panel. Same index, one score per row. Return NaN wherever the signal is undefined — the harness will simply exclude those names from that month’s cross-section. Missing data is expected and handled; do not paper over it.
groupby(level="ticker").shift(12) is the single most important idiom, and the one the coder agent most often gets wrong on its first attempt. It means “this ticker’s value twelve months ago”. The groupby is not optional: without it, shift walks off the end of one stock and into another, silently comparing Apple to Amgen. A negative shift would reach into the future and the validator will reject it.
Guard your denominators. The reference implementations do this everywhere:
.where(assets > 0) yields NaN where the denominator is non-positive, rather than inf or a sign-flipped nonsense value. A single inf can dominate a cross-sectional ranking and quietly wreck a month.
Then set expected_sign honestly and don’t pre-negate anything. If you believe low values predict high returns, write the natural quantity and set -1.
Drop that file into signals/library/, and the rest is automatic: the registry discovers it, the validator truncates the panel and checks it for look-ahead, the harness backtests it identically to every other signal, the robustness battery sweeps costs and sub-periods and universes, the factor model tells you whether it’s new, and the deflation layer tells you whether to believe it.
You do not get to tune any of that. Which is the point — the harness was built before the hypotheses, precisely so it couldn’t be.
What I’d do next
The bottleneck is not the pipeline, and it is fairly clearly not the model’s imagination either. It is the hypothesis space, which is bounded by what the panel contains — prices, fundamentals, a handful of macro series.
Running the same agent against the same columns another thousand times mostly manufactures trials, and every trial raises the bar for everything already found. Adding a genuinely new data source — short interest and borrow cost, 13F ownership churn, analyst revisions and dispersion, options-implied skew — multiplies the space instead of re-searching it. The other half of the same problem is the duplicate clusters above: a correlation gate that refuses a proposal too close to something already on the ledger would buy back hurdle for free, without needing a single new idea.
There is a second direction, and it is about the harness rather than the ideas. The decile long/short construction in the worked example above is deliberately crude: every name in the top decile gets the same weight regardless of how strongly the signal likes it, and every name outside the two extreme deciles gets nothing. That is the right default for judging a signal — it is transparent, it is the standard academic construction, and it cannot be tuned into looking good — but it is not how anyone would actually hold the thing.
So two variants are on the list. Position sizing: weight by signal strength or by inverse volatility instead of equally, so that conviction and risk both show up in the book rather than being flattened away. And a long-only version, because the short leg is where the borrow costs, the crowding and the delisting assumptions do their real damage, and because most of the capital that would ever run something like this cannot short at all. Both change the answer, and neither is guaranteed to change it upwards — a signal whose spread lives entirely in the short leg looks very different once you take the short leg away. That is precisely what makes it worth measuring.
That’s the next move: fewer, better-fed guesses rather than more of the same. The campaign keeps running daily, and the results page regenerates itself on every run, hurdle and all.
Source Code
---title: "Farming Alpha Out of a Language Model"description: "Three agents invent equity signals, one harness judges them all, and a statistics layer decides whether any of it is real. So far nothing farmed has cleared the bar — which is the interesting part."date: "2026-08-22"categories: [finance, quant, ai-agent, llm, meta]image: ""---## The questionA language model has read an enormous amount of finance. Working papers, textbooks,factor zoo surveys, decades of published anomalies. So here is a question worthasking directly:> Can you *farm* tradable equity signals out of that knowledge?Not "can it explain momentum" — obviously it can. The question is whether it canpropose a **new** cross-sectional predictor, one that survives a ruthless backtestit was not allowed to influence.The honest way to answer that is to build the ruthless backtest first, and onlythen let the model start guessing. That inversion is the whole project. It is easyto build a pipeline where the model proposes ideas and you eyeball the Sharperatios and get excited. It is harder, and much more useful, to build one where themodel's ideas are judged by machinery that cannot be talked into anything.This post is about that machinery: what it does, why each piece is there, and howyou would write a signal for it yourself.## What "farming" means hereThe setup is a division of labour between three agents and one harness.```{mermaid}%%{init: {'theme': 'dark', 'themeVariables': {'primaryColor': '#0f3460', 'primaryTextColor': '#ffffff', 'primaryBorderColor': '#e94560', 'lineColor': '#a0a8c0', 'secondaryColor': '#1a1a2e', 'tertiaryColor': '#16213e', 'clusterBkg': '#16213e', 'clusterBorder': '#e94560', 'edgeLabelBackground': '#1a1a2e', 'fontFamily': 'inherit'}}}%%flowchart TD FRONTIER[("Research frontier\nwhat the library holds\nwhich themes are thin")] subgraph AGENTS["The creative half — costs money, can be wrong"] IDEA["Idea agent · Opus 5\nhypothesis + mechanism\nformula, sign, references"] DATA["Data agent · Sonnet 5\nmap concepts to real columns\ngate on coverage"] CODER["Coder agent · Sonnet 5\nwrite the Signal module\nself-correct on failure"] IDEA --> DATA --> CODER end subgraph HARNESS["The judging half — deterministic, identical for everyone"] VAL["No-look-ahead validator"] BT["Decile long/short backtest"] ROB["Robustness battery"] STAT["Factor attribution\nDeflated Sharpe · BH-FDR"] VAL --> BT --> ROB --> STAT end LEDGER[("Ledger\nevery attempt, including\nthe ones that died")] FRONTIER --> IDEA CODER --> VAL STAT --> LEDGER LEDGER -.->|raises the bar| STAT STAT --> ARTICLE["Published results"]```The **idea agent** runs on the most capable model available, because generating agenuinely novel economic hypothesis is the one step where capability plausiblymatters. It gets a research frontier — what is already in the library, whichcategories and data sources are thin — and is told to go somewhere new.The **data agent** runs on a cheaper model and does something unglamorous butessential: map each requested concept onto a column that actually exists, and killthe idea if it doesn't. On the first live run this gate caught the idea agentinventing a column called `macro_cpi_chg12m_scale` out of thin air. No such column.The hypothesis died before a line of code was written, at a cost of about two cents.The **coder agent** writes the actual module, then self-corrects against thevalidator's complaints for up to three attempts. Its output is treated asuntrusted: a static check rejects file, network and process access before themodule is ever imported.Then everything hits the same harness. That part has no model in it at all.## What a signal actually isThis is the concept everything else rests on, and it is simpler than it sounds.A **cross-sectional signal** is one number per stock per month. That's it. Not abuy/sell decision, not a price target — just a score.The crucial property: **only the ranking matters.** The absolute level ismeaningless. A signal that outputs book-to-market ratios between 0.1 and 8, and asignal that outputs their logarithms, are the same signal. All the harness everdoes with your number is sort by it.Each month, mechanically:1. Take every eligible stock — liquid, above $5, common stock, real exchange.2. Compute the score for each one.3. Sort. Long the top 10%, short the bottom 10%, equal-weighted.4. Hold for one month. Earn the spread.5. Repeat, 280 times, from 2000 to 2024.If the ranking carries information about next month's returns, the spread ispositive on average and compounds. If it doesn't, you get a random walk withtransaction costs bolted on. That's the entire test.### Orientation, and why the interface has an `expected_sign`There's a design wrinkle worth explaining because it shapes how you write a signal.Some predictors work the "right" way up: high gross profitability predicts highreturns. Others are inverted: high asset growth predicts *low* returns, and so doeshigh volatility. If every signal had to be written so that "high = good", you'd beconstantly writing `-volatility` or `1/asset_growth`, and guarding the division,and getting the sign wrong at 2am.So the interface separates the two concerns. You write the **natural** formula, anddeclare which way it points:```pythonclass LowVolatility(BaseSignal): name ="low_volatility" category ="low_risk" expected_sign =-1# LOW vol predicts HIGH return description ="Low-volatility anomaly: trailing 1y realized vol (low = long)."def compute(self, panel):return panel["vol_252"] # the natural quantity, not its negation```The harness multiplies by `expected_sign` before ranking, so after orientation"high score = go long" always holds. You get to write `vol_252` and mean it.There is a second archetype too. A `strategy_type = "macro_timing"` signal returnsone number *per date* rather than per stock — a desired market exposure. Theharness standardizes it using only past data, maps it to a position in [−1, 1], andearns that fraction of the market. Same interface, different shape.### A worked example: one month, one signalThe description above is abstract enough that it is worth watching a single monthgo through the machine. Take `inventory_growth_gap`, one of the farmed signals.The hypothesis: if a company's warehouse is filling up faster than its sales aregrowing, it over-forecast demand, and write-downs and discounts are coming. Theformula is a gap between two growth rates:```pythoninv_g = inventory / inventory_12mo_ago -1sales_g = revenue / revenue_12mo_ago -1raw = inv_g - sales_g````expected_sign = -1`, because it is the *large* gap that is the bad case.Now one month-end. Twenty eligible names, which is small enough to print — thereal cross-section runs 1,500 to 3,000 — sorted by the oriented score:| Ticker | `raw` (inv − sales growth) | oriented alpha | decile | weight ||---|---:|---:|---:|---:|| TTT | −0.55 | 0.55 | 9 | **+0.50** || SSS | −0.31 | 0.31 | 9 | **+0.50** || RRR | −0.24 | 0.24 | 8 | 0 || QQQ | −0.19 | 0.19 | 8 | 0 || … | … | … | … | 0 || DDD | 0.25 | −0.25 | 1 | 0 || CCC | 0.33 | −0.33 | 1 | 0 || BBB | 0.41 | −0.41 | 0 | **−0.50** || AAA | 0.62 | −0.62 | 0 | **−0.50** |Four things happened in that table, and they are the entire trading rule.**Orientation flipped the sign.** AAA has the worst inventory bloat — stock pilingup 62 percentage points faster than sales — and the multiplication by`expected_sign = -1` sends it to the bottom of the sort. TTT, running the leanestinventory, goes to the top. After this step the harness never has to know what thesignal measures; it only knows that high means long.**The sort became buckets.** Ranks are converted to deciles, using *average* ranksso that tied scores stay together. Ranking by first-occurrence instead would havebroken ties by the panel's ticker order, i.e. alphabetically, which is a genuinelystrange thing to trade on. The scores are also winsorized at 1% first — that clipsoutlier *values* so a company whose inventory tripled cannot distort anythingdownstream, while leaving its rank, and therefore its decile, untouched.**Buckets became weights.** Equal weight within each leg: two longs at +50%, twoshorts at −50%. In the real cross-section that is roughly 200 names a side at±0.5% each. The book is 100% long and 100% short — dollar neutral, though nothinghere forces the two legs to have matching market exposure. The middle eightdeciles, 80% of the market, are simply not traded.**Weights met next month's returns.** Say the two longs return +3% and +6% and thetwo shorts return −4% and +1%. The long leg earns the average, +4.5%. The bottomdecile's own return averages −1.5%, and since it was held short, subtracting itadds to the spread: **+6.0% gross** for the month. Then turnover is charged. If theentire book had to be rebuilt from scratch, the traded notional is 2.0 and the costat 10 bps is 20 bps for that month — which is why a slow, fundamentals-drivensignal like this one and a fast monthly-reversal signal are not remotely competingon equal terms.Repeat 280 times and that stream of monthly numbers is what every statistic in therest of this post is computed from.## The part that matters: point-in-time correctnessEvery backtest is a claim about what you *would have known* at the time. Almost allthe ways to get that wrong are subtle.**Fundamentals join on the filing date, never the fiscal period.** A company's Q3figures describe September but become public in November. Joining on the fiscaldate hands your backtest two months of clairvoyance and produces beautiful,worthless results.**The universe keeps delisted names.** This one bites in a way that's easy to miss.When a company is acquired or goes bankrupt it stops appearing in the data — so itsfinal month has no "next month" to compute a return from, and the naive fix is todrop the row. But dropping it silently removes the worst outcomes from whicheverdecile was holding them. In this dataset, 12,754 of roughly 19,800 tickers endbefore the sample does. That is not a rounding error; it is a systematic upwardbias aimed squarely at the cheap, distressed names value signals love. The harnessbooks an explicit delisting return instead of letting those rows vanish.**And every signal has to prove it isn't cheating.** Before anything is backtested,the validator recomputes it on a time-truncated panel and compares:> Recomputing a signal with the last 40% of history deleted must not change any> earlier value — or even its missing/not-missing status.That single check catches the entire family of look-ahead bugs. A full-samplez-score fails it. A `.fillna()` that propagates backwards fails it. A negative shiftfails it. And notice that the NaN-status clause matters as much as the values: asignal that quietly becomes computable earlier when it can see the future isleaking, even if the numbers it produces look plausible.## The part nobody builds: deciding whether the result is realHere is the failure mode this whole project is designed around.Suppose you generate a hundred signals, backtest them all, and report the best one.Its Sharpe ratio is not an estimate of its quality. It's an estimate of **how manythings you tried.** Run enough random strategies against the same 280 months andone of them will look excellent, guaranteed, with no alpha anywhere in sight.So the leaderboard is deliberately **not** sorted by Sharpe. Three things sit on topof it.**Factor attribution.** Every signal's returns are regressed on market, size, valueand momentum — factors built from the same panel, so the comparison is like forlike — with Newey-West standard errors. What matters is the intercept: the returnthe known premia *cannot* explain. This is brutally clarifying. The library's`momentum_12_1` signal has a factor R² of 0.92 and an alpha t-stat of −0.10. Itisn't a discovery. It *is* the momentum factor, rediscovered, and it should be paidfor exactly once.**A search hurdle.** Given how dispersed the Sharpes in the library are, you cancompute the Sharpe you'd expect from the *best* of N random strategies. That's thebar luck alone clears. It is not a constant: it is recomputed on every run from theledger's attempt count and the spread of Sharpes across the library, so it climbs asthe search grows. Anything below it is not evidence of anything. The[results page](../../../llm-alpha/index.qmd) carries its current value.**Deflated Sharpe and Benjamini-Hochberg.** The deflated Sharpe is the probability asignal is real *given* the size of the search that found it. The BH q-value controlsthe false-discovery rate across the whole library at once. The denominator for bothcomes from the ledger, which records every attempt — including the ones that died atthe data gate or failed to compile. Counting only survivors would understate thesearch and flatter every number on the page.This machinery has a property I've come to appreciate: **it gets harder to satisfythe more you use it.** Every farming run adds trials, which raises the hurdle foreverything already in the library. A search large enough will push its own earlierdiscoveries back below significance. That is not a bug — it is the honest accountingfinally showing up on the balance sheet.## What it foundThere is deliberately no scoreboard in this post. The campaign runs daily and thelibrary grows most nights, so any count printed here would be stale within a week.Worse, a count is the wrong thing to fixate on: in this setup the number of attemptsis a *cost*, not an achievement, because every trial raises the bar for everythingalready found. The [results page](../../../llm-alpha/index.qmd) regenerates itself onevery run and carries the current figures, hurdle included.What is worth writing down is the shapes that keep recurring, because those have beenstable since the first campaign day.**Nothing farmed has cleared the search hurdle.** Not once, not yet. The only signalin the library that clears it is `gross_profitability` — Novy-Marx's, published in2013, and sitting in the library before any agent ran. Even that one is labelled a*candidate* rather than a survivor, because clearing the hurdle and survivingmultiple-testing correction with a high deflated Sharpe are two different tests, andit passes one of them.**Everything else works, which is the point.** Every farmed signal compiled. Everyone passed the no-look-ahead validator. Every one arrived with a coherent economicstory attached — inventory growth gaps, stock-comp dilution, oil-beta interactions,turn-of-year tax-loss reversals. The pipeline is not failing. The hypotheses are justwrong, in the ordinary way that most hypotheses are wrong.**The strongest t-statistics point the wrong way.** This one I did not expect.Several farmed signals are comfortably significant under Benjamini-Hochberg — andnegative. `attention_spike_return_reversal` posts a t-stat of −5.3 and`filing_staleness_information_neglect` −4.4, both with Sharpes well below zero. Theagent found real cross-sectional structure and then declared the sign backwards. Itis tempting to read that as a discovery in disguise: flip `expected_sign` and youhave a strong signal. Resist it. Choosing the sign after seeing the result is justanother trial, and it is the cheapest way there is to fool yourself. If the mechanismargument only works once you know the answer, it was never a mechanism argument.**The search keeps rediscovering itself.** Left alone, the idea agent converges. Thelibrary holds a cluster of same-month seasonality variants that are recognisably oneanomaly wearing different names, and pairs like `rate_duration_mispricing` and`hidden_rate_duration_mismatch` whose return streams are nearly identical — a noveltyscore of 0.02 between them. This is worse than wasted spend. Near-duplicatesmanufacture trials, trials raise the hurdle, and the library ends up penalising itsown genuine entries for the sin of being asked the same question twice.Deduplication is not housekeeping here; it is part of the statistics.I want to be precise about what this does and doesn't show. It is a fixed dataset, afixed universe, and one family of ideas. It is not proof that the approach can'twork. But it is a real result, and it lines up with the thing quantitativeresearchers already know and outsiders consistently underestimate: **plausible ischeap.** The hard part of finding alpha was never generating hypotheses. A languagemodel generates a dozen excellent-sounding ones for under a dollar. The hard part isthat almost all hypotheses are wrong, and the only way to know which is to test themin a way that can tell you no.One entry is still worth watching. `market_breadth_high_ratio_timing` posts a Sharpeof just 0.12 — nothing — but an alpha t-stat of **2.64**. Real explanatory power thefactor model misses, wrapped in a construction too weak to trade. That is the shape agenuine discovery would first appear in, and it is exactly the shape a Sharpe-sortedleaderboard would throw away.## How you'd implement a signalEverything above exists so this part can be short. Here is the whole contract:```pythonfrom __future__ import annotationsimport pandas as pdfrom llm_alpha.signals.base import BaseSignalclass GrossMarginTrend(BaseSignal): name ="gross_margin_trend" category ="quality" expected_sign =1 description ="Year-over-year change in gross margin: improving pricing power."def compute(self, panel: pd.DataFrame) -> pd.Series: gm = panel["grossmargin"] gm_prior = gm.groupby(level="ticker").shift(12)return gm - gm_prior```That's a complete, working signal. Four class attributes and one method. Walkingthrough what each part is doing:**The panel** is a DataFrame indexed by `(date, ticker)` — one row per stock permonth-end, about 1.9 million of them. It carries price-derived features(`ret_12_1`, `vol_252`, `beta_252`, `high_ratio`), daily valuation ratios (`pe`,`pb`, `marketcap`), and point-in-time quarterly fundamentals (`revenue`, `assets`,`gp`, `ncfo`, `grossmargin`, …) already joined as of their filing date. You neverload data yourself; it's all there.**`compute` returns a Series indexed exactly like the panel.** Same index, one scoreper row. Return `NaN` wherever the signal is undefined — the harness will simplyexclude those names from that month's cross-section. Missing data is expected andhandled; do not paper over it.**`groupby(level="ticker").shift(12)`** is the single most important idiom, and theone the coder agent most often gets wrong on its first attempt. It means "thisticker's value twelve months ago". The `groupby` is not optional: without it, `shift`walks off the end of one stock and into another, silently comparing Apple toAmgen. A **negative** shift would reach into the future and the validator willreject it.**Guard your denominators.** The reference implementations do this everywhere:```pythondef compute(self, panel): assets = panel["assets"]return (panel["gp"] / assets).where(assets >0)````.where(assets > 0)` yields `NaN` where the denominator is non-positive, ratherthan `inf` or a sign-flipped nonsense value. A single `inf` can dominate across-sectional ranking and quietly wreck a month.**Then set `expected_sign` honestly** and don't pre-negate anything. If you believelow values predict high returns, write the natural quantity and set `-1`.Drop that file into `signals/library/`, and the rest is automatic: the registrydiscovers it, the validator truncates the panel and checks it for look-ahead, theharness backtests it identically to every other signal, the robustness batterysweeps costs and sub-periods and universes, the factor model tells you whether it'snew, and the deflation layer tells you whether to believe it.You do not get to tune any of that. Which is the point — the harness was builtbefore the hypotheses, precisely so it couldn't be.## What I'd do nextThe bottleneck is not the pipeline, and it is fairly clearly not the model'simagination either. It is the **hypothesis space**, which is bounded by what thepanel contains — prices, fundamentals, a handful of macro series.Running the same agent against the same columns another thousand times mostlymanufactures trials, and every trial raises the bar for everything already found.Adding a genuinely new data source — short interest and borrow cost, 13F ownershipchurn, analyst revisions and dispersion, options-implied skew — multiplies the spaceinstead of re-searching it. The other half of the same problem is the duplicateclusters above: a correlation gate that refuses a proposal too close to somethingalready on the ledger would buy back hurdle for free, without needing a single newidea.There is a second direction, and it is about the harness rather than the ideas.The decile long/short construction in the worked example above is deliberatelycrude: every name in the top decile gets the same weight regardless of how stronglythe signal likes it, and every name outside the two extreme deciles gets nothing.That is the right default for *judging* a signal — it is transparent, it is thestandard academic construction, and it cannot be tuned into looking good — but itis not how anyone would actually hold the thing.So two variants are on the list. **Position sizing**: weight by signal strength orby inverse volatility instead of equally, so that conviction and risk both show upin the book rather than being flattened away. And a **long-only version**, becausethe short leg is where the borrow costs, the crowding and the delistingassumptions do their real damage, and because most of the capital that would everrun something like this cannot short at all. Both change the answer, and neitheris guaranteed to change it upwards — a signal whose spread lives entirely in theshort leg looks very different once you take the short leg away. That is preciselywhat makes it worth measuring.That's the next move: fewer, better-fed guesses rather than more of the same. Thecampaign keeps running daily, and [the results page](../../../llm-alpha/index.qmd)regenerates itself on every run, hurdle and all.