Work Services Journal Free audit →
Back to home
Flagship · quantmarketing backtester Synthetic · real schema

Backtest the creative, not the spend.

A quant factor backtester pointed at ad creative. Run a thumbstop model over past posts and grade how well it ranks them by attention — before you spend. The same toolkit equity quants use: information coefficient, decile spread, walk-forward.

01 — Run a backtest
Niche — the factor's domain
Model
Holdout — out-of-sample30%
Data pool
Opt-in partner data
— de-identified rows pooled
Information coeff (IC)
Forecast vs realized · rank IC
ICIR · info ratio
Signal Sharpe · mean IC / IC vol
Top-decile lift
Long leg vs niche average
Hit rate
Directional accuracy, top decile
Finance terms. IC is the forecast-to-outcome rank correlation, the standard factor-signal metric — 0 is noise, 0.1+ is real edge. ICIR is the information ratio of that IC, the signal's Sharpe. Lift is the long leg vs average. Breadth is sample size, and IR is roughly IC times the square root of breadth.
02 — Grade the signal
Responds to controls
Gains curve · cumulative capture
Keep top X% ranked, capture Y% of views
Model Oracle Human pick Random
Decile spread · long–short
Mean views by predicted decile · D1 low → D10 high
× D10 / D1 spread · monotone is the tell
Walk-forward IC
IC per time period · stability is the point
of 10 periods positive · rolling window
How to run it
01 · UNIVERSE
Pick the niche
That is the factor's domain — the pool the signal is graded within.
02 · FEATURES
Tag point-in-time
Each creative tagged as it was, no lookahead into the future.
03 · SPLIT
Train / test
Train on the past, test out-of-sample on the held-out future.
04 · GRADE
Score the signal
IC, ICIR, decile spread, and gains versus the baselines.
05 · WALK FWD
Roll the window
Confirm the signal holds period after period, not once.
06 · DEPLOY
Clear the bar
Ship only if it beats the threshold — otherwise kill it cheap.
The data structure · click a column to sort
Every post is a row.

The table the model trains and tests on: creative and context as point-in-time features, views and clicks as the forward label. pred is the model's rank score.

What's working now · the signal feed
Scoped to
The live read per niche.

What a contributor sees: the formats and post windows pulling above average right now, scoped to their niche.

Top formats — lift vs average
Best post windows — mean views
The money pitch
Cut the weak before you pay for it.

The model kills the bottom 30% before spend. Enter your numbers, see the test budget you stop burning. Reallocation upside sits on top.

$
#
$
Wasted test budget saved / mo
Annualized
Spend reallocated off weak creative / mo
Tests killed cheap / mo
Give to get · the data flywheel
Free audits in. Sharper model out.

We run a free audit; you opt in to share de-identified outcomes; the niche pool grows; every contributor's predictions get sharper. The audit is the front door, the pool is the moat.

Free audit
Score their creative
Opt-in share
De-identified rows
Niche pool grows
More labeled data
Model sharpens
Better-fit weights
Better audits
Flywheel turns
Audits collected — drag to compound
120
— rowsIC —
The deal
Contribute rows, get the model

You give de-identified outcomes. You get predictions trained on the full pool, not your thin slice. The contributor advantage compounds as the pool grows.

The guardrail
Pool reads, raw stays private

Per-brand rows never leave your tenant. Only aggregate, thresholded signal feeds the shared model. No creative, caption, or brand identity is exposed.

Run a free audit →