Skip to content
Lab & evidence

Evidence before money

Two honest halves (a deterministic backtest report and the forward-only live track record), plus what each does and does not prove, and how to export, print or email them.

Who it is for
Everyone on the book
Reading time
7 min read
Updated

Before anyone funds a book, Bellwether shows two kinds of evidence and never blends them: a backtest of the rule-based stack that code alone computes, and the live record of what the whole system (the PM agent included) has actually done in this book. The PM agent is not backtested: it is a language model that has read every one of those years, so an in-window "backtest" of it could not tell skill from memory. Its only honest record is forward.

Both live on Performance, which has five tabs: Results, Trades, Track record, Statement and Evidence. Older links to the separate Track record, Statement and Evidence pages still work and open the matching tab.

Performance: return and the gap to SPY, the book against SPY and 60/40, Luck or skill?, the gap after AI cost and the underwater curve; trades have their own tab.
Performance: return and the gap to SPY, the book against SPY and 60/40, Luck or skill?, the gap after AI cost and the underwater curve; trades have their own tab.

Two halves, kept apart

  • Backtests (Performance › Evidence › Backtests) replays every rule-based and your over the last five years with the same that runs live: whole shares, the per-order cap, the daily-loss and halts, the turnover budget, costs on every fill. It is the second view of the Evidence tab, chosen with the control at the top right of the tab (Proof so far or Backtests). The same inputs always give the same report; the seed and the dataset fingerprint it was built from sit in the at the foot of How this was made.
  • Track record (Performance › Track record) is what the system has done in this book since its first funded day, day by day, by proposal. Nothing is replayed; nothing is chosen with hindsight.

Reading the backtest page

The Backtests view opens with one summary panel, What the rules alone would have done, and a short table of contents over three sections: Each strategy on its own, Would other settings have changed the result? and How this was made.

  • The summary panel: four figures for your blend (the yearly return next to what the S&P 500 made, the Sharpe from daily returns with its likely range, the worst fall from a high, the time spent below its high), the growth chart against SPY and a 60/40 mix over the same days with the same money, and three caveats a person can repeat. If the report was built with different limits or weights than the book has now, an amber note at the top of the panel names each difference.
  • The mix in full: one fold under the summary holds every other figure for the blend, year by year and in rough markets, and ends with What this proves, and what it does not, in the report's own words. Read that. The universe is today's mega-caps (survivorship-biased upward), fills are at the open plus a stated spread, dividends are frictionless, and no person, or is part of the replay.
  • Likely range: beside each Sharpe, the range it plausibly lands in if the same years were shuffled and replayed thousands of times. A range that reaches below zero cannot rule out "no edge at all".
  • Each strategy on its own: a table, best first. Choose a row to open its record in place. Tested on years it had not seen means the settings were picked on the past only and then scored on the following year; the stitched line is the honest one. The blend has no such line: nothing about it was chosen by the machine, so its panel reads "Each year on its own, not " and is the same configuration restarted each year. The blend's weights, and the defaults every grid is centered on, were chosen by people who had seen these years; no statistic on the page can undo that.
  • Sharpe after allowing for tries and Chance it is overfit: how much of the best setting's Sharpe is selection luck, counting only the settings this report tried. Below about 0.95 and above 0.5 respectively, treat the result as noise. When only one try was counted the row is named Chance the Sharpe is above zero and says that nothing was .
  • With costs 3 times higher: the same run with spread and impact tripled. If the edge vanishes there, it lived inside the cost assumption.
  • The check for peeking at the future: every price after a cut date is replaced by noise and each strategy is rerun; a decision before the cut that changes means the strategy read the future. How this was made says in one sentence how many strategies came through clean and that a strategy which cheats on purpose was caught; the table per strategy is in the fold under it.
  • Would other settings have changed the result? Each console setting moved on its own from your baseline, grouped in folds: the weights in the mix, how each strategy is used, , the limits one at a time, and sizing, number of stocks and how often. Each fold's header says how many of its settings made a real difference. The bar is the change in Sharpe with its likely range. Too close to call means five years cannot tell better from worse, Changes nothing here means the setting changed nothing at all in the rule-based stack (it may still act on the PM agent live; open the row to read what), Not tested here means a backtest cannot see it by construction.
  • The same rules on a small and a large book: the limits are dollars, not percent: a $150 daily-loss fires on ordinary red days for a $100k book, and a $500 per-order cap cannot buy one share of a $700 stock in a $10k book. The table shows the yearly return, the Sharpe, the worst fall and the days trading was paused at each size.
Performance › Evidence › Backtests: what the rules alone would have done over the last five years, from the starting money shown, and whether the report matches your book.
Performance › Evidence › Backtests: what the rules alone would have done over the last five years, from the starting money shown, and whether the report matches your book.

Does the PM add anything?

On the Track record tab, under Desk vs its rules, one section puts three lines on the same money and days: what the rule-based blend would have done on its own under your weights and caps (simulated on the committed bars), what the book actually did, and the book with your own trades taken out (the model alone). Beside the chart: what the model alone made, what its rules alone made, and the difference between them in percent, with the interval for the average difference a session. Read the sentence at the top literally: "too close to call" means exactly that; only a gap "larger than chance alone would explain" is a claim, and it needs at least twenty sessions and an interval that stays on one side of zero. How this is calculated under the section holds the full sentence with its interval, the window and the weights used.

Sharing it

An owner or operator can share a book read only from the People page (Admin › People, or Settings › People for an operator of their own book). Under Add someone, type the person's email and choose Read only. The Share button on the Track record tab opens the same page. A viewer sees every Performance tab and every number on it, and can act on none of it. The Lab is the one in this area a viewer does not get: it says "Lab · read only" instead, because what-ifs and backtests use the owner's computing budget.

Reports, exports and email

Track record, Statement and the Backtests view each have one Export menu beside their title: the report as a PDF, its tables as spreadsheets, Print this page and Email the PDF. Track record adds A web page that opens anywhere, a single file that carries today's figures. The PDF you download is the same file an email attaches; the spreadsheets carry the raw rows (proposals with outcomes, closed trades, daily equity; for Backtests the figures per strategy, the daily growth curves and the settings comparison). The Statement tab is one calendar month, chosen from the list beside Export: what the book started and ended the month with, the trading result before and after model spend, fees and money paid in or out, positions at the last close, every round trip.

Email the PDF sends one message per recipient with the PDF attached; people who can already open the book also get a read-only link, everyone else gets the PDF only. An email never widens who can open the console. Sending asks for your , is written to the audit trail, and is limited to 20 recipient-sends per book per hour. Scheduled reports, a panel on Settings › Notifications, sends the weekly letter on Fridays after the close and the monthly statement on the 1st to up to ten addresses; adding an address asks for your code, switching a report off does not. Reviewers, viewers and anyone looking at a shared book read only can download and print; the menu does not offer them email and they never see recipient addresses.

Lab and simulation results are not evidence

Anything the Lab or the simulation harness produces is a behavior test, not evidence of returns: it checks that the machinery does what it should on the days it is given. Returns are claimed only in the two places above.