Backtesting your agent — the owner's complete guide
How to read every number, every chip, and every metric in the backtest modal. What a verdict really means, how the engine prevents you from fooling yourself, and a workflow for iterating from a red verdict to a green one.
A backtest is the single most useful tool you have as an agent owner. It replays the strategy against historical candles and produces an honest report card: did this set of rules have edge over the window you tested, or did it just get lucky a few times? This article explains every output you see, the safety checks running behind the scenes, and a step-by-step workflow for iterating from "No Edge" to "Edge Survives".
Opening the backtest modal
Open the agent detail page and click the Backtest button in the owner action row (right-side panel). The modal that appears has three sections from top to bottom: configuration, pipeline animation, and result.
Configuration — picking the window
Before you run the test, you choose the historical window:
- Window length — how many hours of candles to replay. Available windows scale with your agent's timeframe.
- Investment amount — the simulated starting balance the engine "deposits" before the first bar. Default 1,000 USDC.
- Risk override — the simulated risk level for this test. Defaults to the agent's saved risk; you can dial it up or down to see how sensitive the strategy is.
Available windows per timeframe
| Timeframe | Shortest window | Longest window | Recommended for first test |
|---|---|---|---|
| 1m | 1 hour | 24 hours | 24 hours |
| 5m | 4 hours | 7 days | 3–7 days |
| 15m | 12 hours | 14 days | 7 days |
| 1h | 24 hours | 30 days | 14–30 days |
| 4h | 7 days | 90 days | 30 days |
| 1d | 30 days | 365 days | 180 days |
The pipeline — what happens while you wait
When you click Run, the modal shows a live pipeline animation with each check passing in sequence. Here is what each step actually does:
| Step | What the engine is doing | Why it matters |
|---|---|---|
| Fetching candles | Pulls OHLCV bars from SorooshX for the chosen window. | Real data, not synthetic — your strategy runs against actual price action. |
| Loading indicators | Computes SMA / EMA / RSI / ROC / volatility series across the window. | Same indicators the live engine uses on every tick. |
| Replaying bars | Walks the window bar-by-bar, evaluating entry / exit / stop rules in order. | Realistic: only past data is visible at each bar (no future leak). |
| Applying frictions | Subtracts realistic slippage on every fill and funding cost on overnight holds. | Without this, backtests look ~20–80 bps better than reality. |
| Bootstrapping confidence | Resamples the trade list to build a confidence interval around Sharpe. | Tells you whether the result is statistically meaningful or could be luck. |
| Final verdict | Combines return, trade count, drawdown, profit factor, and the Sharpe lower bound into one chip. | A single honest signal instead of cherry-picking one good metric. |
Reading the verdict chip
The verdict at the top of the result is the single most important output. It comes in five flavours:
| Chip | What it means | What to do |
|---|---|---|
| Edge Survives (green) | The strategy cleared every internal check the engine runs. | This is a candidate. Run a longer window or a forward paper run to confirm. |
| Promising (light green) | Strong on most checks but one signal is borderline. | Iterate once or twice. Often becomes Edge Survives after a small tuning prompt. |
| Noisy Win (yellow) | Positive return but the result is statistically inconclusive — could be luck. | Run a longer window. If the verdict stabilises, treat it as Edge Survives. If it flips to No Edge, abandon. |
| Low Activity (yellow) | Not enough trades to draw any conclusion. | Either widen the window, loosen entry filters via Train Agent, or accept this strategy trades rarely. |
| No Edge (red) | The strategy failed one or more internal checks. | Iterate via Train Agent. Most starting configs land here on the first run — this is expected. |
The metrics grid — every number explained
Under the verdict, the result panel shows a grid of numbers. Each one tells you a different thing.
| Metric | What it measures | Healthy range |
|---|---|---|
| Total Return % | Final equity vs starting equity, in percent. | Positive. Compare to buy-and-hold of the same market for context. |
| Final Equity USDC | Closing simulated balance after all trades and fees. | Should match Starting × (1 + Return%). |
| Trades | Number of CLOSED positions (open positions at window end are closed at mark for accounting). | More trades = more statistical confidence in the verdict. |
| Win Rate % | Fraction of trades that closed profitable. | Don't obsess. A 30% win rate with 4x average winner beats 70% with 0.3x average winner. |
| Profit Factor | Sum of winning trades ÷ sum of losing trades. | > 1.0 is the bar. > 1.5 is healthy. > 2.0 with enough trades is excellent. |
| Max Drawdown % | Worst peak-to-trough equity drop during the window. | Lower is more comfortable. A large drawdown can downgrade an otherwise positive verdict. |
| Sharpe | Risk-adjusted return per unit of volatility. | > 1.0 is good. > 2.0 is great. The confidence interval is more honest than the point estimate. |
| Sharpe CI [low, high] | Confidence interval around Sharpe from resampling. | Low end > 0 means edge is likely real. Low end < 0 means it could be luck. |
| Best Trade USD | Largest winning trade in the run. | Sanity check — if one outsized winner explains the entire return, the strategy is fragile. |
| Worst Trade USD | Largest losing trade. | Should be roughly bounded by your risk × leverage settings. |
The equity curve chart
A line plot of simulated equity over the window. What to look for:
- Steady upward slope — the strategy worked across the whole window, not just one streak.
- One vertical jump followed by flat — a single lucky trade carried the run. Fragile.
- Long flat then a cliff — the strategy stopped working partway through. Window-specific edge.
- Sawtooth with shallow dips — healthy. Some losers, no catastrophic drawdown.
The safety gate (the hidden second backtest)
When you tune your agent via Train Agent — or when the nightly self-coach proposes a tweak — the engine runs a SECOND, internal backtest before applying any change. The proposed parameters only stick if they perform at least as well as the current ones on a fresh window. This is why a training prompt can come back with "applied" or "rejected — kept current params": the engine is protecting you from prompts that sound clever but degrade performance.
A workflow for going from red to green
Most starting configurations earn No Edge on the first backtest. That is the normal starting point — not a failure. Here is the loop most successful owners use:
- Baseline. Run a backtest on the longest available window with default risk. Record the verdict and the metrics grid (screenshot the result).
- Identify the weakest dimension. Look at the metrics grid — is win rate decent but profit factor terrible? Is profit factor good but max drawdown blew up? Is the Sharpe CI low end below zero? Pick ONE thing.
- Open Train Agent. Describe the fix in plain English, addressing only that one thing. Examples: "Tighten stops by half so losers don't run away", "Skip entries during low-volume Asian hours", "Only take longs when the 1h trend is up".
- Wait for the safety gate result. Watch the chat reply — it tells you whether the tweak was applied or rejected. If rejected, the message explains why.
- Re-run the SAME backtest window. Compare the new metrics grid to your screenshot. Did the targeted dimension improve without breaking others?
- Iterate. Two to five focused prompts usually moves a No Edge config to Promising or Edge Survives. More than five suggests the underlying idea doesn't fit this market — try a different style or symbol.
What backtests cannot tell you
Be honest with yourself about the limits of any backtest, including ours:
- They cannot tell you the future. Markets change regime; strategies that worked in 2024 trend may fail in 2026 chop.
- They cannot model your psychology. A 25% drawdown looks fine in a backtest and devastating in real time.
- They cannot anticipate black-swan events. A liquidation cascade or exchange outage is not in the historical bar data.
- They cannot replace forward paper trading. Run the agent in paper mode for at least a week before considering real funds.
Saving and sharing your results
The Save Image button at the bottom of the result panel captures the verdict + metrics + chart as a PNG. Good for tracking your iteration history, posting on X, or asking other owners for feedback. On mobile, the share sheet opens directly so you can drop it into a chat.