> ## Documentation Index
> Fetch the complete documentation index at: https://docs.0xarchive.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Prepare data for a reproducible backtest

> Prepare backtest data from a bounded REST window or a WebSocket replay, with integer-millisecond bounds, a quality gate, and a saved run manifest.

Prepare data for a backtest that someone else can reproduce: choose REST history or WebSocket replay, freeze the inputs, check data quality, and save a run manifest beside the result. This covers getting the same records again; modeling when each record would have been known to a strategy is up to your backtest.

## REST history or replay

| Need | Better fit |
| - | - |
| Pull trades for a bounded window | REST history |
| Compare daily funding or OI values | REST history |
| Generate a static research file | REST history unless sequence matters |
| Rebuild a local order book in event order | WebSocket replay |
| Test strategy reaction to event cadence | WebSocket replay |
| Reproduce an incident stream | WebSocket replay |

Use REST history when records can be retrieved and ordered after the fact. Use [WebSocket replay](/websocket/replay) when the channel replays and event cadence, sequence, local book state, or gap handling is part of the test. Core L4 bulk replay ignores `speed`; it uses a checkpoint and bounded event batches instead.

## Backtest contract

Declare the venue family, symbol, data family, route or channel, integer-millisecond window, page budget or replay delivery, quality gate, output schema, and request IDs before the run starts. Keep raw API rows separate from derived features or strategy output. Choose one hypothesis that can be tested with one data family, and confirm the symbol's namespace in [Venues and market types](/core-concepts/venue-taxonomy) if it is not known.

## Freeze the window

Compute the window once, before any request, and save it with the run. This example uses one hour of Hyperliquid core BTC: the hour that ended one hour ago, a window every plan's history reaches, including Free's most recent 30 days. A backtest over older history needs the full retained archive of Build or above.

```bash theme={"theme":"github-dark"}
END=$(( ($(date +%s) - 3600) * 1000 ))   # one hour ago, in Unix milliseconds
START=$(( END - 3600 * 1000 ))           # one hour earlier
printf 'START=%s\nEND=%s\n' "$START" "$END" > run-window.env
```

Every later step uses these two values: the quality check, the REST pull, a replay, and the run manifest. To repeat or resume the run, load them again with `source run-window.env` rather than recomputing them, because recomputing selects a different hour. Requests also accept RFC 3339 times, but integer milliseconds keep the window identical across your shell, scripts, and manifest.

## Run a REST window

<Steps>
  <Step title="Freeze the other inputs">
    Save the family, symbol, data family, page size, quality threshold, gap policy, and output path with `START` and `END` before calling the API.
  </Step>

  <Step title="Check data quality for the same window">
    ```bash theme={"theme":"github-dark"}
    curl -sS "https://api.0xarchive.io/v1/data-quality/coverage/hyperliquid/BTC?from=$START&to=$END" \
      -H "X-API-Key: $OXARCHIVE_API_KEY"

    curl -sS "https://api.0xarchive.io/v1/data-quality/incidents?exchange=hyperliquid&limit=100" \
      -H "X-API-Key: $OXARCHIVE_API_KEY"
    ```

    Read coverage for the frozen window and keep the incidents that overlap it, as shown in [Data quality](/data-quality#a-historical-window). For the fields in each response, see [Data quality routes](/rest-api/data-quality). The current `/v1/data-quality/status` and freshness only matter when the window ends near now. Keep these responses with the run even when the job later stops.
  </Step>

  <Step title="Pull one bounded page">
    ```bash theme={"theme":"github-dark"}
    curl -sS "https://api.0xarchive.io/v1/hyperliquid/trades/BTC?start=$START&end=$END&limit=1000" \
      -H "X-API-Key: $OXARCHIVE_API_KEY"
    ```

    Do not add concurrency until the first response and its request ID are saved.
  </Step>

  <Step title="Follow only returned cursors">
    If `meta.next_cursor` is present, send that exact value within the declared page budget. Do not write an unbounded loop: stop at the page budget even when a cursor remains.
  </Step>

  <Step title="Switch surfaces when order matters">
    If event order rather than returned rows is the requirement, stop the REST run and continue with a replay run below, over the same frozen window. Record it as a separate run.
  </Step>
</Steps>

The run is complete only when the requested window is exhausted within the page budget and the quality decision allows use. Check the first and last returned timestamps against `START` and `END`, and keep the cursor chain and request IDs per page.

## Run a replay

<Steps>
  <Step title="Bound the replay">
    Use the frozen `START` and `END`, an explicit channel and symbol, the delivery model, and the speed when applicable. Avoid endless replay jobs.
  </Step>

  <Step title="Start the replay">
    ```javascript theme={"theme":"github-dark"}
    // The frozen window from run-window.env, the same one the REST run used.
    const start = Number(process.env.START);
    const end = Number(process.env.END);

    ws.send(JSON.stringify({
      op: "replay",
      channel: "orderbook",
      symbol: "BTC",
      start,
      end,
      speed: 10
    }));
    ```

    Run the script with the window loaded, for example `set -a; source run-window.env; set +a; node replay.js`. Speed `10` is inside every plan's standard replay limit, including Free's.
  </Step>

  <Step title="Handle gaps">
    Record `gap_detected` events and decide whether the run is invalid, partial, or rebuildable. Do not silently interpolate missing market events and present the result as complete.
  </Step>

  <Step title="Watch consumer lag">
    If standard replay outpaces the strategy loop or output writer, stop or slow down rather than silently dropping messages. For core L4 bulk replay, reduce the window or stop the run when bounded queues cannot absorb a batch.
  </Step>
</Steps>

Pair every replay with stop conditions, backpressure handling, and output storage. A replay that cannot be consumed in order is not a trustworthy backtest input.

## Result decisions

Label every run complete, partial, or stopped. Complete means the configured window ran to its stop condition without unresolved gaps, a remaining cursor, or consumer backlog. Partial means the output may still be useful for review, but the manifest names the missing interval, the remaining cursor, or the state to rebuild. Stopped means the strategy output should not be compared against clean runs.

## Failure and recovery

* `400`: fix the numeric bounds, symbol, or named parameter before retrying.
* `401`: repair the key source or header.
* `429`: retry `rate_limited` within the retry budget and stop on `insufficient_credits`, as [When you reach a limit](/rate-limits#when-you-reach-a-limit) describes.
* A remaining cursor at the page budget: mark the run partial and resume under a new bounded manifest.
* A quality or `gap_detected` result: stop the derived calculation, narrow the window, or rebuild the relevant state from the documented checkpoint path.
* Missing rights for the intended use: stop and read [Data rights](/data-rights) before creating a benchmark, index, financial product, or bulk distribution.

## Run manifest

Store the same fields with every run so results can be compared later.

```json theme={"theme":"github-dark"}
{
  "purpose": "btc_trade_backtest_hour_001",
  "venue_family": "hyperliquid_core",
  "symbol": "BTC",
  "data_family": "trades",
  "surface": "REST",
  "route": "/v1/hyperliquid/trades/BTC",
  "window": {
    "start": "START from run-window.env",
    "end": "END from run-window.env"
  },
  "limit": 1000,
  "page_budget": "record the finite page count",
  "quality_gate": {
    "status_route": "/v1/data-quality/status",
    "coverage_route": "/v1/data-quality/coverage/hyperliquid/BTC?from=START&to=END",
    "overlapping_incidents": [],
    "decision": "the application's decision from the coverage and incident responses"
  },
  "request_ids": ["00000000-0000-0000-0000-000000000000"],
  "output": "./artifacts/btc-trade-backtest-hour-001.json",
  "code_version": "record the strategy commit or notebook hash",
  "run_status": "complete | partial | stopped"
}
```

For a replay run, replace the route, limit, and page budget with the channel list, the delivery model (`timed` at a given `speed`, or `checkpoint_anchored_bulk` for L4), the gap policy, and the replay events that ended the run. [WebSocket replay](/websocket/replay#record-a-run-manifest) lists every field.

The zero UUID is a shape placeholder only. Replace it with the UUID returned by each request.

## Next step

When a backtest needs more history than you can page through, export it as Parquet files from the [Data Catalog](/data-catalog).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.