Prepare data for a reproducible backtest

Prepare data for a backtest that someone else can reproduce: choose REST history or WebSocket replay, freeze the inputs, check data quality, and save a run manifest beside the result. This covers getting the same records again; modeling when each record would have been known to a strategy is up to your backtest.

REST history or replay

Need	Better fit
Pull trades for a bounded window	REST history
Compare daily funding or OI values	REST history
Generate a static research file	REST history unless sequence matters
Rebuild a local order book in event order	WebSocket replay
Test strategy reaction to event cadence	WebSocket replay
Reproduce an incident stream	WebSocket replay

Use REST history when records can be retrieved and ordered after the fact. Use WebSocket replay when the channel replays and event cadence, sequence, local book state, or gap handling is part of the test. Core L4 bulk replay ignores speed; it uses a checkpoint and bounded event batches instead.

Backtest contract

Declare the venue family, symbol, data family, route or channel, integer-millisecond window, page budget or replay delivery, quality gate, output schema, and request IDs before the run starts. Keep raw API rows separate from derived features or strategy output. Choose one hypothesis that can be tested with one data family, and confirm the symbol’s namespace in Venues and market types if it is not known.

Freeze the window

Compute the window once, before any request, and save it with the run. This example uses one hour of Hyperliquid core BTC: the hour that ended one hour ago, a window every plan’s history reaches, including Free’s most recent 30 days. A backtest over older history needs the full retained archive of Build or above.

END=$(( ($(date +%s) - 3600) * 1000 ))   # one hour ago, in Unix milliseconds
START=$(( END - 3600 * 1000 ))           # one hour earlier
printf 'START=%s\nEND=%s\n' "$START" "$END" > run-window.env

Every later step uses these two values: the quality check, the REST pull, a replay, and the run manifest. To repeat or resume the run, load them again with source run-window.env rather than recomputing them, because recomputing selects a different hour. Requests also accept RFC 3339 times, but integer milliseconds keep the window identical across your shell, scripts, and manifest.

Run a REST window

Freeze the other inputs

Save the family, symbol, data family, page size, quality threshold, gap policy, and output path with START and END before calling the API.

Check data quality for the same window

curl -sS "https://api.0xarchive.io/v1/data-quality/coverage/hyperliquid/BTC?from=$START&to=$END" \
  -H "X-API-Key: $OXARCHIVE_API_KEY"

curl -sS "https://api.0xarchive.io/v1/data-quality/incidents?exchange=hyperliquid&limit=100" \
  -H "X-API-Key: $OXARCHIVE_API_KEY"

Read coverage for the frozen window and keep the incidents that overlap it, as shown in Data quality. For the fields in each response, see Data quality routes. The current /v1/data-quality/status and freshness only matter when the window ends near now. Keep these responses with the run even when the job later stops.

Pull one bounded page

curl -sS "https://api.0xarchive.io/v1/hyperliquid/trades/BTC?start=$START&end=$END&limit=1000" \
  -H "X-API-Key: $OXARCHIVE_API_KEY"

Do not add concurrency until the first response and its request ID are saved.

Follow only returned cursors

If meta.next_cursor is present, send that exact value within the declared page budget. Do not write an unbounded loop: stop at the page budget even when a cursor remains.

Switch surfaces when order matters

If event order rather than returned rows is the requirement, stop the REST run and continue with a replay run below, over the same frozen window. Record it as a separate run.

The run is complete only when the requested window is exhausted within the page budget and the quality decision allows use. Check the first and last returned timestamps against START and END, and keep the cursor chain and request IDs per page.

Run a replay

Bound the replay

Use the frozen START and END, an explicit channel and symbol, the delivery model, and the speed when applicable. Avoid endless replay jobs.

Start the replay

// The frozen window from run-window.env, the same one the REST run used.
const start = Number(process.env.START);
const end = Number(process.env.END);

ws.send(JSON.stringify({
  op: "replay",
  channel: "orderbook",
  symbol: "BTC",
  start,
  end,
  speed: 10
}));

Run the script with the window loaded, for example set -a; source run-window.env; set +a; node replay.js. Speed 10 is inside every plan’s standard replay limit, including Free’s.

Handle gaps

Record gap_detected events and decide whether the run is invalid, partial, or rebuildable. Do not silently interpolate missing market events and present the result as complete.

Watch consumer lag

If standard replay outpaces the strategy loop or output writer, stop or slow down rather than silently dropping messages. For core L4 bulk replay, reduce the window or stop the run when bounded queues cannot absorb a batch.

Pair every replay with stop conditions, backpressure handling, and output storage. A replay that cannot be consumed in order is not a trustworthy backtest input.

Result decisions

Label every run complete, partial, or stopped. Complete means the configured window ran to its stop condition without unresolved gaps, a remaining cursor, or consumer backlog. Partial means the output may still be useful for review, but the manifest names the missing interval, the remaining cursor, or the state to rebuild. Stopped means the strategy output should not be compared against clean runs.

Failure and recovery

400: fix the numeric bounds, symbol, or named parameter before retrying.
401: repair the key source or header.
429: retry rate_limited within the retry budget and stop on insufficient_credits, as When you reach a limit describes.
A remaining cursor at the page budget: mark the run partial and resume under a new bounded manifest.
A quality or gap_detected result: stop the derived calculation, narrow the window, or rebuild the relevant state from the documented checkpoint path.
Missing rights for the intended use: stop and read Data rights before creating a benchmark, index, financial product, or bulk distribution.

Run manifest

Store the same fields with every run so results can be compared later.

{
  "purpose": "btc_trade_backtest_hour_001",
  "venue_family": "hyperliquid_core",
  "symbol": "BTC",
  "data_family": "trades",
  "surface": "REST",
  "route": "/v1/hyperliquid/trades/BTC",
  "window": {
    "start": "START from run-window.env",
    "end": "END from run-window.env"
  },
  "limit": 1000,
  "page_budget": "record the finite page count",
  "quality_gate": {
    "status_route": "/v1/data-quality/status",
    "coverage_route": "/v1/data-quality/coverage/hyperliquid/BTC?from=START&to=END",
    "overlapping_incidents": [],
    "decision": "the application's decision from the coverage and incident responses"
  },
  "request_ids": ["00000000-0000-0000-0000-000000000000"],
  "output": "./artifacts/btc-trade-backtest-hour-001.json",
  "code_version": "record the strategy commit or notebook hash",
  "run_status": "complete | partial | stopped"
}

For a replay run, replace the route, limit, and page budget with the channel list, the delivery model (timed at a given speed, or checkpoint_anchored_bulk for L4), the gap policy, and the replay events that ended the run. WebSocket replay lists every field. The zero UUID is a shape placeholder only. Replace it with the UUID returned by each request.

Next step

When a backtest needs more history than you can page through, export it as Parquet files from the Data Catalog.

Connect

Channels and modes

Replay and backtesting

Message reference

Prepare data for a reproducible backtest

REST history or replay

Backtest contract

Freeze the window

Run a REST window

Run a replay

Result decisions

Failure and recovery

Run manifest

Next step

​REST history or replay

​Backtest contract

​Freeze the window

​Run a REST window

​Run a replay

​Result decisions

​Failure and recovery

​Run manifest

​Next step

REST history or replay

Backtest contract

Freeze the window

Run a REST window

Run a replay

Result decisions

Failure and recovery

Run manifest

Next step