Archive

A sink normally only ever sees perfdata. Sometimes you want the whole story — every decision the engine ever made — kept somewhere durable and queryable without running jaque at all. That’s a sink with type: "archive".

What it archives

The entire event log: every event’s Codec envelope, one JSON line each, sealed into zstd-compressed segments named by the sequence range they cover.

sinks: {
  log: {
    type:          "archive"
    url:           "file:///var/lib/jaque/archive"
    segment_bytes: 67108864
  }
}

Two URL forms — file:///dir for a local directory, or s3://[KEY:SECRET@]bucket/prefix?endpoint=host:port&spool=/dir for an S3-compatible bucket. Credentials for the S3 form come from the URL’s userinfo if present, otherwise from AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY. The deprecated -archive flag still works and is translated into an archive-named sink at startup — see Sinks and perfdata.

Segments

segment_bytes (default 64 MiB) is the uncompressed size that triggers sealing a partial segment. Sealed files are named events-<first>-<last>.jsonl.zst. On startup, Recover truncates any segment that was mid-write to its last complete line, seals it, and the archive follower resumes right after the last sequence already archived — a crash mid-segment never corrupts the archive or loses events silently. Several archive sinks with different urls coexist without conflict — unlike the old single -archive flag, there’s no limit of one.

Reading it without jaque

That’s the point of the format — plain JSON lines, zstd-compressed, no proprietary reader required:

zstd -dc events-*.jsonl.zst | jq .

-- ClickHouse
SELECT * FROM file('events-*.jsonl.zst', JSONEachRow)

-- DuckDB
SELECT * FROM read_json('events-*.jsonl.zst')

Off by default

Like every sink, the archive is opt-in. There’s no archive sink until the operator declares one under sinks: (or the deprecated -archive flag); nothing is written unless you point it somewhere.