HMAC audit chain

Chain construction

Every audit row stores:

id, ts, actor_user_id, project_id, action, target_type, target_id, details,
row_hmac, prev_row_hmac

Where:

row_hmac = HMAC-SHA256(
  Z4J_AUDIT_CHAIN_SECRET,
  canonical_json({
    id, ts, actor_user_id, project_id, action,
    target_type, target_id, details,
    prev_row_hmac
  })
)

The chain is signed with a dedicated key, Z4J_AUDIT_CHAIN_SECRET, which is independent of Z4J_SECRET. Outside development it is required and has no fallback: the brain refuses to start rather than quietly signing the audit log with the same key it uses for everything else.

The separation is the point. If the audit key were derived from application configuration that lives beside the database, anyone who could reach the database could also reach the key, and the chain would only prove that the rows had not been edited by someone with less access than the person most likely to edit them. Keep this key somewhere the database operator cannot read.

canonical_json is sorted-keys, no whitespace, UTF-8. prev_row_hmac is the row_hmac of the previous row in id order. Genesis row has prev_row_hmac = null.

Properties

• Tamper-evident - change any field → row_hmac no longer verifies.
• Insertion-evident - inserting a row in the middle breaks the next row’s prev_row_hmac link.
• Deletion-evident - deleting a row breaks the chain at the gap.

Not prevented:

• Append-only suffix replacement - someone holding the audit-chain key and database access can truncate trailing rows and re-sign. Key separation is what makes this hard rather than routine: database access alone is not enough, so this requires compromising two things rather than one. If you want it to require three, periodically export row_hmac somewhere append-only (an object-store bucket with retention, or a second database) and compare.

Verification

z4j audit verify

Walks the full log; returns (ok, rows_verified, first_broken_id). Runs on demand (CLI / API / dashboard button). For a periodic check, wrap the CLI invocation in cron or a Kubernetes CronJob – the brain does not loop the verification internally.

Activation

A chain has to start somewhere, and the honest question on an existing deployment is what to do with history written before there was a chain.

The upgrade classifies the existing audit history in place. When every row classifies unambiguously, the chain activates during the migration and those rows are frozen as authenticated legacy history, so the ordinary upgrade is an ordinary upgrade.

When the history cannot be classified beyond doubt, the migration stops and returns nonzero, naming the rows and the reason. That covers a forked or truncated chain, rows signed with a key you no longer hold, and an audit table that exists but is empty, because a row count alone cannot distinguish a fresh install from one that was pruned or restored. In that case nothing is half-applied: the preparation revision stands, chain state is not created, and you complete an explicit operator attestation before re-running.

Refusing here is deliberate. Adopting history that cannot be proven would mean anyone who emptied the audit log could obtain a clean genesis and a chain that verifies, which is precisely the outcome the chain exists to make impossible.

Key rotation

Rotating the audit key would break verification of pre-rotation rows under a naive change. Set the previous value or values in Z4J_AUDIT_CHAIN_PREVIOUS_SECRETS (comma-separated) so the verifier accepts rows signed under any of them while new writes use the current Z4J_AUDIT_CHAIN_SECRET. Once every retained row is re-anchored or has aged out, drop the old key.

Rotation is itself an audited action, so the rotation appears in the chain it rotates.

If you would rather start fresh, export the old rows with their HMACs (still verifiable with the old key, offline), rotate, and begin a new chain.

Export integrity

CSV and NDJSON export include row_hmac and prev_row_hmac. Downstream verification needs the audit-chain key, shared with the verifier out of band. Because that key is separate from Z4J_SECRET, you can give an auditor what they need to verify the log without handing over the key that protects sessions and everything else.

Performance

HMAC-SHA256 on a small canonical payload is microseconds; the chain does not meaningfully slow writes. Verification of a million rows takes ~10 seconds on modest hardware.