z4j and agents

z4j is one process per organization that owns user authentication, the dashboard, the REST API, the audit log, and the WebSocket server. Agents are thin in-process companions that live inside your worker or web-app processes; one agent per process. Everything important runs through z4j; agents only know how to talk to their local engine and how to forward events.

z4j (the control plane)

z4j is the stateful, central, user-facing half. One per organization.

Responsibilities:

• Serve the dashboard (React 19.2, TanStack Router v1, Tailwind 4, shadcn/ui).
• Serve the REST API (/api/v1/*).
• Accept WebSocket connections from agents at /ws/agent.
• Persist everything to PostgreSQL (or SQLite, which forces a single-worker brain).
• Authenticate users (argon2id + server-side sessions).
• Enforce RBAC.
• Emit audit log entries (HMAC-chained).
• Run the reconciliation worker (background task).
• Rate-limit unauthenticated endpoints.
• Redact secrets from events before persist.

Does not:

• Connect to your broker. z4j has no knowledge of Redis/RabbitMQ/SQS credentials. Agents do that.
• Connect outward to agents. All WebSockets are inbound (agent -> z4j).
• Hold queue state in memory. It's all in the database.

Agent (z4j-<framework> + z4j-<engine>)

The agent is the thin, distributed, in-your-process half. One per application process.

Responsibilities:

• Open and maintain one WebSocket to z4j.
• On connect, send hello advertising: protocol version, framework, engines, schedulers, capabilities map.
• On engine task events, capture -> redact -> buffer -> dispatch to z4j.
• On command frames from z4j, route to the engine or scheduler adapter and execute: task verbs such as retry, cancel and purge, and the scheduler verbs schedule.enable, schedule.disable, schedule.trigger_now, schedule.delete, schedule.fire, schedule.resync and schedule.external.*. There is no schedule create or update on the wire.
• Heartbeat every 10 seconds; reconnect on failure with backoff.
• Discover schedules on registered schedulers and emit on change.

Does not:

• Keep a local event history. Every captured event is appended to the on-disk SQLite buffer first, whether or not z4j is reachable, and is dropped once z4j acknowledges it (or evicted at the buffer's cap).
• Authenticate users, serve UI, or accept external connections.
• Know about other agents.

Authentication

• User -> z4j -- server-side session cookie (SameSite=Lax, HttpOnly, Secure). Password = argon2id. Session secret from Z4J_SESSION_SECRET.
• Agent -> z4j -- bearer token minted by an admin in the dashboard. Stored HMAC-SHA256 hashed in Postgres. Rotated by minting a new token and revoking the old one.

Tokens are per-agent, not per-project -- revoke individually without affecting other agents.

Data flow summary

   events flow  --------------->
  +------------+                +-------------+
  |   agent    | ----- ws ----> |    z4j      |
  | (in-proc)  | <---- ws ----- | (container) |
  +------------+                +-------------+
   <-------------- commands flow

Both directions on the same WebSocket. Sequence numbers are replay protection: within a session each direction's seq must strictly increase. At-least-once delivery comes from the agent's on-disk buffer, which drops an entry only when the brain's event_batch_ack names that frame's id after the batch is committed; z4j dedupes on persist.

What lives where, one glance

Concern	z4j	Agent
Database (PostgreSQL or SQLite)	yes	no
Queue credentials	no	yes (only your queue's creds)
HTTP UI/API	yes	no
WebSocket endpoint	yes (server)	yes (client)
Session cookies	yes	no
Engine-specific code	no	yes
RBAC rules	yes	no
Audit log	yes	no
Task runtime	no	no (Celery/RQ/... runs separately)

Naming notes

Operator-facing surfaces talk about "z4j" rather than "the brain". Internally the Python module that runs the central process is still named z4j_brain (parallel to z4j_core, z4j_celery, etc., where the suffix indicates scope within the multi-package z4j ecosystem). End users only ever interact with pip install z4j and z4j serve; the internal namespace is invisible to them.