Operations
Normal client lifecycle
Enrol once (ticket → handshake → principal), configure connectors per boundary, run. Day to day there is nothing to operate: sessions renew themselves, spokes reconnect with bounded backoff, the pump drains on its loop. Operator attention belongs to four events: enrolment, revocation, deployments/migrations, and failures below.
Session renewal
Renewal is deterministic per session (jittered 42–48 minutes against a one-hour expiry), drains in-flight calls across the swap, and retires the old session best-effort. A spoke that cannot renew backs off, serves out the expiry, and exits loudly for supervision. If many spokes flap at once, suspect the endpoint or the credential files (modes, paths, revocation) before suspecting renewal math, and check whether sessions hit the per-principal cap (sixteen live sessions; expired-but-untouched entries count until reaped, so a wedged fleet renews into a wall).
Endpoint failures
When calls fail, separate the layers in order:
- Transport: DNS, IPv6 egress, TLS validation (see below).
- Authentication (
-32001 authentication failed): expired, revoked, killed, wrong profile, or replayed credential. Re-handshake; if it persists, the principal entry changed server-side. - Scope surprise: empty results where content was expected usually mean
grants, not outage: confirm with
memory_status(visibility marker) and the project list before escalating. - Genuine service fault:
doctorfull detail, then the operator log.
Never "fix" auth failures by widening grants or sharing principals; fix the credential or the enrolment.
Certificate problems
Public chains verify via system roots alone; private or test anchors go
through --ca-file (additive, never a replacement, failure is
fail-closed). When TLS fails: confirm the endpoint name matches the
certificate, confirm the local clock (see below), confirm the bundle
paths exist on the workstation, and only then suspect the server. Let’s
Encrypt-style rotation is a deployment event with a staged restart, not
a client event.
IPv6 considerations
The public endpoint is IPv6-only. Workstations need working IPv6 egress; without it nothing connects and the failure looks like DNS or timeout rather than anything SHAMPOO-specific. Test with the platform tools first (name resolution, a plain TLS handshake) before debugging the spoke: if the network path is absent, no spoke flag will conjure it.
Clocks, caps, and other sharp edges
- UTC everywhere: expiry, renewal, and replay windows assume sane clocks. Skewed workstation clocks produce mysterious TLS and session failures; fix NTP before anything else.
- Challenge slots are single-use, same-connection, and expire in minutes: enrolment tooling must complete the pair without delay or intermediaries.
- The pump runs per profile with lease-based claiming; concurrent instances are safe, stalled integrations cannot stall the daemon, and pump lag is warn-only in diagnostics for exactly that reason.