Lifecycle events: a message when a webhook endpoint (a source) or a route is created, updated, or deleted.
Off by default. config.lifecycle.sinks (YAML: lifecycle.sinks) is a list of
Ankusa.Sinks, shaped like a source's; with at least one, every change made
through Ankusa.SourceStore.put/5, Ankusa.SourceStore.delete/3, or
Ankusa.Routes (the admin API and the route-management API are both thin
wrappers over those) becomes one CloudEvents 1.0 event delivered to those
sinks.
A durable outbox
The event is not a message handed to a process; it is a row. The change's
audit row (Ankusa.Audit) is flagged publish: true and carries, beside it
and in the same synced batch, the redacted entity the event reports. So the
change and its event are committed or lost together: a kill -9 right after
Ankusa.SourceStore.put/5 returns still delivers source.created after the
restart.
Ankusa.Lifecycle.Publisher is a cursor reader over those rows. It keeps the
last seq it delivered to every sink in m:lifecycle_cursor (synced on every
advance), delivers the next flagged row to every sink, and moves on once each
sink confirmed or gave up. Rows are read in seq order, which is commit
order, so three rapid updates of one source arrive in revision order.
Ankusa.Replay with kind: :lifecycle re-emits a window of the trail
(since/until/tenant), which is how a sink that was down for longer than
dispatch.retry waits is caught up.
Delivery is at least once. A node killed between a sink's confirmation and
the cursor write delivers that event again after the restart, and so does a
replay; both carry the same event id (see below), so a consumer dedupes
on it. A sink that keeps failing holds back the events behind it until it
succeeds or dispatch.retry gives up ([:ankusa, :lifecycle, :dropped], reason
:gave_up); the event is then lost for that sink and can be replayed.
Turning lifecycle events on starts the outbox at the end of the trail: changes made while they were off are not announced.
The reserved id exists only on the event: the edge never resolves it, so
POST /webhooks/ankusa:lifecycle is a 404. The colon keeps it out of reach
of every tenant-scoped id, which is [A-Za-z0-9_-] joined by a dot.
The event
A structured-mode CloudEvent, carried as the body of a Ankusa.Sink.Message
with content_type: "application/cloudevents+json":
id— derived from the row, never minted: a keyed hash of the instance, the row'sseqand (for a row written after a restore from a backup) the incarnation it belongs to, laid out as a UUID. The same row is the same id whenever and however often it is emitted; a restored store that re-issues aseqthe lost one had published gives it a different one.type—io.ankusa.source.{created,updated,deleted},io.ankusa.route.{created,updated,deleted},io.ankusa.tenant.deleted(a tenant erase,Ankusa.Tenant, after thedeletedevent of each source and route it removed; itsdatais{"tenant": <id>}and itssubjectthe tenant id), orio.ankusa.instance.restored(below)source—urn:ankusa:instance:<instance>subject— the source id (<tenant>.<name>) or the route idtime— when the change committed (the row's time, not the publisher's)data— the entity as the admin API shows it, redacted byAnkusa.Admin.Redact, as the change left it; for a deletion, as it was the moment beforerevision— the subject's change counter (seeAnkusa.Audit): a consumer keeps the highest it has seen persubjectand ignores a lower one. Absent for a store that keeps none (the Redis source store)actor— the principal id that made the change (systemfor code)instance— the instance namesequence— the auditseq, a decimal string: events of one instance are totally ordered by itreplayid— only on an event a replay job re-emittedtraceparent,tracestate— the CloudEvents distributed-tracing extension, only when an OpenTelemetry SDK runs: the context of theankusa.lifecycle.eventspan that built the event (traced_envelope/4). Not part of the event's identity: a re-emitted event keeps itsidand gets the trace of the emission
A route event's envelope carries the route's tenant (tenant_id; "default"
for a route that belongs to none), so a replay filtered by tenant and a
tenant-limited reader of GET /v1/audit see their own routes' changes.
A restore is announced
A store restored from a backup (Ankusa.Store.Backup) resumes the audit trail
at the backup's seq, so it re-issues sequence numbers, and per-subject
revisions, that the lost node had already published. Before it takes any
change it emits io.ankusa.instance.restored (subject = the instance name;
data = incarnation, the fresh id the following event ids are keyed with,
and restored_sequence, the last seq the restored trail held). A consumer
that keeps the highest sequence or revision it has seen must reset that
state for this instance when it sees it: ordering restarted at
restored_sequence + 1. Events of the restored trail itself keep the ids they
were published under.
A change nobody saw finish
A source in the Redis store or a route is a dual write (see Ankusa.Audit),
so a node killed in the middle leaves a pending row. reconcile/2 settles
it from the subject's present state instead of guessing: the state hashes to
the row's after_hash, the change was applied ("ok", announced); to its
before_hash, it was not ("failed", not announced); to neither, somebody
changed it meanwhile, and the row stays "interrupted" and announces a
state-refresh event carrying the current entity and revision.
Summary
Types
A Source with no tenant: each lifecycle event names its own.
Functions
Whether instance announces changes (it has lifecycle sinks).
The event of an audit row flagged for publication, as the Ankusa.Envelope a
lifecycle sink delivers. entity is what Ankusa.Audit.entity/2 returned for
the row; replay_id is set only by a replay job; trace is the
Ankusa.Trace the event travels under (the CloudEvents traceparent and
tracestate extensions, and Ankusa.Envelope.trace).
Whether a row's event falls in a replay job's filter (:since and :until
bound the row's time, :tenants is the list of tenants it may belong to).
The audit attrs that put a change in the outbox: %{entity: entity} when
lifecycle events are on, %{} when they are off. entity is a function so
that nothing is redacted for an instance that announces nothing.
Whether reconcile/2 can settle a pending audit row: the change of a source
or a route that lives outside this node's store.
Settle a pending row from the subject's present state: {:ok, update} for
Ankusa.Audit's finish, :not_ready while the subject's store has not
loaded (ask again), :skip for a row that is not a source or route change.
The lifecycle pseudo-source for instance, or :error when lifecycle events
are off (no sinks).
The reserved source id lifecycle events travel under.
The tenant a row's event is delivered under: the row's own (a source's tenant, or the tenant that owns a route), the default one for a row that has none.
envelope/5 for the publisher and replay jobs: the event is built inside an
ankusa.lifecycle.event span, and travels under that span's context. With no
OpenTelemetry SDK running it is envelope/5 with no trace.
The CloudEvents type of an audit row: its action, unless a reconciliation
settled on another verb (row.event).
Reject a lifecycle configuration that cannot work. Raises ArgumentError.
Types
@type lifecycle_source() :: %Ankusa.Source{ dedupe: term(), forward_headers: term(), forward_headers_max_bytes: term(), id: term(), max_headers: term(), on_verify_failure: term(), ordering: term(), retry: term(), sinks: term(), store_headers: term(), tenant_id: nil, trust_url_tenant: term(), verifier: term() }
A Source with no tenant: each lifecycle event names its own.
Functions
Whether instance announces changes (it has lifecycle sinks).
@spec envelope( atom(), Ankusa.Audit.row(), map(), String.t() | nil, Ankusa.Trace.t() | nil ) :: {:ok, Ankusa.Envelope.t()} | {:error, :no_audit_key}
The event of an audit row flagged for publication, as the Ankusa.Envelope a
lifecycle sink delivers. entity is what Ankusa.Audit.entity/2 returned for
the row; replay_id is set only by a replay job; trace is the
Ankusa.Trace the event travels under (the CloudEvents traceparent and
tracestate extensions, and Ankusa.Envelope.trace).
Pure in its arguments: the same row gives the same event id, time and body
however often, and wherever on the instance, it is built.
@spec in_window?(Ankusa.Audit.row(), map()) :: boolean()
Whether a row's event falls in a replay job's filter (:since and :until
bound the row's time, :tenants is the list of tenants it may belong to).
The audit attrs that put a change in the outbox: %{entity: entity} when
lifecycle events are on, %{} when they are off. entity is a function so
that nothing is redacted for an instance that announces nothing.
@spec reconcilable?(Ankusa.Audit.row()) :: boolean()
Whether reconcile/2 can settle a pending audit row: the change of a source
or a route that lives outside this node's store.
@spec reconcile(atom(), Ankusa.Audit.row()) :: {:ok, map()} | :not_ready | :skip
Settle a pending row from the subject's present state: {:ok, update} for
Ankusa.Audit's finish, :not_ready while the subject's store has not
loaded (ask again), :skip for a row that is not a source or route change.
@spec source(atom()) :: {:ok, lifecycle_source()} | :error
The lifecycle pseudo-source for instance, or :error when lifecycle events
are off (no sinks).
Built as a struct literal: Ankusa.Source.new/2 requires a tenant, and the
tenant of a lifecycle event varies per event.
@spec source_id() :: String.t()
The reserved source id lifecycle events travel under.
@spec tenant(Ankusa.Audit.row()) :: String.t()
The tenant a row's event is delivered under: the row's own (a source's tenant, or the tenant that owns a route), the default one for a row that has none.
@spec traced_envelope(atom(), Ankusa.Audit.row(), map(), String.t() | nil) :: {:ok, Ankusa.Envelope.t()} | {:error, :no_audit_key}
envelope/5 for the publisher and replay jobs: the event is built inside an
ankusa.lifecycle.event span, and travels under that span's context. With no
OpenTelemetry SDK running it is envelope/5 with no trace.
@spec type(Ankusa.Audit.row()) :: String.t()
The CloudEvents type of an audit row: its action, unless a reconciliation
settled on another verb (row.event).
@spec validate_config!(Ankusa.Config.t()) :: :ok
Reject a lifecycle configuration that cannot work. Raises ArgumentError.
The reserved source id must not be declared as a static source: it would let any provider post hooks that consumers read as lifecycle events.