The only seq assigner: commits batches of hooks to the store, and answers a
caller only once its hooks are on disk (Ankusa.Queue.Syncer fsyncs the WAL),
so a 2xx to the provider always means the bytes are on disk.
One process per instance, under Ankusa.via(instance, :queue_writer). The
commit is a single atomic store batch: the hook, one pending delivery row and
due key (with its active-tenant marker) per sink, the archive obligation when
the :storage role runs, the counter deltas of all of them, the m:next_seq
marker and, for a tenant seen for the first time, its ordinal (this process is
the only one that allocates them, see queue/tenants.ex). A failed commit
consumes its seqs and ordinals (gaps are allowed) but acks nothing. A hook
whose tenant would pass tenants.max is refused with {:error, :tenant_limit} in its place of the reply, and nothing of it is written.
Group commit
A call carries what needs no seq, prepared by its caller (Ankusa.Queue: the
hook's binary, its encoded delivery rows, the dedupe keys read from the store).
The writer takes the call that woke it and every :enqueue call queued behind
it, from every batcher partition, up to batcher.group_commit_max_bytes of
bodies and batcher.group_commit_max_records hooks, and commits them as one
group: one tenant admission, one dedupe validation, dense seqs across all
of it, one store write (applied with sync: false) and, under wal: :dual,
one publish. The writer hands every caller's reply to Ankusa.Queue.Syncer,
which fsyncs the WAL once for all the groups applied so far and then answers
them: while group N syncs, group N+1 is assembled and applied here. A group's
rows are not visible to dispatch, the archive or a retry's 201 duplicate
before that fsync (Syncer.visible/2, ADR 0002 amendment 17).
The dedupe values a caller read are a prefetch: the first occurrence of a key
in the group wins, and a key this process wrote since the caller read (in an
earlier group, or in flight under wal: :dual) is answered from recent_* and
the pending set, never from the stale read, so two partitions that saw the same
key as absent still commit one hook.
Under wal: :dual (ADR 0002) the local commit runs in parallel with a
confirmed publish to the intake log (Ankusa.Intake), and both are required
before the 201. The writer does not wait: it records the batch in the
pending set, returns {:noreply, _}, and answers the batcher's call when the
publish task's result and the syncer's confirm of the local half have both
arrived, so the next batch's local write overlaps the previous batch's fsync
and publish wait. A half that fails is carried to completion
by Ankusa.Intake.Reconciler, never undone; the caller's reply for such a
hook is {:error, {:half, reason, hook_id}}.
Summary
Functions
Returns a specification to start this module under a supervisor.
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.