Tenant erase: remove everything this node's store, the claim stores and the edge's tables hold for one tenant, as one resumable, audited job.
Ankusa.Tenant.delete(:default, "acme", confirm: "acme", audit: context)DELETE /v1/tenants/:tenant?confirm=<tenant> (scope tenants:write and
tenant:*) is the same call.
What an erase does, in order
delete/3 writes the job's progress row (m:erase:<tenant>, synced) and
answers; Ankusa.Tenant.Eraser runs the job in the background. Its first act
is to close the tenant (the tenant gate): from then on the queue
writer answers the tenant's hooks 503 tenant_erasing, the quarantine pen
refuses to hold them, and dispatch stops claiming its rows. Then, each stage
done and recorded before the next:
| Stage | Removes |
|---|---|
sources | the tenant's API-managed sources, through Ankusa.SourceStore.delete/4: each is an audited change with its own source.deleted lifecycle event |
routes | the tenant's routes (Ankusa.Routes.delete/3: one route.deleted event each) |
rate_limit | the tenant's runtime rate-limit override |
queue | waits for running attempts to settle (the tenant's inflight counter reaches 0), then every due, inflight and dead row (Ankusa.Queue.purge_tenant/3), reclaiming hooks nobody else holds, and the tenant's active markers |
archive | the tenant's archive obligations (?a), reclaiming their hooks |
quarantine | the tenant's held hooks |
claims | every object under claims/tenant=<tenant>/ in every claim store |
dedupe | the tenant's dedupe keys (?u, a range delete) and their expiry entries |
entities | the redacted entities of the tenant's lifecycle events, once the lifecycle publisher has delivered them |
commit | one synced batch: the tombstone m:erased:<tenant>, the job's progress row, the tenant's ordinal (retired, never reused), its counters and source revisions, and one audit row (tenant.delete, which becomes the lifecycle event io.ankusa.tenant.deleted) |
A stage that fails (a store that does not answer, a sink that is still running) is retried every few seconds; a node that is killed resumes the job from its progress row at the next boot. Every stage is idempotent.
What stays, and why
- The audit trail. Its rows hold hashes, counts and names, never a body
or a secret (
Ankusa.Audit); they are the record that the erase happened and who asked for it, and they age out withaudit.retention_days. Only the lifecycle entities (the redacted views of the tenant's sources and routes that events carry) are deleted. - Archived segments. A segment is multi-tenant and immutable. The
tombstone makes
Ankusa.Storage.fetch/2answer{:error, :gone}(HTTP410semantics) for every hook of the tenant received at or before it, and archive replay skip them; the bytes leave the bucket when the bucket's lifecycle rule expires the segment (docs/storage.md). - Backups. A restored backup that predates the erase brings the tenant back, tombstone and all: erase again after such a restore.
- The ordinal. Ordinals are never reused (ADR 0001), so an erased tenant
still counts towards
tenants.max. - Replay job records that name the tenant in their filter: they hold no hook data.
A tenant that sends again after the erase is a new tenant; the tombstone only covers what was received before it.
Limits
"default" cannot be erased. A tenant that a configured source (static,
or seeded into the source store) names cannot be erased either: remove it
from the configuration first, or the next hook would recreate the tenant.
The erase needs the :edge role on this node when routes are enabled (the
route store is the edge's). Sources, routes and overrides written for the
tenant while its erase runs can outlive it; the commit stage re-checks
them right before it commits and the job fails and retries until none is left.
Summary
Functions
The sources that name tenant and that an erase cannot delete: configured
ones (the static source store, or seeds of the writable one), as opposed to
the ones written through the API. Walks the source ids, so it costs the number
of sources.
Start erasing tenant, or report the erase already running.
Every tombstone: {:ok, %{tenant => erased_at_ms}}. One range of the store,
read once per page by the archive and lifecycle replays.
Whether a hook of tenant received at received_at was erased, given the
tombstones erased/1 returned.
When tenant was erased: {:ok, ms} or :not_found.
Where an erase of tenant stands: {:ok, %{state: :erasing, stage: atom, started_at: ms}} while its job exists, {:ok, %{state: :erased, erased_at: ms}} once committed, :not_found for a tenant that was never erased here.
Whether a lifecycle replay must not re-emit the audit row row: it belongs
to a tenant that is being erased, or one that was erased after the row was
written. The erase's own tenant.delete row is the one exception.
Types
@type reason() :: :confirm | :invalid_tenant | :default_tenant | :no_store | :store_unavailable | {:role_not_enabled, :edge} | {:static_sources, [String.t()]}
Functions
The sources that name tenant and that an erase cannot delete: configured
ones (the static source store, or seeds of the writable one), as opposed to
the ones written through the API. Walks the source ids, so it costs the number
of sources.
Start erasing tenant, or report the erase already running.
Options: confirm: (required: must equal tenant) and audit:, the
Ankusa.Audit.Context the erase's audit rows are written under (the actor of
the request, which is kept across a restart).
{:ok, :started} or {:ok, :erasing} (one is already running for it).
@spec erased(atom()) :: {:ok, %{required(String.t()) => non_neg_integer()}} | {:error, term()}
Every tombstone: {:ok, %{tenant => erased_at_ms}}. One range of the store,
read once per page by the archive and lifecycle replays.
@spec erased?( %{required(String.t()) => non_neg_integer()}, String.t() | nil, integer() ) :: boolean()
Whether a hook of tenant received at received_at was erased, given the
tombstones erased/1 returned.
@spec erased_at(atom(), String.t()) :: {:ok, non_neg_integer()} | :not_found | {:error, term()}
When tenant was erased: {:ok, ms} or :not_found.
@spec status(atom(), String.t()) :: {:ok, %{state: :erasing | :erased} | map()} | :not_found | {:error, term()}
Where an erase of tenant stands: {:ok, %{state: :erasing, stage: atom, started_at: ms}} while its job exists, {:ok, %{state: :erased, erased_at: ms}} once committed, :not_found for a tenant that was never erased here.
@spec suppressed?(atom(), %{required(String.t()) => non_neg_integer()}, map()) :: boolean()
Whether a lifecycle replay must not re-emit the audit row row: it belongs
to a tenant that is being erased, or one that was erased after the row was
written. The erase's own tenant.delete row is the one exception.