# ankusa v0.7.0 - Table of Contents > A loosely coupled, high-throughput webhook ingestion framework: durable WAL, pluggable verification/storage/delivery, multi-tenant catch-URL routing. ## Pages - [Ankusa](readme.md) - [Changelog](changelog.md) - Guides - [Ankusa for Elixir](elixir.md) - [Quickstart](quickstart.md) - [Configuration](configuration.md) - [Deployment](deployment.md) - [Architecture](architecture.md) - [Delivery: dispatch, sinks, retries, dead letters](delivery.md) - [Using Ankusa with a job framework, without coupling to one](integrations.md) - [Storage: the local store and the object store](storage.md) - [Claim check](claim-check.md) - [Multi-tenancy and catch-URL routing](multi-tenancy.md) - [AsyncAPI and lifecycle events](asyncapi.md) - Contributing - [Packaging: why this is a mono-repo of separate Mix projects](packaging.md) - [Testing](testing.md) - [Releasing](releasing.md) ## Modules - [Ankusa](Ankusa.md): A loosely coupled, high-throughput webhook ingestion framework. - [Ankusa.AWS.Credentials](Ankusa.AWS.Credentials.md): Where `Ankusa.BlobStore.S3` and `Ankusa.Sink.SQS` get the credentials they sign with, in the order the AWS SDKs look - [Ankusa.Admin.Auth](Ankusa.Admin.Auth.md): Authentication and authorization for the three control planes: the admin API (`Ankusa.Admin.Router`), the route-management API (`Ankusa.Routes.Router`) and the claim-check gateway (`Ankusa.ClaimCheck.Router`). - [Ankusa.Admin.Auth.JWT](Ankusa.Admin.Auth.JWT.md): OIDC / OAuth2 bearer tokens (`mode: :jwt`): the control planes accept the access tokens your identity provider issues, so operators sign in with SSO and revocation is the provider's. - [Ankusa.Admin.Auth.JWT.Keys](Ankusa.Admin.Auth.JWT.Keys.md): The signing keys `Ankusa.Admin.Auth.JWT` verifies against, for every control plane of an instance that runs `mode: :jwt`. Started with the instance (only when some listener needs it). - [Ankusa.Admin.Auth.MTLS](Ankusa.Admin.Auth.MTLS.md): Mutual TLS (`mode: :mtls`): the identity is the client certificate the listener's `tls: %{verify: :verify_peer}` handshake accepted (`Ankusa.Listener`), so a request with no certificate or one from another CA never reaches this module. What the certificate may do is configured - [Ankusa.Admin.Auth.Token](Ankusa.Admin.Auth.Token.md): Static bearer tokens (`mode: :token`), for development, CI and scripts. Use `Ankusa.Admin.Auth.JWT` or `Ankusa.Admin.Auth.MTLS` for people and for anything that must be revocable. - [Ankusa.Admin.Redact](Ankusa.Admin.Redact.md): Turn a `%Ankusa.Config{}` into a JSON-encodable map safe to hand to anyone who can reach the admin API. - [Ankusa.Admin.Router](Ankusa.Admin.Router.md): The operator HTTP API: health, Prometheus metrics, the redacted config, the AsyncAPI document of the channels this instance publishes to, the dead-letter queue, replay jobs, and this node's quarantine pen (list and purge). - [Ankusa.AsyncApi](Ankusa.AsyncApi.md): The AsyncAPI 3.0 document for a running instance: every messaging channel it publishes to, built from what is configured right now. - [Ankusa.AsyncApi.LifecycleEvent](Ankusa.AsyncApi.LifecycleEvent.md): A lifecycle event (`Ankusa.Lifecycle`): a CloudEvents 1.0 structured-mode event. It is the decoded `body_base64` of a lifecycle channel's `SinkMessageV1`. - [Ankusa.AsyncApi.SinkMessage](Ankusa.AsyncApi.SinkMessage.md): The JSON body every queue-style sink publishes: `Ankusa.Sink.Message` version 1, as an AsyncAPI schema. Declared once; `Ankusa.AsyncApi` refers to it from every message. - [Ankusa.AsyncApi.SinkMessageHeaders](Ankusa.AsyncApi.SinkMessageHeaders.md): The application headers a sink with transport headers carries beside a `SinkMessageV1` body: Kafka record headers, NATS headers, and the message attributes of SQS and Google Pub/Sub. They are `Ankusa.Sink.Message.headers/2` (Kafka, NATS) or `Ankusa.Sink.Message.attributes/2` (SQS, Pub/Sub), which differ in one name: `ankusa_tenant_id` is `""` on Kafka and NATS and absent on SQS and Pub/Sub (both refuse an empty value), so it is not required, and a consumer reads the tenant from the message body's `tenant_id` when the header is missing. Redis carries none; RabbitMQ sends only `ankusa_idempotency_key` and the optional dedupe, replay and ordering keys as AMQP headers, which this schema does not describe for it. The document's description of each channel (`Ankusa.AsyncApi`) says which set it carries. - [Ankusa.Audit](Ankusa.Audit.md): The audit trail: one row for every change an operator (or code) makes to the node's configuration or operational state, written with the change. - [Ankusa.Audit.Context](Ankusa.Audit.Context.md): Who is making a change, from where, and under which request: the part of an audit row (`Ankusa.Audit`) that the mutation itself does not know. - [Ankusa.BlobStore](Ankusa.BlobStore.md): Object storage: `PUT` whole immutable objects, range `GET` one record, `DELETE` for retention, `LIST` by prefix. The default is the local filesystem; S3, GCS, Azure Blob and OCI are drop-in adapters. - [Ankusa.BlobStore.Azure](Ankusa.BlobStore.Azure.md): `Ankusa.BlobStore` backed by Azure Blob Storage, via [`Req`](https://hex.pm/packages/req). Works against real Azure or the [Azurite](https://learn.microsoft.com/azure/storage/common/storage-use-azurite) emulator. - [Ankusa.BlobStore.Azure.ManagedIdentity](Ankusa.BlobStore.Azure.ManagedIdentity.md): Azure AD access tokens from the Instance Metadata Service (IMDS) — Azure's *managed identity*, the best credential for a service running on Azure: no secret to store or rotate, short-lived tokens, and least privilege via a role assignment (`Storage Blob Data Contributor`) on the identity rather than an account key. - [Ankusa.BlobStore.Breaker](Ankusa.BlobStore.Breaker.md): A circuit breaker per `{endpoint, bucket}` for the HTTP object stores, shared by every instance and every store on the node that talks to that bucket. - [Ankusa.BlobStore.Checksum](Ankusa.BlobStore.Checksum.md): The integrity headers the HTTP object stores take on a `PUT`, and the check of the digest they send back on a `GET`. - [Ankusa.BlobStore.GCS](Ankusa.BlobStore.GCS.md): `Ankusa.BlobStore` backed by Google Cloud Storage's JSON API, via [`Req`](https://hex.pm/packages/req). Works against real GCS or the `floci-gcp` emulator in `docker-compose.integration.yml`. - [Ankusa.BlobStore.GCS.MetadataServer](Ankusa.BlobStore.GCS.MetadataServer.md): Access tokens for `Ankusa.BlobStore.GCS` from the GCE/GKE metadata server: the best credential for a service running on Google Cloud, with no key to store or rotate (the node's service account, granted `roles/storage.objectAdmin` on the bucket). - [Ankusa.BlobStore.LocalFS](Ankusa.BlobStore.LocalFS.md): Default `Ankusa.BlobStore`: immutable objects on the local filesystem. - [Ankusa.BlobStore.OCI](Ankusa.BlobStore.OCI.md): `Ankusa.BlobStore` backed by Oracle Cloud Infrastructure Object Storage, via [`Req`](https://hex.pm/packages/req). - [Ankusa.BlobStore.OCI.InstancePrincipal](Ankusa.BlobStore.OCI.InstancePrincipal.md): Session credentials for `Ankusa.BlobStore.OCI` from the instance's own identity, for a node on OCI compute or OKE: no API key to store or rotate. - [Ankusa.BlobStore.Options](Ankusa.BlobStore.Options.md): The transport options every HTTP object store takes (`Ankusa.BlobStore.S3`, `GCS`, `Azure`, `OCI`), their defaults and their ranges. They are adapter options, like `:timeout_ms` and `:retries`: Elixir config, and the `s3`/`gcs` sections of the YAML. - [Ankusa.BlobStore.Pool](Ankusa.BlobStore.Pool.md): The connection pool of each HTTP object store's endpoint, owned by the instance (`Ankusa.Instance` starts one [`Finch`](https://hex.pm/packages/finch) per distinct endpoint of `storage.blob_store`, `backup.blob_store`, `backup.secondary` and `claim_check.blob_stores`). - [Ankusa.BlobStore.Retry](Ankusa.BlobStore.Retry.md): Retries one object-store request after a transient failure, for the HTTP blob stores (`Ankusa.BlobStore.S3`, `GCS`, `Azure`, `OCI`), under one deadline and a per-`{endpoint, bucket}` breaker (`Ankusa.BlobStore.Breaker`). - [Ankusa.BlobStore.S3](Ankusa.BlobStore.S3.md): `Ankusa.BlobStore` backed by S3 or any S3-compatible endpoint (MinIO, Cloudflare R2, or the `floci` emulator in `docker-compose.integration.yml`). - [Ankusa.BlobStore.SSE](Ankusa.BlobStore.SSE.md): Server-side encryption for the objects an `Ankusa.BlobStore` writes: the `:sse` option of `Ankusa.BlobStore.S3`, `GCS`, `Azure` and `OCI`. - [Ankusa.ClaimCheck](Ankusa.ClaimCheck.md): Claim check: store payloads too large to ride inline in a queue message, and hand each one back by reference. - [Ankusa.ClaimCheck.Cache](Ankusa.ClaimCheck.Cache.md): The claim gateway's memory of what it has read: pack indexes, pack digests and whole claim bodies, so a repeat read never reaches the object store. - [Ankusa.ClaimCheck.Pack](Ankusa.ClaimCheck.Pack.md): Builds and indexes a claim-check pack: a standard ZIP archive with every entry stored uncompressed, laid out as - [Ankusa.ClaimCheck.Reader](Ankusa.ClaimCheck.Reader.md): The one process every store read of a `:claim_check` node passes through, so the gateway stays fast while an object store throttles or hangs - [Ankusa.ClaimCheck.Ref](Ankusa.ClaimCheck.Ref.md): A claim-check reference: one string that names a claim's bytes. - [Ankusa.ClaimCheck.Stores](Ankusa.ClaimCheck.Stores.md): The object stores claim packs live in — `claim_check.blob_stores`, else `[storage.blob_store]` — and which of them a pack is written to and read from. Stores are named by their index in that list. Claim keys are never prefixed, so one gateway reads every node's claims. - [Ankusa.Codec](Ankusa.Codec.md): Segment record framing. `encode/1` packs many records into one immutable segment binary and returns a per-record index (byte offset + length within the segment). A single range `GET` on the blob store then reads exactly one record, which `decode_record/1` validates and unframes. - [Ankusa.Codec.Raw](Ankusa.Codec.Raw.md): Default `Ankusa.Codec`: length-prefixed, CRC-checked framing with no compression. - [Ankusa.Config](Ankusa.Config.md): The configuration struct passed down the supervision tree at start. No `Application.get_env/2` buried in call sites — instance-scoped config falls out of this for free. - [Ankusa.Crypto.Envelope](Ankusa.Crypto.Envelope.md): Envelope encryption for the secrets Ankusa stores: a source's `verify.secret` and its sinks' credentials (URLs with a password, header values, tokens, keys), which `Ankusa.SourceStore.Persistent` keeps in this node's RocksDB store (and so in every backup of it) and `Ankusa.SourceStore.Redis` keeps in Redis. - [Ankusa.Crypto.Envelope.Keyring](Ankusa.Crypto.Envelope.Keyring.md): The keys resolved from a `secrets` section (`Ankusa.Crypto.Envelope.keyring/1`): the current key and the id it is named by, and every key that may decrypt. `inspect/1` shows the current key's id only, so a crash report that prints a store's state never prints a key. - [Ankusa.Dedupe](Ankusa.Dedupe.md): Per-source provider event keys. - [Ankusa.Dedupe.Store](Ankusa.Dedupe.Store.md): The second phase of a fleet dedupe claim (`dedupe.scope: :fleet`). - [Ankusa.Dispatch.Replayer](Ankusa.Dispatch.Replayer.md): Durable, paced replay jobs over the delivery rows. - [Ankusa.Edge.Fleet](Ankusa.Edge.Fleet.md): Fleet-scoped admission: the tenant quota (`rate_limits.scope: :fleet`) and the provider event claim (`dedupe.scope: :fleet`), counted in `fleet.store` so N edge nodes enforce one limit and one dedupe window instead of N. - [Ankusa.Edge.Publish](Ankusa.Edge.Publish.md): The `wal: :none` ack path: publish one verified envelope to every sink its source declares, in the request process, and answer only once each has confirmed. - [Ankusa.Edge.RateLimiter](Ankusa.Edge.RateLimiter.md): Ingest admission, enforced in this node's memory. Two layers, in the order a request meets them - [Ankusa.Edge.RouteGuard](Ankusa.Edge.RouteGuard.md): The ingress gate: with routes enabled, a `POST` is captured only if it passes the IP rules and matches an enabled route. - [Ankusa.Edge.StoredHeaders](Ankusa.Edge.StoredHeaders.md): Which provider request headers a hook is *stored* with. - [Ankusa.Envelope](Ankusa.Envelope.md): The claim-checked record that flows through every layer. - [Ankusa.Health](Ankusa.Health.md): Readiness, as `GET /ready` on the ingress, admin and metrics listeners answers it. - [Ankusa.Health.Check](Ankusa.Health.Check.md): One thing `GET /ready` asks about (`Ankusa.Health`). - [Ankusa.Health.Check.Backlog](Ankusa.Health.Check.Backlog.md): `backlog`, with `queue.max_oldest_due_ms` set: the oldest due delivery row has waited no longer than that (the figure `ankusa_queue_oldest_due_age_seconds` reports, from the same read). Over it the reason is `over_threshold`. - [Ankusa.Health.Check.Backup](Ankusa.Health.Check.Backup.md): `backup`, with `backup.enabled` and `backup.max_age_ms`: the last successful backup is no older than `max_age_ms`. Failing (reason `stale`) when it is older, or when the uploader is not running. Gates only with `backup.require_fresh`; otherwise it is a label on a node that stays ready. - [Ankusa.Health.Check.Disk](Ankusa.Health.Check.Disk.md): `disk`, with `listener.ready_min_free_bytes` set (YAML `node.min_free_bytes`): the file system holding `data_dir` has at least that many bytes free. Below it the reason is `below_floor`, and the node is unready *before* the store hits `ENOSPC`, which is what lets the load balancer drain it while the operator still has room to act. - [Ankusa.Health.Check.Drain](Ankusa.Health.Check.Drain.md): `drain`, only while the node is being stopped (`Ankusa.Health.drain/1`, which `Ankusa.Instance.Drain` calls first when the VM is told to stop): the state is `draining`, reason `shutting_down`, and `/ready` answers `503` with `status: "draining"`. A node that is not stopping has no `drain` key. - [Ankusa.Health.Check.Intake](Ankusa.Health.Check.Intake.md): `intake`, under `wal: :dual` only: the intake log answers. The reason is `down` (the breaker is open and the log is not answering: the load balancer drains the node) or `recovering` (boot replay is running). - [Ankusa.Health.Check.Sinks](Ankusa.Health.Check.Sinks.md): `sinks`, under `wal: :none` (the direct ack path) only: no sink failed its last publish within `listener.ready_sink_error_window_ms`. Reason `publish_failed`. - [Ankusa.Health.Check.Store](Ankusa.Health.Check.Store.md): `store`: on a node that runs the local store, whether it takes a synced write and no write failed in the last 5 s (`Ankusa.Store.ready/1`). The reason is the store's failure class (`Ankusa.Store.failure_class/1`: `no_space`, `io_error`, `read_only`, `corruption`, `busy`, `other`, plus `timeout` and `not_running` when it does not answer). - [Ankusa.Health.SinkErrors](Ankusa.Health.SinkErrors.md): The last failed publish of every sink under `wal: :none`, for `Ankusa.Health.Check.Sinks`. - [Ankusa.HttpClient.Egress](Ankusa.HttpClient.Egress.md): The egress policy: which URLs Ankusa may send a hook to (CWE-918). - [Ankusa.HttpClient.Pools](Ankusa.HttpClient.Pools.md): The outbound HTTP connection pools: one [`Finch`](https://hex.pm/packages/finch) for the node (`Ankusa.HttpClient.Finch`, started with the application) that every `Ankusa.HttpClient` request without a transport of its own goes through, and the bookkeeping that keeps its pools bounded. - [Ankusa.Instance](Ankusa.Instance.md): Supervises one instance of the framework from a `%Ankusa.Config{}`. Only the children for configured roles boot, so the same code runs any role set. The boundaries between components are durable state (this node's store), not function calls. The store is node-local, so every role that reads or writes hooks has to run on the node that holds it. - [Ankusa.Instance.Drain](Ankusa.Instance.Drain.md): What a node does before its supervision tree stops, so a stop never turns a request that is being answered, or a hook that is being committed, into a connection reset. Both applications run it from `prep_stop/1` (core's for every instance in its own tree, `AnkusaServer.Application`'s for the one it owns), which the VM calls when it is told to stop: SIGTERM, which is what `docker stop` and a Kubernetes pod deletion send. - [Ankusa.Intake](Ankusa.Intake.md): The intake log behind `wal: :dual` (ADR 0002): a replicated log an acked hook is also confirmed in, so a lost host does not lose it. - [Ankusa.Intake.Gate](Ankusa.Intake.Gate.md): The intake breaker of `wal: :dual` (ADR 0002, section 7). Under dual there is no degraded mode: a node that cannot confirm in the log refuses, it never acks on local fsync alone. - [Ankusa.Intake.Memory](Ankusa.Intake.Memory.md): An in-memory `Ankusa.Intake` for tests, benches and the fleet drill's fake. Not a production mode: its "replicas" are one process. - [Ankusa.Intake.Pending](Ankusa.Intake.Pending.md): The pending set of `wal: :dual` (ADR 0002, section 5): one entry per record not yet confirmed in **both** the local store and the intake log, and the small state the dual commit shares between the writer, the reconciler, the breaker and readiness. - [Ankusa.Intake.Reconciler](Ankusa.Intake.Reconciler.md): Carries half-persisted hooks to completion (ADR 0002, sections 3 and 5). Under `wal: :dual` a record the writer issued to both places may end with only one of them: the local store (the log refused or timed out) or the log (the store write failed). The provider got a `503`, and the half that persisted still exists, so the record is **completed, never undone** - [Ankusa.Intake.Recovery](Ankusa.Intake.Recovery.md): Boot recovery of `wal: :dual` (ADR 0002, section 8), and the same code offline as `mix ankusa.recover`. - [Ankusa.Lifecycle](Ankusa.Lifecycle.md): Lifecycle events: a message when a webhook endpoint (a *source*) or a route is created, updated, or deleted. - [Ankusa.Lifecycle.Publisher](Ankusa.Lifecycle.Publisher.md): Delivers lifecycle events (`Ankusa.Lifecycle`) to `config.lifecycle.sinks`: the reader of the outbox. - [Ankusa.Listener](Ankusa.Listener.md): What every HTTP listener the instance runs (the ingest edge, the admin API, the route-management API, the claim-check gateway, the metrics listener) has in common: the connection limits and timeouts of `Ankusa.Config`'s `listener` section, and the optional TLS block each listener carries. - [Ankusa.LogLevel](Ankusa.LogLevel.md): The VM's `Logger` level, changed while the node runs. - [Ankusa.Metrics](Ankusa.Metrics.md): The built-in Prometheus mapping of `Ankusa.Telemetry`'s events, and the reporter that serves it. - [Ankusa.Metrics.Cardinality](Ankusa.Metrics.Cardinality.md): The series ceiling of `Ankusa.Metrics`: how many distinct label sets one metric may keep before the rest are folded into `"other"`. - [Ankusa.Metrics.Gauges](Ankusa.Metrics.Gauges.md): Periodic state measurements for `Ankusa.Metrics`: what an operator alarms on, as opposed to the event counters. - [Ankusa.Metrics.Router](Ankusa.Metrics.Router.md): The metrics listener (`metrics.enabled`, default port 4004): `GET /metrics`, `GET /health` and `GET /ready`, and a `404` for everything else. A scraper or a kubelet probe gets a route to those three and none to the operator API (`Ankusa.Admin.Router`), which mutates sources, replays and the pen and so belongs behind a different network rule than a Prometheus job. - [Ankusa.Net](Ankusa.Net.md): IP addresses in the form the route-management hot path and the `cidr` package already share: an `:inet.ip_address()` tuple — a 4-tuple for IPv4, an 8-tuple for IPv6. - [Ankusa.Net.ClientIP](Ankusa.Net.ClientIP.md): Decide which address a request actually came from. - [Ankusa.Ordering](Ankusa.Ordering.md): Opt-in ordering lanes (ADR 0004): what a source's sink entry may say about order, and how a hook's lane is resolved at ingest. - [Ankusa.Principal](Ankusa.Principal.md): Who is calling a control plane, and what they may do: the result of `c:Ankusa.Admin.Auth.authenticate/2`, assigned to `conn.assigns.principal` for every handler of the admin API, the route-management API and the claim-check gateway. - [Ankusa.Queue](Ankusa.Queue.md): The hook queue: the public face of the store's hook and delivery machinery. - [Ankusa.Queue.DedupeSweeper](Ankusa.Queue.DedupeSweeper.md): Expires ingest-dedupe keys (`?u`, with their `?e` expiry index) once their ttl has passed, so the dedupe index stays bounded by `dedupe.max_entries`. - [Ankusa.Queue.DlqReaper](Ankusa.Queue.DlqReaper.md): Keeps the dead-letter queue bounded (`queue.dlq`), without ever discarding a hook the provider was told `201` for. - [Ankusa.Queue.Syncer](Ankusa.Queue.Syncer.md): The fsync of the queue's WAL, off the writer's critical path (DUR-4.b). - [Ankusa.RateLimiter.Store](Ankusa.RateLimiter.Store.md): A shared store for the tenant quota and the provider-event claim, behind `rate_limits.scope: :fleet` and `dedupe.scope: :fleet` (`fleet.store`). - [Ankusa.Reload](Ankusa.Reload.md): Changing a running instance's config without a restart: the part of a config reload that core owns. - [Ankusa.Replay](Ankusa.Replay.md): Operator API for replay jobs (`Ankusa.Dispatch.Replayer`). - [Ankusa.RetryPolicy](Ankusa.RetryPolicy.md): Backoff and give-up rules for dispatch. - [Ankusa.RetryPolicy.Exponential](Ankusa.RetryPolicy.Exponential.md): Capped binary exponential backoff with optional full jitter. - [Ankusa.Route](Ankusa.Route.md): The resolved identity of an ingest request: which tenant and source a catch URL maps to. Produced by a `Ankusa.RouteResolver` from the raw `Plug.Conn`, so the URL *scheme* (single path segment, tenant-in-path, opaque token, host routing) is a pluggable concern the rest of the framework never sees. - [Ankusa.RouteResolver](Ankusa.RouteResolver.md): Maps a raw ingest request (`Plug.Conn`) to a `Ankusa.Route`. This is the seam that makes the catch-URL scheme pluggable, so the framework fits a single static server, a multi-tenant SaaS handing every customer their own path, or a product minting opaque catch URLs of its own shape at runtime — without touching the hot path. - [Ankusa.RouteResolver.Path](Ankusa.RouteResolver.Path.md): Default resolver: a single path segment after a fixed prefix is the source id. `POST /webhooks/:source_id` (the original scheme). Tenant is left `nil`, so it is taken from the resolved `%Ankusa.Source{}` — the single-tenant / static case. - [Ankusa.RouteResolver.TenantPath](Ankusa.RouteResolver.TenantPath.md): Multi-tenant resolver: the tenant is carried in the URL. `POST /webhooks/:tenant_id/:source_id` maps to `%Route{tenant_id: t, source_id: s}`, so one instance serves many tenants and the tenant is authoritative from the path rather than inferred from the source. A tenant outside `[A-Za-z0-9_-]{1,64}` doesn't resolve (a `404`): it names a storage partition and a claim-check path segment, so it is refused before anything is written. - [Ankusa.Routes](Ankusa.Routes.md): Route management: the allowlist guard's decisions, the CRUD over route definitions, and the boot validation for `config.routes`. - [Ankusa.Routes.Cache](Ankusa.Routes.Cache.md): The decision cache: `{epoch, method, segments}` → matched route id, or the reason no route matched. - [Ankusa.Routes.Matcher](Ankusa.Routes.Matcher.md): Path patterns: the grammar (`segments/1`), the compiled form (`compile/1`), matching (`match?/2`), and request-path normalization (`normalize/2`). - [Ankusa.Routes.Route](Ankusa.Routes.Route.md): One route definition: which requests the edge captures when route management is on. - [Ankusa.Routes.Router](Ankusa.Routes.Router.md): The route-management API: CRUD over route definitions, the global IP rules, and the dry run. - [Ankusa.Routes.Snapshot](Ankusa.Routes.Snapshot.md): The compiled, in-memory view of the route table: what the guard reads on every request, and what makes a mutation take effect everywhere at once. - [Ankusa.Routes.Store](Ankusa.Routes.Store.md): Where route definitions live. This module is both the **behaviour** every store implements and the instance-scoped **facade** the context calls, the same split as `Ankusa.SourceStore`: the facade resolves `config.routes.store`'s `{module, opts}` and delegates. - [Ankusa.Routes.Store.ETS](Ankusa.Routes.Store.ETS.md): The default route store: definitions in this node's memory, with a hard cap and a config seed. - [Ankusa.Routes.TableOwner](Ankusa.Routes.TableOwner.md): Keeps an instance's route snapshot table alive across a routes-store restart. It creates the table and is its heir: the store writes into it (`Ankusa.Routes.Snapshot` asks for it with `hand_over/2`), and when the store dies the table comes back here instead of being deleted. The restarted store takes it over with `Ankusa.Routes.Snapshot.adopt/1` and resumes from the routes in it, rather than starting again from its config. - [Ankusa.SecretProvider](Ankusa.SecretProvider.md): Where a secret comes from when it is not in the config file: a secret manager. - [Ankusa.Sink](Ankusa.Sink.md): What happens to a delivered hook. The interesting extension point: it lets the framework be a library inside a Phoenix app, a standalone forwarding gateway, or the front door to someone else's pipeline — with the same ingest guarantees. - [Ankusa.Sink.Batcher](Ankusa.Sink.Batcher.md): The batching worker behind `Ankusa.Sink.SQS` (`SendMessageBatch`, 10 entries) and `Ankusa.Sink.GooglePubSub` (`publish`, 1 000 messages): many deliveries to one destination, one HTTP request. - [Ankusa.Sink.Connections](Ankusa.Sink.Connections.md): The broker connections a sink adapter opens on demand. An adapter package starts one tree with `start_link/1` from its `Application`: a `DynamicSupervisor` (`.Supervisor`) that owns the connections and an `Ankusa.Sink.Reaper` (`.Reaper`) that closes idle ones, under `.TopSup`. A sink calls `ensure/5` before every publish. - [Ankusa.Sink.Description](Ankusa.Sink.Description.md): Where a sink delivers, as `c:Ankusa.Sink.describe/2` reports it: the broker, the channel address, and the protocol bindings a consumer needs. - [Ankusa.Sink.GooglePubSub](Ankusa.Sink.GooglePubSub.md): Publishes delivered hooks to a **Google Cloud Pub/Sub** topic (or to anything that speaks Pub/Sub's REST API, such as the `floci-gcp` emulator in `docker-compose.integration.yml`). The message data is `Ankusa.Sink.Message`, byte-identical to what `Ankusa.Sink.Kafka`, `Ankusa.Sink.NATS` and `Ankusa.Sink.SQS` publish: inline up to `:inline_max_bytes`, a claim ticket above it. - [Ankusa.Sink.GooglePubSub.Batcher](Ankusa.Sink.GooglePubSub.Batcher.md): `Ankusa.Sink.GooglePubSub`'s codec for `Ankusa.Sink.Batcher`: up to 1 000 deliveries to one topic become one `topics.publish` request. - [Ankusa.Sink.Http](Ankusa.Sink.Http.md): Forward the raw hook body to an HTTP endpoint, via [`Req`](https://hex.pm/packages/req). - [Ankusa.Sink.Http.Signer](Ankusa.Sink.Http.Signer.md): [Standard Webhooks](https://www.standardwebhooks.com/) signatures for `Ankusa.Sink.Http`'s `:secret` option, so a worker can tell a delivery from this node apart from anything else that can reach its URL. - [Ankusa.Sink.Log](Ankusa.Sink.Log.md): A sink that logs each delivered hook. Useful as a default/no-op. - [Ankusa.Sink.Message](Ankusa.Sink.Message.md): The wire format every queue-style sink publishes (`Sink.RabbitMQ`, `Sink.Kafka`, `Sink.NATS`, `Sink.Redis`, `Sink.SQS`, `Sink.GooglePubSub`), so a consumer parses one format regardless of transport. - [Ankusa.Sink.Messaging](Ankusa.Sink.Messaging.md): What a queue-style sink shares: `use` it, implement `c:publish/4`, and the sink is an `Ankusa.Sink` whose `deliver/3` encodes the hook as `Ankusa.Sink.Message` (inline up to `:inline_max_bytes`, a claim ticket above it — `c:Ankusa.Sink.inline_max_bytes/1` is implemented for you) and hands the payload to `publish/4`. Everything transport-specific — the destination, headers (`Ankusa.Sink.Message.headers/2` / `Ankusa.Sink.Message.attributes/2`), the size gate (`Ankusa.Sink.Message.fits/2`), the connection (`Ankusa.Sink.Connections`) and the error classes — is the sink's. `durable?/1` and `describe/2` are still the sink's to add. - [Ankusa.Sink.Reaper](Ankusa.Sink.Reaper.md): Closes the connections a sink adapter opened and then stopped using. - [Ankusa.Sink.SQS](Ankusa.Sink.SQS.md): Sends delivered hooks to an **Amazon SQS** queue, standard or FIFO (or to anything that speaks SQS's JSON protocol, such as the `floci` emulator in `docker-compose.integration.yml`). The message body is `Ankusa.Sink.Message`, byte-identical to what `Ankusa.Sink.Kafka` and `Ankusa.Sink.NATS` publish: inline up to `:inline_max_bytes`, a claim ticket above it. - [Ankusa.Sink.SQS.Batcher](Ankusa.Sink.SQS.Batcher.md): `Ankusa.Sink.SQS`'s codec for `Ankusa.Sink.Batcher`: up to 10 deliveries to one queue become one SigV4-signed `SendMessageBatch`. - [Ankusa.Source](Ankusa.Source.md): Per-source configuration: how to verify, what to do on a verification failure, and where delivered hooks go. - [Ankusa.SourceStore](Ankusa.SourceStore.md): Source config, secrets, and policy. This module is both the behaviour and the instance-scoped facade (resolves `config.source_store` and delegates). - [Ankusa.SourceStore.Persistent](Ankusa.SourceStore.Persistent.md): Writable source store: seed sources from config plus API-managed, tenant-scoped sources persisted to disk. - [Ankusa.SourceStore.Static](Ankusa.SourceStore.Static.md): Default source store: sources are declared in config as a map of `source_id => keyword/map` (see `Ankusa.Source.new/2`) and cached in `:persistent_term`. Change rarely, read on every request. - [Ankusa.Storage](Ankusa.Storage.md): Read side of long-term segment storage. `fetch/2` resolves an event id to its segment, range-reads exactly its frame from the blob store, and unframes it back into the original `Ankusa.Envelope`. - [Ankusa.Storage.Index](Ankusa.Storage.Index.md): The index object beside a segment: `%{event_id => {offset, length, seq}}`. - [Ankusa.Storage.Layout](Ankusa.Storage.Layout.md): Where archive objects live in the object store. - [Ankusa.Storage.Segment](Ankusa.Storage.Segment.md): The segment object: what the compactor `PUT`s and a replay reads whole. - [Ankusa.Store.Backup](Ankusa.Store.Backup.md): Continuous backup of the node's store to an object store, and its restore into an empty `data_dir` at boot. Off unless `backup.enabled`. - [Ankusa.Telemetry](Ankusa.Telemetry.md): The cross-component contract. Components emit `:telemetry` events; they never call each other's reporters. - [Ankusa.Tenant](Ankusa.Tenant.md): Tenant erase: remove everything this node's store, the claim stores and the edge's tables hold for one tenant, as one resumable, audited job. - [Ankusa.Tenant.Eraser](Ankusa.Tenant.Eraser.md): Runs tenant erases (`Ankusa.Tenant`) in the background, one stage at a time, and keeps the progress of each in the store (`m:erase:`, synced after every stage) so a node that is killed resumes them at its next boot. - [Ankusa.Trace](Ankusa.Trace.md): W3C trace context for a hook, and the spans every stage opens (ground rule 8). - [Ankusa.ULID](Ankusa.ULID.md): The ULID text form of 128 bits: 26 characters of Crockford base32 (`0-9`, `A-Z` minus `I`, `L`, `O`, `U`), most significant bits first, so the first 48 bits (a Unix millisecond timestamp) make the string sort by time. - [Ankusa.Verification](Ankusa.Verification.md): Result of running a `Ankusa.Verifier` against an envelope. - [Ankusa.Verifier](Ankusa.Verifier.md): Signature and timestamp verification. HMAC is microseconds, so it runs inline on the hot path, before the ack. - [Ankusa.Verifier.Asymmetric](Ankusa.Verifier.Asymmetric.md): Public-key signature verifier: the provider signs with a private key only it holds, and Ankusa holds nothing secret, only the provider's *public* key. A leak of the node's config cannot be used to forge a hook, which is the point over `Ankusa.Verifier.Hmac`. - [Ankusa.Verifier.Asymmetric.Scheme](Ankusa.Verifier.Asymmetric.Scheme.md): Declarative description of one provider's public-key signature scheme: the wire fields of `Ankusa.Verifier.Hmac.Scheme` (`signature_header`, `parse`, `sig_prefix`, `sig_key`, `version`, `signed`, `timestamp`; see `Ankusa.Verifier.Template`) with `algorithm` and `encoding` in place of `hash`, `encoding` and `secret_decode`. - [Ankusa.Verifier.ClientCert](Ankusa.Verifier.ClientCert.md): The sender is whoever holds the client certificate the edge listener's mutual TLS handshake accepted (`tls.verify: :verify_peer`, `Ankusa.Listener`), for providers and internal systems that authenticate by certificate rather than by signing the body. Like `Ankusa.Verifier.Token` it says who is calling, not what the body is. - [Ankusa.Verifier.Hmac](Ankusa.Verifier.Hmac.md): Configurable HMAC signature verifier, driven by a `%Ankusa.Verifier.Hmac.Scheme{}` descriptor instead of one module per provider. - [Ankusa.Verifier.Hmac.Scheme](Ankusa.Verifier.Hmac.Scheme.md): Declarative description of one provider's HMAC signature scheme. - [Ankusa.Verifier.None](Ankusa.Verifier.None.md): No-op verifier: accepts every envelope. Use for sources whose authenticity is guaranteed by transport (mTLS, private network) or that are intentionally open. - [Ankusa.Verifier.Schemes](Ankusa.Verifier.Schemes.md): Named presets for `Ankusa.Verifier.Hmac` and `Ankusa.Verifier.Asymmetric` — the wire details of well-known providers as data, so adding a provider is editing a table, not writing a module. - [Ankusa.Verifier.Template](Ankusa.Verifier.Template.md): The wire half of a signature scheme, shared by every verifier that reads a signature out of a request header: `Ankusa.Verifier.Hmac` and `Ankusa.Verifier.Asymmetric`. - [Ankusa.Verifier.Token](Ankusa.Verifier.Token.md): A static credential the provider sends with every request: a bearer token, a basic-auth login, a token in a header of its choosing, or a token in the query string. For providers that cannot sign (many internal systems, and several SaaS "webhook secret" fields that are just a header value). It proves the sender knows the secret, nothing about the body: a captured request can be replayed and a body can be swapped. Prefer `Ankusa.Verifier.Hmac` or `Ankusa.Verifier.Asymmetric` whenever the provider can sign. - Internals (no stability guarantee) - [Ankusa.ClaimCheck.Router](Ankusa.ClaimCheck.Router.md): The `:claim_check` role's HTTP API: read-only byte transport for claim-check refs. Deliberately its own listener (`claim_check.port`, default `4001`), separate from `Ankusa.Edge.Router`: the edge is internet-facing, this is internal-network-only. - [Ankusa.ClaimCheck.Sweeper](Ankusa.ClaimCheck.Sweeper.md): Retention for `LocalFS`-backed claims, and the upkeep of the claim leases (rows `?l` of the node's store) that decide what retention may delete. Runs on a node with the `:dispatch` role (the role that writes leases), and on a `:storage` node when `claim_check.retention_days` is set; `Ankusa.ClaimCheck.validate_config!/1` already rejects that setting against any other blob store at boot, so the retention pass only ever runs against `BlobStore.LocalFS`. - [Ankusa.Dispatch.Pipeline](Ankusa.Dispatch.Pipeline.md): Async dispatch pipeline: a scheduler over *delivery rows*. Every committed hook has one row per sink of its source (`Ankusa.Queue`); this process claims the rows that are due, runs each delivery in a `Task.Supervisor` task, and records the outcome as a single store batch. At-least-once: a row is only deleted after its sink answered `:ok`, and a restart makes every claimed row due again. - [Ankusa.Edge.Batcher](Ankusa.Edge.Batcher.md): Group-commit batcher, one process per partition. Requests hand it an envelope and **block on the reply**. The batcher buffers callers, then flushes the whole batch to the queue in one `enqueue`, which the queue writer merges with the batches of the other partitions into one group commit (one fsync). Every blocked caller is replied to *after* the commit returns — that is what makes the ack honest. - [Ankusa.Edge.BatcherSupervisor](Ankusa.Edge.BatcherSupervisor.md): A fixed pool of `Ankusa.Edge.Batcher` processes, one per partition (four by default). A partition holds its callers' bodies and prepares their commit (`Ankusa.Queue.enqueue/3`: the encoded rows and the dedupe read) in its own task, in parallel with the others; `Ankusa.Queue.Writer` merges whatever every partition has queued into one group commit, so more partitions are not more fsyncs. The pool also buys fault isolation: one wedged partition leaves the others committing. - [Ankusa.Edge.Ingest](Ankusa.Edge.Ingest.md): Orchestrates one webhook on the hot path: build the envelope, verify inline, apply the per-source failure policy, then ack once the hook is durable. - [Ankusa.Edge.Quarantine](Ankusa.Edge.Quarantine.md): A durable holding pen for envelopes that failed verification under a `:quarantine` policy. A bad secret rotation should never silently eat real events, and the pen is how they come back: a `kind: :quarantine` replay job (`Ankusa.Replay`) re-verifies each held hook against its source's *current* verifier and commits the ones that now pass; `purge/3` drops the rest. - [Ankusa.Edge.Router](Ankusa.Edge.Router.md): The Bandit edge. Minimal work on the hot path, in the order that keeps an unauthenticated caller from costing anything: resolve the catch URL (`Ankusa.RouteResolver`, default `POST /webhooks/:source_id`) to a `Ankusa.Route`, refuse on a `Content-Length` over the limit, look the source up (`Ankusa.Edge.Ingest.lookup/2`), refuse a `Content-Length` over the tenant's own `max_body_bytes` (`413`), charge the pre-read limits (per client address, per node, per source bytes: `429`, `Ankusa.Edge.RateLimiter.pre_read/4`), refuse header bytes no sink can carry (`400 invalid_header`), only then read the body (bounded in size by the tenant's cap, and in time by `listener.request_deadline_ms`: `408`), and hand off to `Ankusa.Edge.Ingest`. A body with no declared length is charged to its source once it has been read. A request refused at any step before the read is answered without its body being read. No JSON parsing here. - [Ankusa.Http](Ankusa.Http.md): Small `Plug.Conn` helpers shared by every Bandit-fronted router in the framework (`Ankusa.Edge.Router`, `Ankusa.ClaimCheck.Router`). Kept here instead of duplicated so both surfaces enforce request bodies identically. - [Ankusa.HttpClient](Ankusa.HttpClient.md): The one place the outbound adapters issue HTTP requests: the object stores (`Ankusa.BlobStore.S3`, `GCS`, `Azure`, `OCI`), `Ankusa.Sink.Http`, `Ankusa.Sink.SQS`, and `Ankusa.Sink.GooglePubSub`. - [Ankusa.Instance.Isolated](Ankusa.Instance.Isolated.md): Owns one optional subtree of an instance (dispatch, storage, lifecycle, metrics, or one of the secondary listeners) and never escalates its failure. - [Ankusa.Instance.RegistryWatch](Ankusa.Instance.RegistryWatch.md): Stops an instance when `Ankusa.Registry` restarts, so its supervisor starts it again. - [Ankusa.Queue.Writer](Ankusa.Queue.Writer.md): The only seq assigner: commits batches of hooks to the store, and answers a caller only once its hooks are on disk (`Ankusa.Queue.Syncer` fsyncs the WAL), so a `2xx` to the provider always means the bytes are on disk. - [Ankusa.Storage.Compactor](Ankusa.Storage.Compactor.md): Packs committed hooks into immutable segments and clears their archive obligations. - [Ankusa.Store](Ankusa.Store.md): The node's local store: one RocksDB database per instance, under `Config.path(config, "store")`. - [Ankusa.UUIDv7](Ankusa.UUIDv7.md): UUIDv7 generation (RFC 9562 §5.7): 48-bit Unix millisecond timestamp followed by 74 random bits. ## Mix Tasks - [mix ankusa.conformance.gen](Mix.Tasks.Ankusa.Conformance.Gen.md): Writes `conformance/cases/producer.json`: the queue messages `Ankusa.Sink.Message.encode/3` produces for a fixed set of hooks, each paired with what an SDK's `decode_message` must return for it (feature `message.producer`). - [mix ankusa.recover](Mix.Tasks.Ankusa.Recover.md): Brings a stopped node's store forward from the intake log (`wal: :dual`, ADR 0002), by the same code a booting node runs. - [mix ankusa.secrets.rewrap](Mix.Tasks.Ankusa.Secrets.Rewrap.md): Moves every API-managed source a node stored in its own `Ankusa.Store` (`Ankusa.SourceStore.Persistent`) to the current key-encryption key: rows written before a KEK existed are encrypted, and rows encrypted under a `kek_previous` key are rewritten under the current one. Afterwards the old key can be dropped from `secrets.kek_previous`. - [mix ankusa.store.fixture](Mix.Tasks.Ankusa.Store.Fixture.md): Seeds an instance and packs its data directory as an upgrade fixture. - [mix ankusa.store.migrate](Mix.Tasks.Ankusa.Store.Migrate.md): Shows, or applies, the store format migrations a data directory needs before this release can serve from it.