# `Ankusa.Audit`
[🔗](https://github.com/jamescarr/ankusa/blob/ankusa-v0.7.0/packages/ankusa/lib/ankusa/audit.ex#L1)

The audit trail: one row for every change an operator (or code) makes to the
node's configuration or operational state, written with the change.

## What is recorded

A row says who (`actor`, `remote_ip`, `request_id`, from `Ankusa.Audit.Context`),
what (`action`, `tenant`, `resource`, `detail`), and what it did to the
subject: `before_hash` and `after_hash`, an HMAC-SHA256 of the subject before
and after, `revision` (the subject's own change counter, where it has one) and
`result`. **A row never holds a secret or a spec**: a hash is keyed with a
random per-store key (`m:audit_key`), so it shows that a source's secret
changed without giving anyone with `audit:read` a way to test a guess
against it, and `detail` carries only what the caller puts there (counts,
filters, a rate).

| action | change |
| --- | --- |
| `source.create`, `source.update`, `source.delete` | an API-managed source |
| `rate_limit.put`, `rate_limit.delete` | a tenant's rate-limit override |
| `quarantine.purge` | held hooks deleted |
| `replay.create`, `replay.update`, `replay.cancel` | a replay job |
| `route.create`, `route.update`, `route.delete` | a route definition |
| `tenant.delete` | a tenant erase (`Ankusa.Tenant`): one row, with what each stage removed as its `detail`; the erase's own source, route and rate-limit deletions are rows of their own |
| `ip_rules.put` | the global IP rules |
| `log_level.put` | the node's log level (`PUT /v1/log-level`): `detail` has the new `level` and the `previous` one |
| `config.reload` | a config reload (`POST /v1/config/reload`, `SIGHUP`; `Ankusa.Reload`): `detail.keys` lists the dotted keys that changed, `result` is `ok` or `failed`; a refused reload writes none |

A request that changes nothing (a validation error, a missing subject) writes
no row; one that is refused by the store writes none either.

## Order and durability

Rows live in the store's `audit` column family under
`<<?a, seq::64, at::64>>`. This process hands out `seq` **and commits the
batch**, so key order is commit order whatever the wall clock does, and a
reader that remembers the last `seq` it saw (SEC-2.b's lifecycle outbox) can
never miss a row that lands later with a smaller key.

There are two ways in:

  * `commit/4` is for a change that lives in this node's `Ankusa.Store`
    (API-managed sources, rate-limit overrides, quarantine, replay jobs): the
    change's own ops and the audit row go in **one** `sync: true` batch.
    Either both survive a `kill -9` or neither does.

  * `begin/3` + `finish/3` (or `record/4`) is for a change that lives
    somewhere else (route definitions, whose store is pluggable: ETS, or
    Redis shared by a fleet; the Redis source store). That is a dual write,
    and there is no transaction across the two. The row is written first, as
    a *pending* row (`<<?p, seq::64>>`, synced) that says what is about to
    happen; the change is applied; `finish/3` then moves it into the trail
    with the outcome, in one batch. A crash in between leaves the pending row,
    and the next boot (or the caller dying, while this node runs) settles it
    as `result: "interrupted"`: *the change may or may not have been
    applied*; `after_hash` is the hash of what was about to be written. For a
    source or route, `Ankusa.Lifecycle.reconcile/2` does better than that: it
    reads the subject's present state and compares its hash to the row's
    `after_hash` (the change was applied: `"ok"`) and `before_hash` (it was
    not: `"failed"`); a state that is neither (someone else changed it
    meanwhile) stays `"interrupted"` and is flagged for a state-refresh
    event. A boot has to wait for the subject's store to load, so such rows
    are settled by a tick (every second, for a minute, then as
    `"interrupted"`). The trail never loses a change that was applied; it can
    show one that was not only as `"interrupted"`.

A change whose audit row cannot be written is not made: the caller gets
`{:error, :store_unavailable}`.

## The lifecycle outbox

When lifecycle events are on (`Ankusa.Lifecycle`), the row of a source or
route change that took effect is flagged `publish: true`, and the redacted
entity its event carries is written beside it at `<<?o, seq::64>>` in the
same batch. The entity is not in the row: a row never holds a spec, and even
redacted an entity is one. `Ankusa.Lifecycle.Publisher` reads the flagged
rows in `seq` order with `page/3` and keeps its place in `m:lifecycle_cursor`.

## Restores and event identity

A store that came back from a backup (`Ankusa.Store` leaves
`m:incarnation_pending`) re-issues the `seq` numbers of every change the lost
one made after the backup, and some of those were already published. So the
first thing this process does on such a boot is open a new *incarnation*:
one history entry `{from seq, fresh id}` in `m:incarnation`, written with an
`instance.restore` row (event `io.ankusa.instance.restored`, carrying the
incarnation and the restored `seq`) in one synced batch, before any change is
taken. `incarnation/2` tells `Ankusa.Lifecycle` which id a row's event id is
keyed with: rows of the restored trail keep the ids they were published
under, every row from the new incarnation on gets a different one.

## Retention

`audit.retention_days` (default 365). An hourly sweep deletes rows (and the
lifecycle entities beside them) older than that, oldest first, in bounded
batches, and emits `[:ankusa, :audit, :pruned]`. While `lifecycle.sinks` is
set it never deletes a row at or above `m:lifecycle_cursor`, however old: a
lifecycle sink that is down longer than the retention keeps the events it has
not seen. The expired rows it holds back are emitted as `blocked_by_cursor` on
`[:ankusa, :audit, :retention]` after every sweep. Every committed row emits
`[:ankusa, :audit]`.

## Ceilings

The trail is bounded in `audit.max_rows` and `audit.max_bytes`, from counters
kept in every batch (`m:audit_usage`; recounted at boot when it does not match
the trail). Before a change is accepted the trail is pruned to them, oldest
first and never past `m:lifecycle_cursor`; a trail that is still over a ceiling,
or a lifecycle outbox holding `audit.outbox_max_backlog` or more events no sink
has confirmed (`m:audit_seq` minus the cursor), refuses the *new* change with
`{:error, :audit_backlog}` (`503 audit_backlog`, `Retry-After: 60` on the admin
APIs). An unpublished row is never dropped to make room, and a change already
begun is always recorded. A change the node makes itself (`Context.system/0`) is
not refused.

# `attrs`

```elixir
@type attrs() :: %{
  :action =&gt; String.t(),
  optional(:tenant) =&gt; String.t() | nil,
  optional(:resource) =&gt; String.t() | nil,
  optional(:before_hash) =&gt; String.t() | nil,
  optional(:after_hash) =&gt; String.t() | nil,
  optional(:revision) =&gt; pos_integer() | nil,
  optional(:detail) =&gt; map() | nil,
  optional(:result) =&gt; String.t(),
  optional(:entity) =&gt; map() | nil
}
```

What a caller says about its change. `action` is required.

# `row`

```elixir
@type row() :: %{required(atom()) =&gt; term()}
```

# `begin`

```elixir
@spec begin(atom(), Ankusa.Audit.Context.t(), attrs()) ::
  {:ok, pos_integer()} | {:error, :store_unavailable | :audit_backlog}
```

Write the pending row of a change that is about to be made elsewhere. Returns
the handle `finish/3` takes. The caller is monitored: if it dies first, the
row is finished as `"interrupted"`.

# `child_spec`

```elixir
@spec child_spec(keyword()) :: Supervisor.child_spec()
```

Returns a specification to start this module under a supervisor.

See `Supervisor`.

# `commit`

```elixir
@spec commit(atom(), Ankusa.Audit.Context.t(), attrs(), [Ankusa.Store.op()]) ::
  :ok | {:error, :store_unavailable | :audit_backlog}
```

Commit `ops` (the change, for the node's own `Ankusa.Store`) and its audit row
in one synced batch.

# `digest`

```elixir
@spec digest(atom(), term()) :: String.t() | nil
```

The keyed hash of `term` an audit row carries (`nil` for `nil`). Computed by
the caller from the key in an ETS table, so a secret never travels to this
process in a message that a crash report could print.

# `discard`

```elixir
@spec discard(atom(), pos_integer()) :: :ok | {:error, term()}
```

Drop a pending row whose change turned out to be no change at all (the store
rejected the request): no row reaches the trail.

# `entity`

```elixir
@spec entity(atom(), pos_integer()) :: {:ok, map()} | :not_found | {:error, term()}
```

The redacted entity of the row `seq` (a row flagged `publish: true`), or
`:not_found` when it was pruned.

# `finish`

```elixir
@spec finish(atom(), pos_integer(), map()) :: :ok | {:error, term()}
```

Move a pending row into the trail. `update` may set `:result` (`"ok"`,
`"failed"`), `:after_hash`, `:before_hash`, `:revision`, `:detail`, `:tenant`
and `:resource`; the row keeps what `begin/3` recorded for anything left out.

# `incarnation`

```elixir
@spec incarnation(atom(), pos_integer()) :: String.t() | nil
```

The incarnation the audit row `seq` was written in: the id of the newest
restore at or before it, `nil` for a store that was never restored from a
backup (and for the rows before its first restore).

# `last_seq`

```elixir
@spec last_seq(atom()) :: {:ok, non_neg_integer()} | {:error, term()}
```

The last `seq` the trail handed out (`0` for an empty trail).

# `list`

```elixir
@spec list(atom(), keyword()) ::
  {:ok, %{entries: [row()], next_cursor: String.t() | nil}} | {:error, term()}
```

The trail, newest first.

Options: `:since` / `:until` (inclusive bounds on `at`, ms), `:actor`,
`:action`, `:tenant` (exact matches), `:tenants` (`:all`, or the tenants the
reader may see: rows with no tenant need `:all`), `:cursor` (the
`next_cursor` of the previous page) and `:limit` (default 100, at most 1000).
`next_cursor` is `nil` on the last page; a page can be short when
the scan budget (10 000 rows) ran out, and `next_cursor` says to go on.

# `page`

```elixir
@spec page(atom(), non_neg_integer(), pos_integer()) ::
  {:ok, [{pos_integer(), {:ok, row()} | {:error, term()}}]} | {:error, term()}
```

Rows after `after_seq`, oldest first: at most `max_rows` of them, as
`{seq, {:ok, row} | {:error, reason}}` (a row that cannot be decoded is
reported, not skipped, so the reader decides). The lifecycle outbox
(`Ankusa.Lifecycle.Publisher`) and its replay read the trail this way: `seq`
is commit order, so a reader that keeps the last one it handled cannot miss
a row.

# `prune`

```elixir
@spec prune(atom()) :: {:ok, non_neg_integer()} | {:error, term()}
```

Delete the rows older than `audit.retention_days`, oldest first. Runs on the
hourly tick; callable on its own. `{:ok, deleted}`.

# `record`

```elixir
@spec record(atom(), Ankusa.Audit.Context.t(), attrs(), (-&gt; {reply, map() | :discard})) ::
  reply | {:error, :store_unavailable | :audit_backlog}
when reply: term()
```

Run `fun`, the change itself, between a pending row and its finish. `fun`
returns `{reply, update}` (see `finish/3`), or `{reply, :discard}` when it
changed nothing (see `discard/2`); `record/4` returns `reply`, or
`{:error, :store_unavailable}` without running `fun` when the pending row
could not be written. A `fun` that raises, throws or exits leaves the row
`"interrupted"`.

# `refusing?`

```elixir
@spec refusing?(atom()) :: boolean()
```

Whether a control-plane change is refused right now (`503 audit_backlog`,
`Retry-After: 60`): the trail holds more than `audit.max_rows` rows or
`audit.max_bytes` bytes (and pruning it to its ceilings, which the trail is then
asked to do, may not pass the lifecycle cursor), or the lifecycle outbox holds
`audit.outbox_max_backlog` events or more that no sink has confirmed. Read
from counters, never a scan; `false` where no trail runs.

# `start_link`

```elixir
@spec start_link(keyword()) :: GenServer.on_start()
```

# `stats`

```elixir
@spec stats(atom()) :: %{
  rows: non_neg_integer(),
  bytes: non_neg_integer(),
  outbox_backlog: non_neg_integer() | nil
}
```

The trail's size and the lifecycle outbox's lag, from counters: `rows`,
`bytes`, and `outbox_backlog`, the audit `seq`s the lifecycle publisher has not
yet confirmed (`nil` without lifecycle sinks, or before the publisher wrote
its cursor). All zero where no trail runs.

# `to_json`

```elixir
@spec to_json(row()) :: map()
```

The JSON view of a row, as `GET /v1/audit` returns it.

---

*Consult [api-reference.md](api-reference.md) for complete listing*