Ankusa.Storage (ankusa v0.7.0)

Copy Markdown View Source

Read side of long-term segment storage. fetch/2 resolves an event id to its segment, range-reads exactly its frame from the blob store, and unframes it back into the original Ankusa.Envelope.

A segment's catalogue row lives in the node's Ankusa.Store, with the range of event ids it holds; its index object (….idx beside the segment in the blob store, Ankusa.Storage.Index) maps each id to {offset, length, seq}. Event ids are time-ordered, so the id range narrows a lookup to a segment or two. Events archived by a 0.3 node resolve through the locations imported from its index log.

When the catalogue has no segment for an id (a fresh node, or a segment another node wrote) and the id carries a time (a UUIDv7), the dated layout (Ankusa.Storage.Layout) says where else to look: the id's day, and the day before, are listed in the object store, every node's _manifest.json that can hold the id is kept, and each such node's segments are searched by a bisection over their index objects. That is a handful of object reads per fetch, not a catalogue lookup: Queue.Archive.reconcile/2 puts the segments in the catalogue.

Nothing read from the object store is trusted: an index that does not decode (Ankusa.Storage.Index) or a frame whose CRC fails is :error.

A hook of a tenant that was erased (Ankusa.Tenant) after the hook was received is {:error, :gone} (410 where it is served): the segment still holds its frame until the bucket's lifecycle expires it, and the tombstone is what keeps it from being read in the meantime.

Summary

Functions

The index object of a segment (%{event_id => {offset, length, seq}}), as the catalogue row row names it: one object read, no frame. :error when the object is missing, unreadable, or does not decode (Ankusa.Storage.Index).

Functions

fetch(instance, event_id)

@spec fetch(atom(), String.t()) ::
  {:ok, Ankusa.Envelope.t()} | :error | {:error, :gone}

segment_index(instance, map)

@spec segment_index(atom(), map()) :: {:ok, Ankusa.Storage.Index.t()} | :error

The index object of a segment (%{event_id => {offset, length, seq}}), as the catalogue row row names it: one object read, no frame. :error when the object is missing, unreadable, or does not decode (Ankusa.Storage.Index).