The segment object: what the compactor PUTs and a replay reads whole.
Format 1
"ANKSEG" <> <<version::8, codec_id::8, count::32>> (12 bytes)
<codec frames, back to back>
sha256(everything above) (32 bytes)codec_id names the Ankusa.Codec that framed the records (1 is
Ankusa.Codec.Raw; 0 is a codec with no id, which the catalogue row names by
module). count is the number of frames. The index (Ankusa.Storage.Index)
holds each record's absolute offset and length within the object, header
included, so a range read of [offset, offset + length) is exactly one frame
and the codec's decode_record/1 is unchanged.
A reader that fetches one frame (Storage.fetch/2) is guarded by the frame's
own CRC; a reader that has the whole object (verify/1, used by replay and
reconcile) also checks the sha256, so a damaged or truncated object is
{:error, _}, never a silently short read.
Format 0
The segments written before the header existed: bare frames, no trailer. They
stay readable (verify/2 passes them through); the catalogue row's format: 0
says so. A reader tells the two apart by the "ANKSEG" magic, and a format
newer than this release is {:error, :unsupported_format}.
Summary
Functions
The codec module a recorded codec_id stands for, nil when it names none.
The codec id a segment records for codec: its id/0 when it has one, else
0.
Pack records with codec: {object, index, sha256_hex}. index is the
codec's, its offsets moved past the header so they are positions in object.
Bytes before the first frame (format 1).
Read a segment's header from the first bytes of the object (at least
header_bytes/0 of them): {:ok, header} for format 1, {:ok, :legacy} for
an object that has no magic.
The codec to read a catalogue row's segment with: the module the row recorded
when it is loaded here, else default (the configured codec).
Bytes after the last frame (format 1).
Check a whole object against the format the catalogue row recorded (0 for a
legacy segment, which has nothing to check, 1): {:ok, header | :legacy} or
{:error, :bad_magic | :unsupported_format | :truncated | :sha_mismatch}.
The current segment format.
Types
@type format() :: 0 | 1
@type header() :: %{format: 1, codec_id: 0..255, count: non_neg_integer()}
Functions
@spec codec_for_id(0..255) :: module() | nil
The codec module a recorded codec_id stands for, nil when it names none.
@spec codec_id(module()) :: 0..255
The codec id a segment records for codec: its id/0 when it has one, else
0.
@spec encode(module(), [Ankusa.Codec.segment_record()]) :: {binary(), [Ankusa.Codec.index_entry()], String.t()}
Pack records with codec: {object, index, sha256_hex}. index is the
codec's, its offsets moved past the header so they are positions in object.
@spec header_bytes() :: pos_integer()
Bytes before the first frame (format 1).
@spec parse_header(binary()) :: {:ok, header() | :legacy} | {:error, :unsupported_format | :truncated}
Read a segment's header from the first bytes of the object (at least
header_bytes/0 of them): {:ok, header} for format 1, {:ok, :legacy} for
an object that has no magic.
The codec to read a catalogue row's segment with: the module the row recorded
when it is loaded here, else default (the configured codec).
@spec trailer_bytes() :: pos_integer()
Bytes after the last frame (format 1).
Check a whole object against the format the catalogue row recorded (0 for a
legacy segment, which has nothing to check, 1): {:ok, header | :legacy} or
{:error, :bad_magic | :unsupported_format | :truncated | :sha_mismatch}.
@spec version() :: pos_integer()
The current segment format.