Ankusa.BlobStore.Options (ankusa v0.7.0)

Copy Markdown View Source

The transport options every HTTP object store takes (Ankusa.BlobStore.S3, GCS, Azure, OCI), their defaults and their ranges. They are adapter options, like :timeout_ms and :retries: Elixir config, and the s3/gcs sections of the YAML.

OptionDefaultWhat
:timeout_ms10_000One metadata-sized request (GET, DELETE, LIST, a body up to 1 MiB): connect and response
:upload_timeout_ms60_000One request that carries a large body: a bigger single PUT, or one multipart part
:op_deadline_ms120_000One absolute deadline per blob operation, covering every attempt, every multipart part and (for Ankusa.ClaimCheck.check_in/4) every pack-id retry together. Nothing sleeps or sends past it
:multipart_threshold_bytes8_388_608A put of more bytes goes up as a multipart (S3, OCI), block-list (Azure) or resumable (GCS) upload
:multipart_part_bytes8_388_608The size of each part or chunk (a multiple of 256 KiB, at least 5 MiB)
:retries2Extra attempts after a transient failure (Ankusa.BlobStore.Retry)
:breaker_failures5Consecutive failures of a {endpoint, bucket} that open its breaker; 0 turns the breaker off, the latency trip with it
:breaker_open_ms30_000How long an open breaker answers {:error, :breaker_open} before it lets one request through
:breaker_slow_ms5_000The breaker also opens when the p99 of the last 100 metadata-sized requests passes this
:pool_size, :pool_count32, 1Connections per pool and pools, of the Finch pool the instance starts for this store's endpoint (Ankusa.BlobStore.Pool)
:pool_timeout5_000How long a request waits for a free connection of that pool

Every bound is checked when the config is validated (validate!/3), so a bad value is a boot error and never a request that waits an hour.

Summary

Functions

The option's default.

opts[key], else the default.

:small for a request that carries at most 1 MiB, :bulk for one that carries more: which timeout it runs under, and whether its latency is a breaker sample.

Milliseconds left to opts[:deadline] (negative when passed).

Raise ArgumentError naming where unless every transport option opts sets is an integer inside its range, and the part size is a multiple of 256 KiB. Stores that are not an HTTP object store are not checked.

opts with :deadline (an absolute System.monotonic_time(:millisecond)) set to op_deadline_ms from now, or to the deadline the caller already passed when that is sooner. Every adapter entry point calls it once, so every request the operation makes shares one deadline.

Functions

default(key)

@spec default(atom()) :: pos_integer() | non_neg_integer()

The option's default.

get(opts, key)

@spec get(keyword(), atom()) :: non_neg_integer()

opts[key], else the default.

kind(size)

@spec kind(non_neg_integer()) :: :small | :bulk

:small for a request that carries at most 1 MiB, :bulk for one that carries more: which timeout it runs under, and whether its latency is a breaker sample.

remaining(opts)

@spec remaining(keyword()) :: integer()

Milliseconds left to opts[:deadline] (negative when passed).

validate!(module, opts, where)

@spec validate!(module(), keyword(), String.t()) :: :ok

Raise ArgumentError naming where unless every transport option opts sets is an integer inside its range, and the part size is a multiple of 256 KiB. Stores that are not an HTTP object store are not checked.

with_deadline(opts)

@spec with_deadline(keyword()) :: keyword()

opts with :deadline (an absolute System.monotonic_time(:millisecond)) set to op_deadline_ms from now, or to the deadline the caller already passed when that is sooner. Every adapter entry point calls it once, so every request the operation makes shares one deadline.