abczarr.metadata.v3.codecs
Classes
CodecConfig
Codec
Codec(name: str | bytes = '', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: Extension
A Codec names a codec through name and carries that codec's
own configuration.
Each Codec represents one stage of an array's codec pipeline. See
ArrayToArrayCodec,
ArrayToBytesCodec
and
BytesToBytesCodec
for what each stage does.
See
abczarr.metadata.v2.codecs.base.Codec
for a worked example comparing the same codec across the v1, v2 and
v3 metadata models.
Attributes
Methods:
to_json
to_json() -> JsonDict
Serialize this codec to its JSON representation.
A codec with no configuration parameters, such as crc32c
or a bytes codec for a single-byte dtype, is written as a
bare name, not with an empty configuration object.
Returns:
| Type | Description |
|---|---|
dict
|
The JSON-compatible representation of this codec. |
to_version
to_version(version: ZarrVersion) -> Codec
Convert this codec to another Zarr version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
version
|
ZarrVersion
|
The target Zarr format version: 1, 2 or 3. |
required |
Returns:
| Type | Description |
|---|---|
Codec
|
The equivalent codec for |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
ArrayToArrayCodec
ArrayToArrayCodec(name: str | bytes = '', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: Codec
Transforms an array into another array.
Runs before the pipeline's single array-to-bytes codec, for example by transposing axes, reshaping, or rounding values.
Methods:
ArrayToBytesCodec
ArrayToBytesCodec(name: str | bytes = '', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: Codec
Serializes an array to bytes, or parses bytes back into an array.
Every codec pipeline has exactly one array-to-bytes codec, forming the boundary between the array-to-array codecs before it and the bytes-to-bytes codecs after.
Methods:
BytesToBytesCodec
BytesToBytesCodec(name: str | bytes = '', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: Codec
Transforms a byte sequence into another byte sequence.
Runs after the pipeline's array-to-bytes codec. A typical example is a compressor, or a checksum appended for integrity.
Methods:
CompressorCodec
CompressorCodec(name: str | bytes = '', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: BytesToBytesCodec
This class is the base for a bytes-to-bytes codec that compresses its input.
Methods:
BloscConfig
BloscConfig(cname: Literal['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'] = 'lz4', clevel: Literal[0, 1, 2, 3, 4, 5, 6, 7, 8, 9] = 5, shuffle: Literal['noshuffle', 'shuffle', 'bitshuffle'] = 'shuffle', blocksize: Integral | integer | bool = 0, typesize: Integral | integer | bool | None = None)
Bases: CodecConfigImpl
Holds the Blosc codec's parameters: compressor, level, shuffle, and block size.
Attributes
cname
class-attribute
instance-attribute
The inner compressor Blosc applies.
clevel
class-attribute
instance-attribute
The compression level, from 0 to 9.
shuffle
class-attribute
instance-attribute
The byte-shuffle filter applied before compression:
"noshuffle", "shuffle" or "bitshuffle".
blocksize
class-attribute
instance-attribute
blocksize: int = 0
The block size Blosc compresses in, in bytes. 0 lets Blosc
choose automatically.
typesize
class-attribute
instance-attribute
typesize: int | None = None
The size, in bytes, of the array's element type. Blosc uses this to group same-position bytes together when shuffling.
Methods:
to_version
to_version(version: ZarrVersion) -> Metadata
Convert this configuration to another Zarr version's Blosc codec.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
version
|
ZarrVersion
|
The target Zarr format version: 1, 2 or 3. |
required |
Returns:
| Type | Description |
|---|---|
Metadata
|
The equivalent Blosc codec for |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
BloscCodec
BloscCodec(name: Literal['blosc'] = 'blosc', configuration: BloscConfig | JsonDict | Literal['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'] = BloscConfig(cname='lz4', clevel=5, shuffle='shuffle', blocksize=0, typesize=None), must_understand: bool | int | str = True)
Bases: CompressorCodec
Compresses data with Blosc, a meta-compressor that shuffles bytes and then applies an inner compressor.
Blosc groups same-typed bytes together before handing them to
cname, then compresses the result in blocks so multiple threads
can be used.
Attributes
Methods:
BytesConfig
BytesConfig(endian: Literal['big', 'little'] | None = None)
Bases: CodecConfigImpl
Holds the bytes codec's parameters: the byte order to serialize with.
Attributes
endian
instance-attribute
endian: Literal['big', 'little'] | None
The byte order to serialize with, "big" or "little", or
None for a dtype whose byte order does not matter, such as a
single-byte type.
Methods:
BytesCodec
BytesCodec(name: Literal['bytes'] = 'bytes', configuration: BytesConfig | JsonDict | Literal['big', 'little'] | None = BytesConfig(endian=None), must_understand: bool | int | str = True)
Bases: ArrayToBytesCodec
Serializes an array to its raw bytes, in configuration.endian
byte order.
This is the plain array-to-bytes codec. It applies no compression and no transform, laying the array's values out contiguously.
Attributes
Methods:
CRC32CConfig
Bases: CodecConfigImpl
This configuration is empty, since the CRC-32C codec takes no parameters.
Methods:
CRC32CCodec
CRC32CCodec(name: Literal['crc32c'] = 'crc32c', configuration: CRC32CConfig | JsonDict = CRC32CConfig(), must_understand: bool | int | str = True)
Bases: BytesToBytesCodec
Appends a CRC-32C checksum of the preceding bytes, for integrity.
The checksum is verified and stripped on decode, which catches corrupted or truncated chunk data. This codec neither compresses nor transforms the chunk data itself.
Attributes
Methods:
GzipConfig
GzipConfig(level: Literal[0, 1, 2, 3, 4, 5, 6, 7, 8, 9] = 5)
GzipCodec
GzipCodec(name: Literal['gzip'] = 'gzip', configuration: GzipConfig | JsonDict | Literal[0, 1, 2, 3, 4, 5, 6, 7, 8, 9] = GzipConfig(level=5), must_understand: bool | int | str = True)
Bases: CompressorCodec
Applies DEFLATE compression (gzip) at a configurable compression level.
Attributes
Methods:
ShardingConfig
ShardingConfig(chunk_shape: Iterable[Integral | integer | bool] = (), codecs: Iterable[Codec | JsonDict | str] = (), index_codecs: Iterable[Codec | JsonDict | str] = (), index_location: Literal['start', 'end'] = 'end')
Bases: CodecConfigImpl
Holds the sharding codec's parameters: the inner chunking and its two codec chains.
Attributes
chunk_shape
instance-attribute
chunk_shape: Shape
The shape of the sub-chunks packed into each shard.
codecs
instance-attribute
codecs: tuple[Codec, ...]
The codec pipeline that encodes each sub-chunk's data.
index_codecs
instance-attribute
index_codecs: tuple[Codec, ...]
The codec pipeline that encodes the index locating each sub-chunk within the shard.
index_location
class-attribute
instance-attribute
index_location: Literal['start', 'end'] = 'end'
Where the index is stored within the shard: "start" or
"end".
Methods:
ShardingCodec
ShardingCodec(name: Literal['sharding_indexed'] = 'sharding_indexed', configuration: ShardingConfig | JsonDict | Shape = ShardingConfig(chunk_shape=(), codecs=(), index_codecs=(), index_location='end'), must_understand: bool | int | str = True)
Bases: ArrayToArrayCodec
Packs many sub-chunks into one storage object, indexed for lookup.
Splits a chunk into smaller sub-chunks, encodes each with its own codec chain, and stores the sub-chunks together in a single shard alongside an index that maps each sub-chunk to its offset and length. Sharding this way reduces the number of files or objects a store holds for arrays with many small chunks.
Attributes
Methods:
TransposeConfig
TransposeConfig(order: Iterable[Integral | integer | bool] = ())
Bases: CodecConfigImpl
Holds the transpose codec's parameters: the permutation of axes to apply.
Attributes
Methods:
TransposeCodec
TransposeCodec(name: Literal['transpose'] = 'transpose', configuration: TransposeConfig | JsonDict | tuple[int, ...] = TransposeConfig(order=()), must_understand: bool | int | str = True)
Bases: ArrayToArrayCodec
Permutes an array's axes into configuration.order before the
rest of the pipeline.
The permutation is reversed on decode, so the array's logical shape is unchanged. It controls the memory layout the later codecs see, for example to make an axis contiguous for a compressor.
Attributes
Methods:
BitroundConfig
BitroundConfig(keepbits: Integral | integer | bool = 1)
Bases: CodecConfigImpl
Holds the bitround codec's parameters: how many mantissa bits to keep.
Attributes
keepbits
class-attribute
instance-attribute
keepbits: int = 1
The number of mantissa bits kept. The rest are zeroed.
Methods:
BitroundCodec
BitroundCodec(name: Literal['bitround', 'numcodecs.bitround'] = 'bitround', configuration: BitroundConfig | JsonDict | int = BitroundConfig(keepbits=1), must_understand: bool | int | str = True)
Bases: ArrayToArrayCodec
Rounds a float's mantissa to configuration.keepbits bits,
zeroing the rest.
The rounding is lossy and improves the compressibility of floating-point data by discarding low-order precision the data does not need.
Attributes
name
instance-attribute
name: Literal['bitround', 'numcodecs.bitround']
Either "bitround" or "numcodecs.bitround".
Methods:
ScalarMap
ScalarMap(encode: Iterable[tuple[Real | floating | bool | bool | int | str | bytes | None, Real | floating | bool | bool | int | str | bytes | None]] = (), decode: Iterable[tuple[Real | floating | bool | bool | int | str | bytes | None, Real | floating | bool | bool | int | str | bytes | None]] = ())
Bases: Metadata
This class represents an explicit value-to-value mapping used by a cast-value codec.
encode and decode each list (from, to) pairs that map
individual values falling outside the ordinary numeric cast. An
example is a sentinel value or a small set of categorical codes.
CastValueConfig
CastValueConfig(data_type: Any, rounding: Literal['nearest-even', 'towards-zero', 'towards-positive', 'towards-negative', 'nearest-away'] = 'nearest-even', out_of_range: Literal['clamp', 'wrap'] | None = None, scalar_map: ScalarMap | JsonDict | tuple[tuple[JsonScalar, JsonScalar], ...] = ScalarMap(encode=(), decode=()))
Bases: CodecConfigImpl
Holds the cast-value codec's parameters: target type, rounding, and value mapping.
Attributes
rounding
class-attribute
instance-attribute
rounding: Literal['nearest-even', 'towards-zero', 'towards-positive', 'towards-negative', 'nearest-away'] = 'nearest-even'
How a value that does not fit data_type exactly is rounded:
one of "nearest-even", "towards-zero",
"towards-positive", "towards-negative" or
"nearest-away".
out_of_range
class-attribute
instance-attribute
out_of_range: Literal['clamp', 'wrap'] | None = None
How a value outside the range of data_type is handled:
"clamp", "wrap", or None for unspecified behavior.
scalar_map
instance-attribute
scalar_map: ScalarMap
An explicit mapping for values that need one instead of the ordinary numeric cast.
Methods:
CastValueCodec
CastValueCodec(name: Literal['cast_value'] = 'cast_value', configuration: CastValueConfig | JsonDict | dtype, must_understand: bool | int | str = True)
Bases: ArrayToArrayCodec
Casts an array's values to configuration.data_type, and back
on decode.
configuration.rounding and configuration.out_of_range control
how a value that does not fit the target type exactly is rounded
and clamped or wrapped. configuration.scalar_map maps any
values that need an explicit, non-numeric mapping instead.
Attributes
Methods:
ConditionalConfig
Bases: CodecConfigImpl
Holds the conditional codec's parameters: the candidate codecs to choose among.
Attributes
codecs
instance-attribute
codecs: tuple[Codec, ...]
The candidate codecs the conditional codec chooses among.
Methods:
ConditionalCodec
ConditionalCodec(name: Literal['conditional'] = 'conditional', configuration: ConditionalConfig | JsonDict | tuple[Codec, ...] = ConditionalConfig(codecs=()), must_understand: bool | int | str = True)
Bases: Codec
Selects one of several candidate codecs, based on the array to which it is applied.
Lets a single pipeline entry vary by a property of the array, such as its data type, instead of naming one codec unconditionally.
Attributes
Methods:
N5DefaultCodecList
Bases: list
The fixed codec chain of an n5_default codec.
A transpose codec, then a big-endian bytes codec, then any number of trailing codecs. Each element is converted to its concrete codec type on construction. The list holds codec objects, not the raw dicts it was built from.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
codecs
|
iterable
|
The codecs to store, as codec objects or as JSON-like mappings. The first is read as a transpose codec, the second as a big-endian bytes codec, and the rest as arbitrary codecs. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If fewer than two codecs are given, or the second codec is little-endian. |
Methods:
N5DefaultConfig
Bases: CodecConfigImpl
Holds the n5_default codec's parameters: the fixed chain of codecs it applies.
Attributes
codecs
instance-attribute
codecs: N5DefaultCodecList
The codec chain: a transpose codec, a big-endian bytes codec, then any number of trailing codecs.
Methods:
N5DefaultCodec
N5DefaultCodec(name: Literal['n5_default'] = 'n5_default', configuration: N5DefaultConfig | JsonDict | N5DefaultCodecList, must_understand: bool | int | str = True)
Bases: Codec
Applies the codec chain an N5 array uses by default, for interop with N5.
The chain is fixed to a transpose codec followed by a big-endian bytes codec, with any number of trailing codecs, matching how the N5 format lays out a chunk.
Attributes
Methods:
PackBitsConfig
PackBitsConfig(padding_encoding: Literal['first_byte', 'last_byte', 'none'] = 'none', first_bit: Integral | integer | bool | None = None, last_bit: Integral | integer | bool | None = None)
Bases: CodecConfigImpl
Holds the packbits codec's parameters: padding placement and the used bit range.
Attributes
padding_encoding
class-attribute
instance-attribute
padding_encoding: Literal['first_byte', 'last_byte', 'none'] = 'none'
Where the padding bits sit within the packed bytes:
"first_byte", "last_byte" or "none".
first_bit
instance-attribute
first_bit: int | None
The index of the first meaningful bit, when padding is present.
last_bit
instance-attribute
last_bit: int | None
The index of the last meaningful bit, when padding is present.
Methods:
PackBitsCodec
PackBitsCodec(name: Literal['packbits'] = 'packbits', configuration: PackBitsConfig | JsonDict | Literal['first_byte', 'last_byte', 'none'] = PackBitsConfig(padding_encoding='none', first_bit=None, last_bit=None), must_understand: bool | int | str = True)
Bases: ArrayToBytesCodec
Packs a boolean array down to one bit per element for storage.
Attributes
Methods:
ScaleOffsetConfig
ScaleOffsetCodec
ScaleOffsetCodec(name: Literal['scale_offset'] = 'scale_offset', configuration: ScaleOffsetConfig | JsonDict | int | float = ScaleOffsetConfig(offset=0, scale=0), must_understand: bool | int | str = True)
Bases: ArrayToArrayCodec
Quantizes values as (value - offset) * scale, and reverses
the transform on decode.
The transform is lossy and useful for storing a bounded floating-point range in fewer bits.
Attributes
Methods:
VLenBytesCodec
VLenBytesCodec(name: Literal['vlen-bytes'] = 'vlen-bytes', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: ArrayToBytesCodec
Serializes an array of variable-length byte strings to bytes.
VLenUTF8Codec
VLenUTF8Codec(name: Literal['vlen-utf8'] = 'vlen-utf8', configuration: CodecConfig | JsonDict = CodecConfig(extra_items=FrozenDict({})), must_understand: bool | int | str = True)
Bases: ArrayToBytesCodec
Serializes an array of variable-length UTF-8 strings to bytes.
ReshapeConfig
ReshapeCodec
ReshapeCodec(name: Literal['reshape'] = 'reshape', configuration: ReshapeConfig | JsonDict | tuple[int | tuple[int, ...], ...] = ReshapeConfig(shape=()), must_understand: bool | int | str = True)
Bases: ArrayToArrayCodec
Reshapes an array to configuration.shape before the rest of
the pipeline.
The reshape is reversed on decode, so the array's logical shape is unchanged.
Attributes
Methods:
ZfpConfig
ZfpConfig(mode: Literal['reversible', 'expert', 'fixed_accuracy', 'fixed_rate', 'fixed_precision'] = 'reversible', minbits: Integral | integer | bool | None = None, maxbits: Integral | integer | bool | None = None, maxprec: Integral | integer | bool | None = None, minexp: Integral | integer | bool | None = None, tolerance: Real | floating | bool | None = None, rate: Real | floating | bool | None = None, precision: Integral | integer | bool | None = None)
Bases: CodecConfigImpl
Holds the ZFP codec's parameters: the mode, and that mode's own parameters.
Only the parameters belonging to mode apply.
"expert" takes minbits, maxbits, maxprec and minexp.
"fixed_accuracy" takes tolerance. "fixed_rate" takes
rate. "fixed_precision" takes precision.
"reversible" takes none of them.
Attributes
mode
instance-attribute
The accuracy mode ZFP compresses in: "reversible",
"expert", "fixed_accuracy", "fixed_rate" or
"fixed_precision".
minbits
class-attribute
instance-attribute
minbits: int | None = None
The minimum number of bits per block, in expert mode.
maxbits
class-attribute
instance-attribute
maxbits: int | None = None
The maximum number of bits per block, in expert mode.
maxprec
class-attribute
instance-attribute
maxprec: int | None = None
The maximum precision retained, in expert mode.
minexp
class-attribute
instance-attribute
minexp: int | None = None
The minimum bit-plane coding exponent, in expert mode.
tolerance
class-attribute
instance-attribute
tolerance: float | None = None
The maximum absolute error allowed, in fixed-accuracy mode.
rate
class-attribute
instance-attribute
rate: float | None = None
The number of bits per value, in fixed-rate mode.
precision
class-attribute
instance-attribute
precision: int | None = None
The number of bits of precision retained, in fixed-precision mode.
Methods:
ZfpCodec
ZfpCodec(name: Literal['zfp'] = 'zfp', configuration: ZfpConfig | JsonDict | Literal['reversible', 'expert', 'fixed_accuracy', 'fixed_rate', 'fixed_precision'] = ZfpConfig(mode='reversible', minbits=None, maxbits=None, maxprec=None, minexp=None, tolerance=None, rate=None, precision=None), must_understand: bool | int | str = True)
Bases: ArrayToBytesCodec
Compresses floating-point arrays with ZFP.
configuration.mode selects the accuracy/size trade-off.
Compression is lossy, except in "reversible" mode.
ZstdConfig
Bases: CodecConfigImpl
Holds the Zstd codec's parameters: compression level and an optional checksum.
Attributes
level
class-attribute
instance-attribute
level: int = 0
The compression level. Zstd's levels run from -131072 to 22, wider than the 0-9 range the core compressors use, and negative for the fast modes.
checksum
class-attribute
instance-attribute
checksum: bool = False
Whether a checksum of each frame is appended for integrity.
Methods:
to_version
to_version(version: ZarrVersion) -> Metadata
Convert this configuration to another Zarr version's Zstd codec.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
version
|
ZarrVersion
|
The target Zarr format version: 1, 2 or 3. |
required |
Returns:
| Type | Description |
|---|---|
Metadata
|
The equivalent Zstd codec for |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
ZstdCodec
ZstdCodec(name: Literal['zstd'] = 'zstd', configuration: ZstdConfig | JsonDict | int = ZstdConfig(level=0, checksum=False), must_understand: bool | int | str = True)
Bases: CompressorCodec
Compresses each chunk with Zstandard at a configurable compression level.
configuration.checksum, when set, appends a checksum of each
frame for integrity.