Skip to content

abczarr.abc.store

The store surface: a key to bytes map beneath every Zarr node.

A store is what every Zarr array and group reads and writes through. Keys are "/"-joined relative strings, such as "zarr.json" or "c/0/0". Values are bytes. The surface is deliberately small, with five primitives: get reads a key and returns None when the key is missing, set writes a key, delete removes a key, exists reports whether a key is present, and list_keys iterates the keys under a prefix.

Everything richer than the five primitives is built from them for free. This includes listing one directory level, clearing the store, sizing a key, and batching writes into a transaction. A new store therefore only has to implement the primitives. A backend that can do better than the built-in version of one of those operations overrides it, and advertises the faster path through [supports][abczarr.abc.capabilities.SupportsCapabilities.supports].

PathBasedStore is the default store. It turns each key into a path under a root and delegates to bagof.paths, so a local directory and every fsspec or cloud scheme, such as s3://, gs:// or memory://, works with no extra code. AsyncStore and AsyncPathBasedStore are the coroutine twins, built over bagof.paths.AsyncPath.

Classes

StorePath 

StorePath(*args: Any, read_only: bool = False, **kwargs: Any)

Bases: Path

A Zarr store location: a path, plus a read_only flag.

A StorePath accepts everything bagof.paths.Path accepts, a local path or a URL for any scheme it supports, plus a read_only keyword.

Attributes

read_only instance-attribute 
read_only = read_only

Methods:

AsyncStorePath 

AsyncStorePath(*args: Any, read_only: bool = False, **kwargs: Any)

Bases: AsyncPath

The async counterpart of StorePath.

Attributes

read_only instance-attribute 
read_only = read_only

Methods:

Store 

Store(store_path: str | PathLike | StorePath | None = None)

Bases: SupportsCapabilities, ABC

A key to bytes map, addressed under a StorePath root.

A subclass implements the five primitives, get, set, delete, exists and list_keys, and gets everything else for free. Keys are "/"-separated relative strings, such as "zarr.json" or "c/0/0". Values are bytes. Use [capability][abczarr.abc.capabilities.SupportsCapabilities.capability] or [supports][abczarr.abc.capabilities.SupportsCapabilities.supports] to check whether a given operation is native to the backend or built from the primitives.

A Store is also a context manager. It can be used with with to make sure its resources are released.

Example

>>> store = PathBasedStore("/tmp/demo-store")
>>> store.set("a", b"1")
>>> store.get("a")
b'1'
>>> store.get("missing") is None
True
>>> list(store.list_keys())
['a']

Attributes

native property 
native: Any

The underlying backend object.

This property is the escape hatch for anything the uniform surface does not name.

store_path property 
store_path: StorePath | None

The store's root, or None when it has no path.

Returns:

Type Description
[StorePath][StorePath] or None
read_only property 
read_only: bool

Whether the store refuses writes.

url property 
url: str | None

The store root as a URL, or None when it has no path.

Methods:

get abstractmethod 
get(key: str) -> bytes | None

Read key.

Parameters:

Name Type Description Default
key str

The key to read.

 required

Returns:

Type Description
bytes or None

The stored value, or None when key is not present. The result is bytes-like. Wrap it in bytes(...) to obtain exactly bytes.
set abstractmethod 
set(key: str, value: _BytesLike) -> None

Write value at key, creating any parent structure.

Parameters:

Name Type Description Default
key str

The key to write.

 required
value bytes - like

The value to store: bytes or any buffer object such as bytearray or memoryview.

 required
delete abstractmethod 
delete(key: str) -> None

Remove key.

Removing a missing key is not an error.

Parameters:

Name Type Description Default
key str

The key to remove.

 required
exists abstractmethod 
exists(key: str) -> bool

Whether key is present.

Parameters:

Name Type Description Default
key str

The key to check.

 required
list_keys abstractmethod 
list_keys(prefix: str = '') -> Iterator[str]

Iterate every key at or below prefix, "/"-joined.

Parameters:

Name Type Description Default
prefix str

The prefix keys are listed under. The empty string, the default, lists every key in the store.

 ''
capability 
capability(capability: str) -> Support

How this store provides capability.

A store declares what it does natively. For an operation this base class can always build from the primitives, such as a byte-range read, the answer is at least Support.SYNTHESIZED, even when the store declares nothing.

get_many 
get_many(keys: Iterable[str]) -> dict[str, bytes | None]

Read several keys at once.

Parameters:

Name Type Description Default
keys iterable of str

The keys to read.

 required

Returns:

Type Description
dict

Maps each key to its value, or to None when the key is missing. A store with a real batched read overrides this method. The default implementation reads one key at a time.
get_partial 
get_partial(key: str, start: int, length: int | None = None) -> bytes | None

Read a byte range of key.

Parameters:

Name Type Description Default
key str

The key to read.

 required
start int

The offset, in bytes, to start reading from.

 required
length int

How many bytes to read. Reads to the end of the value when omitted.

 None

Returns:

Type Description
bytes or None

The requested range, or None when key is missing. The default implementation reads the whole value and slices it. A store with a native byte-range read overrides this method and declares "partial_read" as Support.NATIVE.
set_if_not_exists 
set_if_not_exists(key: str, value: _BytesLike) -> bool

Write value only when key is absent.

Parameters:

Name Type Description Default
key str

The key to write.

 required
value bytes - like

The value to store.

 required

Returns:

Type Description
bool

Whether the write happened.
This method is built from `exists` then `set`, so it is
racy under concurrent writers. A store with an atomic put
overrides it.
delete_prefix 
delete_prefix(prefix: str = '') -> None

Delete every key at or below prefix.

Parameters:

Name Type Description Default
prefix str

The prefix to delete under. The empty string, the default, deletes every key in the store.

 ''
transaction 
transaction(*, atomic: bool = True) -> Transaction

Open a transaction: a store view whose writes commit together.

A store whose backend has real transactions returns a native transaction. Every other store returns a BufferedTransaction, which can only honor atomic=False. Passing atomic=True on such a store raises, rather than pretend that a partial write is atomic.

Parameters:

Name Type Description Default
atomic bool

Whether the commit must be all-or-nothing.

 True

Returns:

Type Description
Transaction

The opened transaction.
list_dir 
list_dir(prefix: str = '') -> Iterator[str]

Iterate the immediate child names one level below prefix.

list_keys walks the whole subtree. This method yields only the distinct first segments below prefix, the way listdir names one level.

Parameters:

Name Type Description Default
prefix str

The prefix children are listed under. The empty string, the default, lists the top level of the store.

 ''
getsize 
getsize(key: str) -> int | None

The size of key in bytes, or None when it is missing.

Parameters:

Name Type Description Default
key str

The key to size.

 required
clear 
clear() -> None

Remove every key in the store.

close 
close() -> None

Release any resources the store holds. A no-op by default.

__contains__ 
__contains__(key: str) -> bool
__iter__ 
__iter__() -> Iterator[str]
__enter__ 
__enter__() -> Self
__exit__ 
__exit__(exc_type: Type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None) -> None

PathBasedStore 

PathBasedStore(store_path: str | StorePath)

Bases: Store

The default store: keys are paths under a root.

A PathBasedStore turns each key into a path below its root and delegates to bagof.paths. Every filesystem and cloud scheme bagof.paths understands works here with no extra code. The root's scheme picks the driver, and credentials ride on the root path's storage_options.

Example

>>> store = PathBasedStore("/tmp/demo-store")
>>> store.set("zarr.json", b'{"zarr_format": 3}')
>>> store.get("zarr.json")
b'{"zarr_format": 3}'

Parameters:

Name Type Description Default
store_path str or [StorePath][StorePath]

The root all keys are resolved under: a local path or a URL such as "s3://bucket/prefix".

 required

Methods:

get 
get(key: str) -> bytes | None
set 
set(key: str, value: _BytesLike) -> None
delete 
delete(key: str) -> None
exists 
exists(key: str) -> bool
list_keys 
list_keys(prefix: str = '') -> Iterator[str]

AsyncStore 

AsyncStore(store_path: str | PathLike | AsyncStorePath | None = None)

Bases: SupportsCapabilities, ABC

The coroutine twin of Store.

The five primitives are coroutines, and list_keys is an async iterator. Everything else mirrors Store. Location and capability queries never touch the backend, so they stay synchronous.

Attributes

native property 
native: Any

The underlying backend object.

This property is the escape hatch for anything the uniform surface does not name.

store_path property 
store_path: AsyncStorePath | None

The store's root, or None when it has no path.

Returns:

Type Description
[AsyncStorePath][AsyncStorePath] or None
read_only property 
read_only: bool

Whether the store refuses writes.

url property 
url: str | None

The store root as a URL, or None when it has no path.

Methods:

get abstractmethod async 
get(key: str) -> bytes | None

Read key.

Parameters:

Name Type Description Default
key str

The key to read.

 required

Returns:

Type Description
bytes or None

The stored value, or None when key is not present. The result is bytes-like. Wrap it in bytes(...) to obtain exactly bytes.
set abstractmethod async 
set(key: str, value: _BytesLike) -> None

Write value at key, creating any parent structure.

Parameters:

Name Type Description Default
key str

The key to write.

 required
value bytes - like

The value to store: bytes or any buffer object such as bytearray or memoryview.

 required
delete abstractmethod async 
delete(key: str) -> None

Remove key.

Removing a missing key is not an error.

Parameters:

Name Type Description Default
key str

The key to remove.

 required
exists abstractmethod async 
exists(key: str) -> bool

Whether key is present.

Parameters:

Name Type Description Default
key str

The key to check.

 required
list_keys abstractmethod 
list_keys(prefix: str = '') -> AsyncIterator[str]

Async-iterate every key at or below prefix.

Parameters:

Name Type Description Default
prefix str

The prefix keys are listed under. The empty string, the default, lists every key in the store.

 ''
capability 
capability(capability: str) -> Support

How this store provides capability.

See Store.capability.

get_many async 
get_many(keys: Iterable[str]) -> dict[str, bytes | None]

Read several keys concurrently.

Parameters:

Name Type Description Default
keys iterable of str

The keys to read.

 required

Returns:

Type Description
dict

Maps each key to its value, or to None when the key is missing.
get_partial async 
get_partial(key: str, start: int, length: int | None = None) -> bytes | None

Read a byte range of key.

Parameters:

Name Type Description Default
key str

The key to read.

 required
start int

The offset, in bytes, to start reading from.

 required
length int

How many bytes to read. Reads to the end of the value when omitted.

 None

Returns:

Type Description
bytes or None

The requested range, or None when key is missing. The default implementation reads the whole value and slices it. A store with a native byte-range read overrides this method and declares "partial_read" as Support.NATIVE.
set_if_not_exists async 
set_if_not_exists(key: str, value: _BytesLike) -> bool

Write value only when key is absent.

Parameters:

Name Type Description Default
key str

The key to write.

 required
value bytes - like

The value to store.

 required

Returns:

Type Description
bool

Whether the write happened.
This method is built from `exists` then `set`, so it is
racy under concurrent writers.
delete_prefix async 
delete_prefix(prefix: str = '') -> None

Delete every key at or below prefix.

Parameters:

Name Type Description Default
prefix str

The prefix to delete under. The empty string, the default, deletes every key in the store.

 ''
transaction 
transaction(*, atomic: bool = True) -> AsyncTransaction

Open a transaction.

Parameters:

Name Type Description Default
atomic bool

Whether the commit must be all-or-nothing.

 True

Returns:

Type Description
AsyncTransaction

The opened transaction.
See[transaction][transaction]
for the full behavior.
list_dir async 
list_dir(prefix: str = '') -> AsyncIterator[str]

Async-iterate the child names one level below prefix.

Parameters:

Name Type Description Default
prefix str

The prefix children are listed under. The empty string, the default, lists the top level of the store.

 ''
getsize async 
getsize(key: str) -> int | None

The size of key in bytes, or None when it is missing.

Parameters:

Name Type Description Default
key str

The key to size.

 required
clear async 
clear() -> None

Remove every key in the store.

close async 
close() -> None

Release any resources the store holds. A no-op by default.

__aenter__ async 
__aenter__() -> Self
__aexit__ async 
__aexit__(exc_type: Type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None) -> None

AsyncPathBasedStore 

AsyncPathBasedStore(store_path: str | AsyncStorePath)

Bases: AsyncStore

The default async store, built over bagof.paths.AsyncPath.

A local directory and every fsspec or cloud scheme work here with no extra code. A synchronous backend runs in a worker thread. A natively async backend, such as an fsspec cloud backend, is awaited directly. Both present the same coroutine surface.

Methods:

get async 
get(key: str) -> bytes | None
set async 
set(key: str, value: _BytesLike) -> None
delete async 
delete(key: str) -> None
exists async 
exists(key: str) -> bool
list_keys async 
list_keys(prefix: str = '') -> AsyncIterator[str]