Skip to content

store views #3731

@d-v-b

Description

@d-v-b

Given some arbitrary key: value store (like the file system), in zarr we almost always want a small subset of that store. For example, when we work with a Zarr v3 array, we only want to expose the keys that are defined by that array (zarr.json and chunks). When we work with a group, we only want to expose the keys for metadata documents for sub-groups and the keys for arrays (see above).

Any other key (.ds_store, random jpgs) should probably not be visible to the array. This would allow a safe store-level "delete this array" operation that wouldn't risk hitting random objects that happen to share a prefix with the array. It would also allow us to safely expose arrays and groups over http without risk of accidentally leaking random user data.

This suggests a store_a -> store_b transformation that changes the keys of store_b relative to store_a, but keeps the same underlying kvstore backend. (this is actually supported today by a StorePath -> StorePath transformation -- the path attribute sets a lower bound in the 1D sequence of prefixes).

For arrays and groups this has to be dynamic -- for each key, we would need to evaluate a predicate against that key that checks if it's a metadata document or a chunk, based on the current zarr format and available metadata documents.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions