HTTP server that exposes stores, arrays, groups

changes/3732.feature.md

@@ -0,0 +1,5 @@
+Adds an experimental HTTP server that can expose `Store`, `Array`, or `Group` instances over HTTP.
+`store_app` and `node_app` build ASGI applications; `serve_store` and `serve_node` additionally
+start a Uvicorn server (blocking by default, or in a background thread with `background=True`).
+See the [user guide](https://zarr.readthedocs.io/en/latest/user-guide/experimental.html#http-server)
+and the [example](https://zarr.readthedocs.io/en/latest/user-guide/examples/serve.html).

docs/api/zarr/experimental.md

@@ -4,6 +4,10 @@ title: experimental
 
 Experimental functionality is not stable and may change or be removed at any point.
 
-## Classes
+## Cache Store
 
 ::: zarr.experimental.cache_store
+
+## HTTP Server
+
+::: zarr.experimental.serve

docs/user-guide/examples/serve.md

@@ -0,0 +1,7 @@
+--8<-- "examples/serve/README.md"
+
+## Source Code
+
+```python
+--8<-- "examples/serve/serve.py"
+```

docs/user-guide/experimental.md

@@ -273,3 +273,132 @@ print(f"Cache contains {info['cached_keys']} keys with {info['current_size']} by
 This example shows how the CacheStore can significantly reduce access times for repeated
 data reads, particularly important when working with remote data sources. The dual-store
 architecture allows for flexible cache persistence and management.
+
+## HTTP Server
+
+Zarr Python provides an experimental HTTP server that exposes a Zarr `Store`, `Array`,
+or `Group` over HTTP as an [ASGI](https://asgi.readthedocs.io/) application.
+This makes it possible to serve zarr data to any HTTP-capable client (including
+another Zarr Python process backed by an `HTTPStore`).
+
+The server is built on [Starlette](https://www.starlette.io/) and can be run with
+any ASGI server such as [Uvicorn](https://www.uvicorn.org/).
+
+Install the server dependencies with:
+
+```bash
+pip install zarr[server]
+```
+
+### Building an ASGI App
+
+[`zarr.experimental.serve.store_app`][] creates an ASGI app that exposes every key
+in a store:
+
+```python
+import zarr
+from zarr.experimental.serve import store_app
+
+store = zarr.storage.MemoryStore()
+zarr.create_array(store, shape=(100, 100), chunks=(10, 10), dtype="float64")
+
+app = store_app(store)
+
+# Run with any ASGI server, e.g. Uvicorn:
+# uvicorn my_module:app --host 0.0.0.0 --port 8000
+```
+
+[`zarr.experimental.serve.node_app`][] creates an ASGI app that only serves keys
+belonging to a specific `Array` or `Group`.  Requests for keys outside the node
+receive a 404, even if those keys exist in the underlying store:
+
+```python
+import zarr
+from zarr.experimental.serve import node_app
+
+store = zarr.storage.MemoryStore()
+root = zarr.open_group(store)
+root.create_array("a", shape=(10,), dtype="int32")
+root.create_array("b", shape=(20,), dtype="float64")
+
+# Only serve the array at "a" — requests for "b" will return 404.
+arr = root["a"]
+app = node_app(arr)
+```
+
+### Running the Server
+
+[`zarr.experimental.serve.serve_store`][] and [`zarr.experimental.serve.serve_node`][]
+build an ASGI app *and* start a [Uvicorn](https://www.uvicorn.org/) server.
+By default they block until the server is shut down:
+
+```python
+from zarr.experimental.serve import serve_store
+
+serve_store(store, host="127.0.0.1", port=8000)
+```
+
+Pass `background=True` to start the server in a daemon thread and return
+immediately.  The returned [`BackgroundServer`][zarr.experimental.serve.BackgroundServer]
+can be used as a context manager for automatic shutdown:
+
+```python
+import numpy as np
+
+import zarr
+from zarr.experimental.serve import serve_node
+from zarr.storage import MemoryStore
+
+store = MemoryStore()
+arr = zarr.create_array(store, shape=(100,), chunks=(10,), dtype="float64")
+arr[:] = np.arange(100, dtype="float64")
+
+with serve_node(arr, host="127.0.0.1", port=8000, background=True) as server:
+    # Now open the served array from another zarr client.
+    remote = zarr.open_array(server.url, mode="r")
+    np.testing.assert_array_equal(remote[:], arr[:])
+# Server is shut down automatically when the block exits.
+```
+
+### CORS Support
+
+Both `store_app` and `node_app` (and their `serve_*` counterparts) accept a
+[`CorsOptions`][zarr.experimental.serve.CorsOptions] parameter to enable
+[CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) middleware for
+browser-based clients:
+
+```python
+from zarr.experimental.serve import CorsOptions, store_app
+
+app = store_app(
+    store,
+    cors_options=CorsOptions(
+        allow_origins=["*"],
+        allow_methods=["GET"],
+    ),
+)
+```
+
+### HTTP Range Requests
+
+The server supports the standard `Range` header for partial reads.  The three
+forms defined by [RFC 7233](https://httpwg.org/specs/rfc7233.html) are supported:
+
+| Header               | Meaning                        |
+| -------------------- | ------------------------------ |
+| `bytes=0-99`         | First 100 bytes                |
+| `bytes=100-`         | Everything from byte 100       |
+| `bytes=-50`          | Last 50 bytes                  |
+
+A successful range request returns HTTP 206 (Partial Content).
+
+### Write Support
+
+By default only `GET` requests are accepted.  To enable writes, pass
+`methods={"GET", "PUT"}`:
+
+```python
+app = store_app(store, methods={"GET", "PUT"})
+```
+
+A `PUT` request stores the request body at the given path and returns 204 (No Content).

examples/serve/README.md

@@ -0,0 +1,17 @@
+# Serve a Zarr Array over HTTP
+
+This example creates an in-memory Zarr array, serves it over HTTP with
+`zarr.experimental.serve.serve_node`, and fetches the `zarr.json` metadata
+document and a raw chunk using `httpx`.
+
+## Running the Example
+
+```bash
+python examples/serve/serve.py
+```
+
+Or run with uv:
+
+```bash
+uv run examples/serve/serve.py
+```

examples/serve/serve.py

@@ -0,0 +1,43 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#   "zarr[server] @ git+https://github.com/zarr-developers/zarr-python.git@main",
+#   "httpx",
+# ]
+# ///
+"""
+Serve a Zarr array over HTTP and fetch its metadata and chunks.
+
+This example creates an in-memory array, serves it in a background thread,
+then uses ``httpx`` to request the ``zarr.json`` metadata document and a raw
+chunk.
+"""
+
+import json
+
+import httpx
+import numpy as np
+
+import zarr
+from zarr.experimental.serve import serve_node
+from zarr.storage import MemoryStore
+
+# -- create an array --------------------------------------------------------
+store = MemoryStore()
+data = np.arange(1000, dtype="uint8").reshape(10, 10, 10)
+# no compression
+arr = zarr.create_array(store, data=data, chunks=(5, 5, 5), write_data=True, compressors=None)
+
+# -- serve it in the background ---------------------------------------------
+with serve_node(arr, host="127.0.0.1", port=8000, background=True) as server:
+    # -- fetch metadata ------------------------------------------------------
+    resp = httpx.get(f"{server.url}/zarr.json")
+    assert resp.status_code == 200
+    meta = resp.json()
+    print("zarr.json:")
+    print(json.dumps(meta, indent=2))
+
+    # -- fetch a raw chunk ---------------------------------------------------
+    resp = httpx.get(f"{server.url}/c/0/0/0")
+    assert resp.status_code == 200
+    print(f"\nchunk c/0/0/0: {len(resp.content)} bytes")

mkdocs.yml

@@ -29,6 +29,7 @@ nav:
       - user-guide/experimental.md
   - Examples:
       - user-guide/examples/custom_dtype.md
+      - user-guide/examples/serve.md
   - API Reference:
       - api/zarr/index.md
       - api/zarr/array.md

pyproject.toml

@@ -70,6 +70,11 @@ gpu = [
     "cupy-cuda12x",
 ]
 cli = ["typer"]
+server = [
+    "starlette",
+    "httpx",
+    "uvicorn",
+]
 optional = ["rich", "universal-pathlib"]
 
 [project.scripts]

src/zarr/core/chunk_key_encodings.py

@@ -79,7 +79,11 @@ def __post_init__(self) -> None:
     def decode_chunk_key(self, chunk_key: str) -> tuple[int, ...]:
         if chunk_key == "c":
             return ()
-        return tuple(map(int, chunk_key[1:].split(self.separator)))
+        # Strip the "c<sep>" prefix (e.g. "c/" or "c.") before splitting.
+        prefix = "c" + self.separator
+        if chunk_key.startswith(prefix):
+            return tuple(map(int, chunk_key[len(prefix) :].split(self.separator)))
+        raise ValueError(f"Invalid chunk key for default encoding: {chunk_key!r}")
 
     def encode_chunk_key(self, chunk_coords: tuple[int, ...]) -> str:
         return self.separator.join(map(str, ("c",) + chunk_coords))