Xpublish lets you publish Xarray datasets via a Zarr-compatible REST API.
On the server-side, datasets are published using a simple Xarray accessor:
On the client-side, datasets are accessed using Zarr and fsspec.
In : import xarray as xr In : import zarr In : from fsspec.implementations.http import HTTPFileSystem In : fs = HTTPFileSystem() In : http_map = fs.get_mapper('http://0.0.0.0:9000') # open as a zarr group In : zg = zarr.open_consolidated(http_map, mode='r') # or open as another xarray dataset In : ds = xr.open_zarr(http_map, consolidated=True)
Xpublish lets you share, publish, and serve Xarray datasets via a web application. The data in the Xarray datasets (on the server side) can be backed by dask to facilitate on-demand computation via a simple REST API.
We are exploring applications of xpublish that include:
- publish on-demand or derived data products
- turning Xarray objects into streaming services (e.g. OPENDAP)
Under the hood, xpublish is using a web app (FastAPI and Uvicorn) that is exposing a minimal Zarr compatible REST-like API.
Key attributes of the API are:
- serves a Zarr store API from the root of the dataset.
- provides Zarr metadata keys (
.zattrs) as a JSON strings.
- provides access to data keys (e.g.
var/0.0.0) as binary strings.