Building a dataset provider plugin

So far, we’ve been eagerly loading datasets for Xpublish to serve, but this tends not to scale well between memory needs and slow startup. Xpublish plugins can also be Dataset Providers and handle loading of datasets on request.

This also allows organizations to quickly be able to adapt Xpublish to work in their own environment, rather than needing Xpublish to explicitly support it.

import xarray as xr
from requests import HTTPError

from xpublish import Plugin, Rest, hookimpl


class TutorialDataset(Plugin):
    name: str = 'xarray-tutorial-dataset'

    @hookimpl
    def get_datasets(self):
        return list(xr.tutorial.file_formats)

    @hookimpl
    def get_dataset(self, dataset_id: str):
        try:
            return xr.tutorial.open_dataset(dataset_id)
        except HTTPError:
            return None


rest = Rest({})
rest.register_plugin(TutorialDataset())
rest.serve()

With this plugin, Xpublish can serve the same datasets as we explictly defined and loaded in serving multiple datasets, as well as any others supported by xr.tutorial

Note

For more details on building dataset provider plugins, please see the plugin user guide