Skip to content

cannot use negative step to sel from zarr (without dask) #8252

Closed
@mathause

Description

@mathause

What happened?

As per: #8246 (comment)

Passing a negative step in a slice to select a non-chunked zarr-backed datasets raises an error.

What did you expect to happen?

zarr should allow negative step (probably?)

Minimal Complete Verifiable Example

import xarray as xr

# create a zarr dataset
air = xr.tutorial.open_dataset("air_temperature")
air.to_zarr("test.zarr")

ds = xr.open_dataset("test.zarr", engine="zarr")
ds.air[::-1, ].load()

# note that this works if the dataset is backed by dask
ds_dask = xr.open_dataset("test.zarr", engine="zarr", chunks="auto")
ds_dask.air[::-1, ].load()

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

File ~/code/xarray/xarray/core/parallelcompat.py:93, in guess_chunkmanager(manager)
     91 if isinstance(manager, str):
     92     if manager not in chunkmanagers:
---> 93         raise ValueError(
     94             f"unrecognized chunk manager {manager} - must be one of: {list(chunkmanagers)}"
     95         )
     97     return chunkmanagers[manager]
     98 elif isinstance(manager, ChunkManagerEntrypoint):
     99     # already a valid ChunkManager so just pass through

ValueError: unrecognized chunk manager dask - must be one of: []

Anything else we need to know?

The error comes from https://github.com/zarr-developers/zarr-python/blob/6ec746ef1242dd9fec26b128cc0b3455d28ad6f0/zarr/indexing.py#L174 so it would need an upstream fix first.

cc @dcherian is this what you had in mind?

Environment

INSTALLED VERSIONS

commit: f6d69a1
python: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0]
python-bits: 64
OS: Linux
OS-release: 6.2.0-33-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.2
libnetcdf: 4.9.2

xarray: 2023.9.1.dev8+gf6d69a1f
pandas: 2.1.1
numpy: 1.24.4
scipy: 1.11.3
netCDF4: 1.6.4
pydap: installed
h5netcdf: 1.2.0
h5py: 3.9.0
Nio: None
zarr: 2.16.1
cftime: 1.6.2
nc_time_axis: 1.4.1
PseudoNetCDF: 3.2.2
iris: 3.7.0
bottleneck: 1.3.7
dask: 2023.9.2
distributed: None
matplotlib: 3.8.0
cartopy: 0.22.0
seaborn: 0.12.2
numbagg: 0.2.2
fsspec: 2023.9.2
cupy: None
pint: 0.20.1
sparse: 0.14.0
flox: 0.7.2
numpy_groupies: 0.10.1
setuptools: 68.2.2
pip: 23.2.1
conda: None
pytest: 7.4.2
mypy: None
IPython: 8.15.0
sphinx: None

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions