Skip to content

combine_by_coords should allow for missing panels in hypercube #3648

Closed
@bolliger32

Description

@bolliger32

MCVE Code Sample

import numpy as np
import xarray as xr
x1 = xr.Dataset(
     {
         "temperature": (("y", "x"), 20 * np.random.rand(6).reshape(2, 3))
     },
     coords={"y": [0, 1], "x": [10, 20, 30]},
)
x2 = xr.Dataset(
     {
         "temperature": (("y", "x"), 20 * np.random.rand(6).reshape(2, 3))
     },
     coords={"y": [2, 3], "x": [10, 20, 30]},
)
x3 = xr.Dataset(
     {
         "temperature": (("y", "x"), 20 * np.random.rand(6).reshape(2, 3))
     },
     coords={"y": [2, 3], "x": [40, 50, 60]},
)
xr.combine_by_coords([x1,x2,x3])

Expected Output

<xarray.Dataset>
Dimensions:      (x: 6, y: 4)
Coordinates:
  * x            (x) int64 10 20 30 40 50 60
  * y            (y) int64 0 1 2 3
Data variables:
    temperature  (y, x) float64 14.11 19.19 10.77 nan ... 4.86 10.57 4.38 15.09

Problem Description

Currently, it throws the following error:

ValueError: The supplied objects do not form a hypercube because sub-lists do not have consistent lengths along dimension0

This is b/c combine_by_coords calls xr.core.combine._check_shape_tile_ids, which mandates that the passed datasets form a complete hypercube. This check functiono also serves the purpose of mandating that the dimension depths are the same. Could we pull that part out as a separate function and, for combine_by_coords, only call this first part but NOT mandate that the hypercube is complete? The expected behavior, in my mind, should be to simply replace the missing tiles of the hypercube with fill_value. I'll file a PR to this effect and welcome comments.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.7.3 | packaged by conda-forge | (default, Dec 6 2019, 08:54:18) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.14.150+ machine: x86_64 processor: x86_64 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.1

xarray: 0.14.1
pandas: 0.25.3
numpy: 1.17.3
scipy: 1.3.2
netCDF4: 1.5.3
pydap: None
h5netcdf: 0.7.4
h5py: 2.10.0
Nio: None
zarr: 2.3.2
cftime: 1.0.4.2
nc_time_axis: 1.2.0
PseudoNetCDF: None
rasterio: 1.1.1
cfgrib: None
iris: 2.2.0
bottleneck: None
dask: 2.8.1
distributed: 2.8.1
matplotlib: 3.1.2
cartopy: 0.17.0
seaborn: 0.9.0
numbagg: None
setuptools: 42.0.2.post20191201
pip: 19.3.1
conda: 4.8.0
pytest: 5.3.1
IPython: 7.10.1
sphinx: 2.2.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions