-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Coarsen keep attrs 3376 #3801
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Coarsen keep attrs 3376 #3801
Changes from 9 commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
dc6ca01
Add test of DataWithCoords.coarsen() for #3376
amcnicho cb1e6c4
Add test of Variable.coarsen() for #3376
amcnicho ade6a49
Add keep_attrs kwarg to DataWithCoords.coarsen() for #3376
amcnicho 523cca4
Style and spelling fixes (#3376)
amcnicho 5113bf3
Fix test_coarsen_keep_attrs by removing self from input
amcnicho df1b0dc
Pass keep_attrs through to _coarsen_cls and _rolling_cls returns (#3376)
amcnicho af7bd10
Move keyword from coarsen to mean in test_coarsen_keep_attrs
amcnicho 4a45840
Start handling keep_attrs in rolling class constructors (#3376)
amcnicho 28f705f
Update Coarsen constructor and DatasetCoarsen class method (GH3376)
amcnicho 2c61115
Incorporate code review from @max-sixty
amcnicho e05ca96
Fix Dataset.coarsen and Variable.coarsen for GH3376
amcnicho d326404
Update tests for GH3376
amcnicho 735dc5a
Incorporate review changes to test_dataset for GH3376
amcnicho cd43608
Merge branch 'master' into coarsen-keep_attrs-3376
amcnicho 496838c
Change Rolling._dataset_implementation for GH3376
amcnicho 975a796
style fixes
amcnicho 98edaa8
Remove duplicate variable assignment and document change (GH3776)
amcnicho 4968c44
Merge branch 'master' into coarsen-keep_attrs-3376 to resolve doc/wha…
amcnicho File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -7,6 +7,7 @@ | |||||
from . import dtypes, duck_array_ops, utils | ||||||
from .dask_array_ops import dask_rolling_wrapper | ||||||
from .ops import inject_reduce_methods | ||||||
from .options import _get_keep_attrs | ||||||
from .pycompat import dask_array_type | ||||||
|
||||||
try: | ||||||
|
@@ -42,10 +43,10 @@ class Rolling: | |||||
DataArray.rolling | ||||||
""" | ||||||
|
||||||
__slots__ = ("obj", "window", "min_periods", "center", "dim") | ||||||
_attributes = ("window", "min_periods", "center", "dim") | ||||||
__slots__ = ("obj", "window", "min_periods", "center", "dim", "keep_attrs") | ||||||
_attributes = ("window", "min_periods", "center", "dim", "keep_attrs") | ||||||
|
||||||
def __init__(self, obj, windows, min_periods=None, center=False): | ||||||
def __init__(self, obj, windows, min_periods=None, center=False, keep_attrs=None): | ||||||
""" | ||||||
Moving window object. | ||||||
|
||||||
|
@@ -65,6 +66,10 @@ def __init__(self, obj, windows, min_periods=None, center=False): | |||||
setting min_periods equal to the size of the window. | ||||||
center : boolean, default False | ||||||
Set the labels at the center of the window. | ||||||
keep_attrs : bool, optional | ||||||
If True, the object's attributes (`attrs`) will be copied from | ||||||
the original object to the new one. If False (default), the new | ||||||
object will be returned without attributes. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
|
@@ -89,6 +94,10 @@ def __init__(self, obj, windows, min_periods=None, center=False): | |||||
self.center = center | ||||||
self.dim = dim | ||||||
|
||||||
if keep_attrs is None: | ||||||
keep_attrs = _get_keep_attrs(default=False) | ||||||
self.keep_attrs = keep_attrs | ||||||
|
||||||
@property | ||||||
def _min_periods(self): | ||||||
return self.min_periods if self.min_periods is not None else self.window | ||||||
|
@@ -143,7 +152,7 @@ def count(self): | |||||
class DataArrayRolling(Rolling): | ||||||
__slots__ = ("window_labels",) | ||||||
|
||||||
def __init__(self, obj, windows, min_periods=None, center=False): | ||||||
def __init__(self, obj, windows, min_periods=None, center=False, keep_attrs=None): | ||||||
""" | ||||||
Moving window object for DataArray. | ||||||
You should use DataArray.rolling() method to construct this object | ||||||
|
@@ -165,6 +174,10 @@ def __init__(self, obj, windows, min_periods=None, center=False): | |||||
setting min_periods equal to the size of the window. | ||||||
center : boolean, default False | ||||||
Set the labels at the center of the window. | ||||||
keep_attrs : bool, optional | ||||||
If True, the object's attributes (`attrs`) will be copied from | ||||||
the original object to the new one. If False (default), the new | ||||||
object will be returned without attributes. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
|
@@ -177,7 +190,9 @@ def __init__(self, obj, windows, min_periods=None, center=False): | |||||
Dataset.rolling | ||||||
Dataset.groupby | ||||||
""" | ||||||
super().__init__(obj, windows, min_periods=min_periods, center=center) | ||||||
if keep_attrs is None: | ||||||
keep_attrs = _get_keep_attrs(default=False) | ||||||
super().__init__(obj, windows, min_periods=min_periods, center=center, keep_attrs=keep_attrs) | ||||||
|
||||||
self.window_labels = self.obj[self.dim] | ||||||
|
||||||
|
@@ -374,7 +389,7 @@ def _numpy_or_bottleneck_reduce( | |||||
class DatasetRolling(Rolling): | ||||||
__slots__ = ("rollings",) | ||||||
|
||||||
def __init__(self, obj, windows, min_periods=None, center=False): | ||||||
def __init__(self, obj, windows, min_periods=None, center=False, keep_attrs=None): | ||||||
""" | ||||||
Moving window object for Dataset. | ||||||
You should use Dataset.rolling() method to construct this object | ||||||
|
@@ -396,6 +411,10 @@ def __init__(self, obj, windows, min_periods=None, center=False): | |||||
setting min_periods equal to the size of the window. | ||||||
center : boolean, default False | ||||||
Set the labels at the center of the window. | ||||||
keep_attrs : bool, optional | ||||||
If True, the object's attributes (`attrs`) will be copied from | ||||||
the original object to the new one. If False (default), the new | ||||||
object will be returned without attributes. | ||||||
|
||||||
Returns | ||||||
------- | ||||||
|
@@ -408,15 +427,15 @@ def __init__(self, obj, windows, min_periods=None, center=False): | |||||
Dataset.groupby | ||||||
DataArray.groupby | ||||||
""" | ||||||
super().__init__(obj, windows, min_periods, center) | ||||||
super().__init__(obj, windows, min_periods, center, keep_attrs) | ||||||
if self.dim not in self.obj.dims: | ||||||
raise KeyError(self.dim) | ||||||
# Keep each Rolling object as a dictionary | ||||||
self.rollings = {} | ||||||
for key, da in self.obj.data_vars.items(): | ||||||
# keeps rollings only for the dataset depending on slf.dim | ||||||
if self.dim in da.dims: | ||||||
self.rollings[key] = DataArrayRolling(da, windows, min_periods, center) | ||||||
self.rollings[key] = DataArrayRolling(da, windows, min_periods, center, keep_attrs) | ||||||
|
||||||
def _dataset_implementation(self, func, **kwargs): | ||||||
from .dataset import Dataset | ||||||
|
@@ -466,7 +485,7 @@ def _numpy_or_bottleneck_reduce( | |||||
**kwargs, | ||||||
) | ||||||
|
||||||
def construct(self, window_dim, stride=1, fill_value=dtypes.NA): | ||||||
def construct(self, window_dim, stride=1, fill_value=dtypes.NA, keep_attrs=None): | ||||||
""" | ||||||
Convert this rolling object to xr.Dataset, | ||||||
where the window dimension is stacked as a new dimension | ||||||
|
@@ -487,6 +506,9 @@ def construct(self, window_dim, stride=1, fill_value=dtypes.NA): | |||||
|
||||||
from .dataset import Dataset | ||||||
|
||||||
if keep_attrs is None: | ||||||
keep_attrs = _get_keep_attrs(default=True) | ||||||
|
||||||
dataset = {} | ||||||
for key, da in self.obj.data_vars.items(): | ||||||
if self.dim in da.dims: | ||||||
|
@@ -509,10 +531,10 @@ class Coarsen: | |||||
DataArray.coarsen | ||||||
""" | ||||||
|
||||||
__slots__ = ("obj", "boundary", "coord_func", "windows", "side", "trim_excess") | ||||||
__slots__ = ("obj", "boundary", "coord_func", "windows", "side", "trim_excess", "keep_attrs") | ||||||
_attributes = ("windows", "side", "trim_excess") | ||||||
|
||||||
def __init__(self, obj, windows, boundary, side, coord_func): | ||||||
def __init__(self, obj, windows, boundary, side, coord_func, keep_attrs): | ||||||
""" | ||||||
Moving window object. | ||||||
|
||||||
|
@@ -541,6 +563,7 @@ def __init__(self, obj, windows, boundary, side, coord_func): | |||||
self.windows = windows | ||||||
self.side = side | ||||||
self.boundary = boundary | ||||||
self.keep_attrs = keep_attrs | ||||||
|
||||||
absent_dims = [dim for dim in windows.keys() if dim not in self.obj.dims] | ||||||
if absent_dims: | ||||||
|
@@ -586,7 +609,7 @@ def wrapped_func(self, **kwargs): | |||||
from .dataarray import DataArray | ||||||
|
||||||
reduced = self.obj.variable.coarsen( | ||||||
self.windows, func, self.boundary, self.side, **kwargs | ||||||
self.windows, func, self.boundary, self.side, self.keep_attrs, **kwargs | ||||||
) | ||||||
coords = {} | ||||||
for c, v in self.obj.coords.items(): | ||||||
|
@@ -626,6 +649,11 @@ def _reduce_method(cls, func: Callable, include_skipna: bool, numeric_only: bool | |||||
def wrapped_func(self, **kwargs): | ||||||
from .dataset import Dataset | ||||||
|
||||||
if self.keep_attrs == True: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
_attrs = self.obj.attrs | ||||||
else: | ||||||
attrs = {} | ||||||
|
||||||
reduced = {} | ||||||
for key, da in self.obj.data_vars.items(): | ||||||
reduced[key] = da.variable.coarsen( | ||||||
|
@@ -644,7 +672,7 @@ def wrapped_func(self, **kwargs): | |||||
) | ||||||
else: | ||||||
coords[c] = v.variable | ||||||
return Dataset(reduced, coords=coords) | ||||||
return Dataset(reduced, coords=coords, attrs=attrs) | ||||||
|
||||||
return wrapped_func | ||||||
|
||||||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.