win_type for rolling() ?

[peterkamatej]

Hi, is there any plan to include support for various win_types such as "gaussian" to the xarray's rolling() methods like in pandas rolling()? I started using something like DataArray.to_pandas().rolling(...).. .to_xarray(), but in this way it's tricky to adhere to the xarray's flexibility when it comes to N-dimensionality..

[shoyer]

There are no immediate plans that I know of, but I think we are certainly open to adding this functionality if someone wants to look into it.

cc @jhamman @MaximilianR

[max-sixty]

Think it would be useful generally. @jhamman will have a better view of how difficult it is to implement. Our use cases are fine with simple windows, although an ewm object would be helpful.

One API choice is to what extent we want .rolling, .expanding & .ewm to be separate objects vs kwargs to .rolling

[jhamman]

Certainly open to adding this functionality. Bottleneck isn't going to help so the code will all live in xarray. I think it makes sense to have a bit of a design discussion here prior to getting started. Questions:

1. What window types are you interested in adding? Pandas includes:

• boxcar
• triang
• blackman
• hamming
• bartlett
• parzen
• bohman
• blackmanharris
• nuttall
• barthann
• kaiser (needs beta)
• gaussian (needs std)
• general_gaussian (needs power, width)
• slepian (needs width)

2. Can we maintain compatibility with pandas and bottleneck.

We have tried to maintain compatibility with both pandas and bottleneck in our rolling implementation. This is proving somewhat difficult (e.g. #1046) but is a design consideration that we should keep in mind.

Also, our current implementation falls back to operating on individual slices of the DataArray when bottleneck cannot be utilized. We should think a bit about the applicability of other windows when using other window types. Presumably, each win_type listed above would just provide a different set of weights to be associated with the members of each window.

3. What about nd windows?

Our current implementation left open the possibility of n-dimensional windows (see #819). While we haven't implemented it yet, the utility of the rolling object for a 2+ dimensional smoother would be quite a nice feature. That said, I'd be hesitant to implement anything on the rolling object that would not allow us to make this addition in the future.

[peterkamatej]

Sorry for not replying sooner. So far it works fine for me when I switch to pandas, use their gaussian rolling window, and then switch back to xarray. As I'm in a hurry with something else now, I will get back to this discussion a bit later.

[serazing]

Hi, I have taken another approach for using nd window over several dimensions of xarray objects to perform filtering and tapering, based on scipy.ndimage, scipy.signal and dask.map_overlap. @shoyer @jhamman it is somewhat similar to what I have presented during the aospy meeting. It also refers to the issue #819.

For the moment, I have something that works like this :

shape = (50, 30, 40)
dims = ('x', 'y', 'z')
dummy_array = xr.DataArray(np.random.random(shape), dims=dims)
# Define and set a window object
w = dummy_array.window
w.set(n={'x':24, 'y':24}, cutoff={'x':0.01, 'y':0.01}, window='hanning')

where n is the filter order (i.e. the size), cutoff is the cutoff frequency, window is any window name that can be found in the scipy.signal.windows collection.

Then the filtering can be perform using the w.convolve() method, which build a dask graph for the convolution product.

I also want to add a tapering method 'w.taper()' which would be useful for spectral analysis. For multi-tapering, it should also generate an object with an additional dimension corresponding to the number of windows. To do that, I first need to handle the window building using dask.

Let me know if you are interesting in this approach. For the moment, I have planned to upload a github project for signal processing tools in the framework of pangeo-data. It sould be online by the end of December and I will happy to have feedback on it. I am not sure it falls into the xarray framework and it may need a dedicated project, but I might be wrong.

[stale]

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity
If this issue remains relevant, please comment here; otherwise it will be marked as closed automatically