Merge branch 'master' into yohai-ds_scatter

dcherian · dcherian · commit 84d4cbc6553c · 2018-12-19T09:54:27.000-07:00
* master: .resample now supports loffset. (pydata#2608) Remove failing Appveyor Python 2.7 32-bit build (pydata#2617) Remove meaningless tz argument in cftime_range (pydata#2613) doc fixes. (pydata#2611) Fix parsing '_Unsigned' attribute (pydata#2584) fix a few typos in rst files (pydata#2607)
diff --git a/appveyor.yml b/appveyor.yml
@@ -7,11 +7,6 @@ branches:
 environment:
 
   matrix:
-    - PYTHON: "C:\\Python27-conda32"
-      PYTHON_VERSION: "2.7"
-      PYTHON_ARCH: "32"
-      CONDA_ENV: "py27-windows"
-
     - PYTHON: "C:\\Python27-conda64"
       PYTHON_VERSION: "2.7"
       PYTHON_ARCH: "64"
diff --git a/doc/api.rst b/doc/api.rst
@@ -376,7 +376,7 @@ Universal functions
 .. warning::
 
    With recent versions of numpy, dask and xarray, NumPy ufuncs are now
-   supported directly on all xarray and dask objects. This obliviates the need
+   supported directly on all xarray and dask objects. This obviates the need
    for the ``xarray.ufuncs`` module, which should not be used for new code
    unless compatibility with versions of NumPy prior to v1.13 is required.
 
diff --git a/doc/computation.rst b/doc/computation.rst
@@ -267,7 +267,7 @@ This means, for example, that you always subtract an array from its transpose:
 
     c - c.T
 
-You can explicitly broadcast xaray data structures by using the
+You can explicitly broadcast xarray data structures by using the
 :py:func:`~xarray.broadcast` function:
 
 .. ipython:: python
diff --git a/doc/examples/monthly-means.rst b/doc/examples/monthly-means.rst
@@ -130,7 +130,7 @@ for each monthly data record - calculate weights using
 ``groupby('time.season')``
 
 Finally, we just need to multiply our weights by the ``Dataset`` and sum
-allong the time dimension.
+along the time dimension.
 
 .. code:: python
 
diff --git a/doc/examples/weather-data.rst b/doc/examples/weather-data.rst
@@ -17,7 +17,7 @@ Shared setup:
 .. ipython:: python
    :suppress:
 
-    fpath = "examples/_code/weather_data_setup.py"
+    fpath = "doc/examples/_code/weather_data_setup.py"
     with open(fpath) as f:
         code = compile(f.read(), fpath, 'exec')
         exec(code)
diff --git a/doc/internals.rst b/doc/internals.rst
@@ -111,7 +111,7 @@ Back in an interactive IPython session, we can use these properties:
 .. ipython:: python
    :suppress:
 
-   exec(open("examples/_code/accessor_example.py").read())
+   exec(open("doc/examples/_code/accessor_example.py").read())
 
 .. ipython:: python
 
diff --git a/doc/interpolation.rst b/doc/interpolation.rst
@@ -57,7 +57,7 @@ To interpolate data with a :py:func:`numpy.datetime64` coordinate you can pass a
     da_dt64.interp(time='2000-01-02')
 
 The interpolated data can be merged into the original :py:class:`~xarray.DataArray`
-by specifing the time periods required.
+by specifying the time periods required.
 
 .. ipython:: python
 
@@ -232,7 +232,7 @@ then make the interpolation
     dropped
     dropped.interp(x=[0.5, 1.5, 2.5], method='cubic')
 
-If NaNs are distributed rondomly in your multidimensional array,
+If NaNs are distributed randomly in your multidimensional array,
 dropping all the columns containing more than one NaNs by
 :py:meth:`~xarray.DataArray.dropna` may lose a significant amount of information.
 In such a case, you can fill NaN by :py:meth:`~xarray.DataArray.interpolate_na`,
diff --git a/doc/pandas.rst b/doc/pandas.rst
@@ -152,7 +152,7 @@ However, you will need to set dimension names explicitly, either with the
 Transitioning from pandas.Panel to xarray
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-:py:class:`~pandas.Panel`, pandas's data structure for 3D arrays, has always
+:py:class:`~pandas.Panel`, pandas' data structure for 3D arrays, has always
 been a second class data structure compared to the Series and DataFrame. To
 allow pandas developers to focus more on its core functionality built around
 the DataFrame, pandas plans to eventually deprecate Panel.
diff --git a/doc/plotting.rst b/doc/plotting.rst
@@ -481,7 +481,7 @@ about three or four dimensional arrays? That's where facets become helpful.
 Consider the temperature data set. There are 4 observations per day for two
 years which makes for 2920 values along the time dimension.
 One way to visualize this data is to make a
-seperate plot for each time period.
+separate plot for each time period.
 
 The faceted dimension should not have too many values;
 faceting on the time dimension will produce 2920 plots. That's
diff --git a/doc/roadmap.rst b/doc/roadmap.rst
@@ -186,7 +186,7 @@ Engaging more users
 
 Like many open-source projects, the documentation of xarray has grown
 together with the library's features. While we think that the xarray
-documentation is comprehensive already, we aknowledge that the adoption
+documentation is comprehensive already, we acknowledge that the adoption
 of xarray might be slowed down because of the substantial time
 investment required to learn its working principles. In particular,
 non-computer scientists or users less familiar with the pydata ecosystem
@@ -198,7 +198,7 @@ In order to lower this adoption barrier, we propose to:
 -  Develop entry-level tutorials for users with different backgrounds. For
    example, we would like to develop tutorials for users with or without
    previous knowledge of pandas, numpy, netCDF, etc. These tutorials may be
-   built as part of xarray's documentation or included in a seperate repository
+   built as part of xarray's documentation or included in a separate repository
    to enable interactive use (e.g. mybinder.org).
 -  Document typical user workflows in a dedicated website, following the example
    of `dask-stories
diff --git a/doc/time-series.rst b/doc/time-series.rst
@@ -82,7 +82,7 @@ Datetime indexing
 
 xarray borrows powerful indexing machinery from pandas (see :ref:`indexing`).
 
-This allows for several useful and suscinct forms of indexing, particularly for
+This allows for several useful and succinct forms of indexing, particularly for
 `datetime64` data. For example, we support indexing with strings for single
 items and with the `slice` object:
 
diff --git a/doc/whats-new.rst b/doc/whats-new.rst
@@ -47,12 +47,19 @@ Enhancements
 - Support Dask ``HighLevelGraphs`` by `Matthew Rocklin <https://matthewrocklin.com>`_.
 - Dataset plotting API! Currently only :py:meth:`Dataset.plot.scatter` is implemented.
   By `Yohai Bar Sinai <https://github.com/yohai>`_ and `Deepak Cherian <https://github.com/dcherian>`_
+- :py:meth:`DataArray.resample` and :py:meth:`Dataset.resample` now supports the
+  ``loffset`` kwarg just like Pandas.
+  By `Deepak Cherian <https://github.com/dcherian>`_
+
 
 Bug fixes
 ~~~~~~~~~
 
 - Fix h5netcdf saving scalars with filters or chunks (:issue:`2563`).
   By `Martin Raspaud <https://github.com/mraspaud>`_.
+- Fix parsing of ``_Unsigned`` attribute set by OPENDAP servers. (:issue:`2583`).
+  By `Deepak Cherian <https://github.com/dcherian>`_
+
 
 .. _whats-new.0.11.0:
 
diff --git a/xarray/backends/api.py b/xarray/backends/api.py
@@ -487,6 +487,7 @@ def open_mfdataset(paths, chunks=None, concat_dim=_CONCAT_DIM_DEFAULT,
     """Open multiple files as a single dataset.
     Requires dask to be installed. See documentation for details on dask [1].
     Attributes from the first dataset file are used for the combined dataset.
+
     Parameters
     ----------
     paths : str or sequence
diff --git a/xarray/coding/cftime_offsets.py b/xarray/coding/cftime_offsets.py
@@ -553,7 +553,7 @@ def _count_not_none(*args):
 
 
 def cftime_range(start=None, end=None, periods=None, freq='D',
-                 tz=None, normalize=False, name=None, closed=None,
+                 normalize=False, name=None, closed=None,
                  calendar='standard'):
     """Return a fixed frequency CFTimeIndex.
 
diff --git a/xarray/coding/variables.py b/xarray/coding/variables.py
@@ -249,7 +249,11 @@ class UnsignedIntegerCoder(VariableCoder):
     def encode(self, variable, name=None):
         dims, data, attrs, encoding = unpack_for_encoding(variable)
 
-        if encoding.get('_Unsigned', False):
+        # from netCDF best practices
+        # https://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html
+        #     "_Unsigned = "true" to indicate that
+        #      integer data should be treated as unsigned"
+        if encoding.get('_Unsigned', 'false') == 'true':
             pop_to(encoding, attrs, '_Unsigned')
             signed_dtype = np.dtype('i%s' % data.dtype.itemsize)
             if '_FillValue' in attrs:
@@ -266,7 +270,7 @@ def decode(self, variable, name=None):
             unsigned = pop_to(attrs, encoding, '_Unsigned')
 
             if data.dtype.kind == 'i':
-                if unsigned:
+                if unsigned == 'true':
                     unsigned_dtype = np.dtype('u%s' % data.dtype.itemsize)
                     transform = partial(np.asarray, dtype=unsigned_dtype)
                     data = lazy_elemwise_func(data, transform, unsigned_dtype)
diff --git a/xarray/core/combine.py b/xarray/core/combine.py
@@ -560,6 +560,7 @@ def auto_combine(datasets, concat_dim=_CONCAT_DIM_DEFAULT,
     ``auto_combine`` works well if you have N years of data and M data
     variables, and each combination of a distinct time period and set of data
     variables is saved its own dataset.
+
     Parameters
     ----------
     datasets : sequence of xarray.Dataset
@@ -589,6 +590,7 @@ def auto_combine(datasets, concat_dim=_CONCAT_DIM_DEFAULT,
         Details are in the documentation of concat
     coords : {'minimal', 'different', 'all' or list of str}, optional
         Details are in the documentation of conca
+
     Returns
     -------
     combined : xarray.Dataset
diff --git a/xarray/core/common.py b/xarray/core/common.py
@@ -592,7 +592,7 @@ def rolling(self, dim=None, min_periods=None, center=False, **dim_kwargs):
                                  center=center)
 
     def resample(self, indexer=None, skipna=None, closed=None, label=None,
-                 base=0, keep_attrs=None, **indexer_kwargs):
+                 base=0, keep_attrs=None, loffset=None, **indexer_kwargs):
         """Returns a Resample object for performing resampling operations.
 
         Handles both downsampling and upsampling. If any intervals contain no
@@ -612,6 +612,9 @@ def resample(self, indexer=None, skipna=None, closed=None, label=None,
             For frequencies that evenly subdivide 1 day, the "origin" of the
             aggregated intervals. For example, for '24H' frequency, base could
             range from 0 through 23.
+        loffset : timedelta or str, optional
+            Offset used to adjust the resampled time labels. Some pandas date
+            offset strings are supported.
         keep_attrs : bool, optional
             If True, the object's attributes (`attrs`) will be copied from
             the original object to the new one.  If False (default), the new
@@ -700,7 +703,9 @@ def resample(self, indexer=None, skipna=None, closed=None, label=None,
 
         group = DataArray(dim_coord, coords=dim_coord.coords,
                           dims=dim_coord.dims, name=RESAMPLE_DIM)
-        grouper = pd.Grouper(freq=freq, closed=closed, label=label, base=base)
+        # TODO: to_offset() call required for pandas==0.19.2
+        grouper = pd.Grouper(freq=freq, closed=closed, label=label, base=base,
+                             loffset=pd.tseries.frequencies.to_offset(loffset))
         resampler = self._resample_cls(self, group=group, dim=dim_name,
                                        grouper=grouper,
                                        resample_dim=RESAMPLE_DIM)
diff --git a/xarray/core/dataarray.py b/xarray/core/dataarray.py
@@ -997,8 +997,8 @@ def interp(self, coords=None, method='linear', assume_sorted=False,
         interpolated: xr.DataArray
             New dataarray on the new coordinates.
 
-        Note
-        ----
+        Notes
+        -----
         scipy is required.
 
         See Also
@@ -1053,8 +1053,8 @@ def interp_like(self, other, method='linear', assume_sorted=False,
             Another dataarray by interpolating this dataarray's data along the
             coordinates of the other object.
 
-        Note
-        ----
+        Notes
+        -----
         scipy is required.
         If the dataarray has object-type coordinates, reindex is used for these
         coordinates instead of the interpolation.
@@ -2291,13 +2291,13 @@ def quantile(self, q, dim=None, interpolation='linear', keep_attrs=None):
             use when the desired quantile lies between two data points
             ``i < j``:
 
-                * linear: ``i + (j - i) * fraction``, where ``fraction`` is
+                - linear: ``i + (j - i) * fraction``, where ``fraction`` is
                   the fractional part of the index surrounded by ``i`` and
                   ``j``.
-                * lower: ``i``.
-                * higher: ``j``.
-                * nearest: ``i`` or ``j``, whichever is nearest.
-                * midpoint: ``(i + j) / 2``.
+                - lower: ``i``.
+                - higher: ``j``.
+                - nearest: ``i`` or ``j``, whichever is nearest.
+                - midpoint: ``(i + j) / 2``.
         keep_attrs : bool, optional
             If True, the dataset's attributes (`attrs`) will be copied from
             the original object to the new one.  If False (default), the new
diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py
@@ -1946,8 +1946,8 @@ def interp(self, coords=None, method='linear', assume_sorted=False,
         interpolated: xr.Dataset
             New dataset on the new coordinates.
 
-        Note
-        ----
+        Notes
+        -----
         scipy is required.
 
         See Also
@@ -2038,8 +2038,8 @@ def interp_like(self, other, method='linear', assume_sorted=False,
             Another dataset by interpolating this dataset's data along the
             coordinates of the other object.
 
-        Note
-        ----
+        Notes
+        -----
         scipy is required.
         If the dataset has object-type coordinates, reindex is used for these
         coordinates instead of the interpolation.
@@ -2549,7 +2549,6 @@ def merge(self, other, inplace=None, overwrite_vars=frozenset(),
                   'no_conflicts'}, optional
             String indicating how to compare variables of the same name for
             potential conflicts:
-
             - 'broadcast_equals': all values must be equal when variables are
               broadcast against each other to ensure common dimensions.
             - 'equals': all values and dimensions must be the same.
diff --git a/xarray/core/groupby.py b/xarray/core/groupby.py
@@ -3,6 +3,7 @@
 import functools
 import warnings
 
+import datetime
 import numpy as np
 import pandas as pd
 
@@ -154,6 +155,32 @@ def _unique_and_monotonic(group):
         return index.is_unique and index.is_monotonic
 
 
+def _apply_loffset(grouper, result):
+    """
+    (copied from pandas)
+    if loffset is set, offset the result index
+
+    This is NOT an idempotent routine, it will be applied
+    exactly once to the result.
+
+    Parameters
+    ----------
+    result : Series or DataFrame
+        the result of resample
+    """
+
+    needs_offset = (
+        isinstance(grouper.loffset, (pd.DateOffset, datetime.timedelta))
+        and isinstance(result.index, pd.DatetimeIndex)
+        and len(result.index) > 0
+    )
+
+    if needs_offset:
+        result.index = result.index + grouper.loffset
+
+    grouper.loffset = None
+
+
 class GroupBy(SupportsArithmetic):
     """A object that implements the split-apply-combine pattern.
 
@@ -235,6 +262,7 @@ def __init__(self, obj, group, squeeze=False, grouper=None, bins=None,
                 raise ValueError('index must be monotonic for resampling')
             s = pd.Series(np.arange(index.size), index)
             first_items = s.groupby(grouper).first()
+            _apply_loffset(grouper, first_items)
             full_index = first_items.index
             if first_items.isnull().any():
                 first_items = first_items.dropna()
diff --git a/xarray/tests/test_backends.py b/xarray/tests/test_backends.py
diff --git a/xarray/tests/test_dataarray.py b/xarray/tests/test_dataarray.py
diff --git a/xarray/tests/test_dataset.py b/xarray/tests/test_dataset.py
