ARROW-3953: [Python] Compat with pandas 0.24 rename of MultiIndex labels -> codes

[jorisvandenbossche]

No description provided.

[jorisvandenbossche]

I am not fully sure if I should add a test for this. The result of this patch is that there should be no warnings about the deprecation, so I could add an assert that there are no warnings in one of the tests?

Is pandas master tested only in a nightly test build? Or also in travis?

[codecov-io]

Codecov Report

Merging #3120 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3120      +/-   ##
==========================================
- Coverage   87.09%   87.07%   -0.02%     
==========================================
  Files         492      492              
  Lines       69160    69164       +4     
==========================================
- Hits        60233    60226       -7     
- Misses       8830     8837       +7     
- Partials       97      101       +4

Impacted Files	Coverage Δ
python/pyarrow/pandas_compat.py	98.01% <100%> (+0.02%)	⬆️
go/arrow/math/int64_avx2_amd64.go	0% <0%> (-100%)	⬇️
go/arrow/memory/memory_avx2_amd64.go	0% <0%> (-100%)	⬇️
go/arrow/math/float64_avx2_amd64.go	0% <0%> (-100%)	⬇️
go/arrow/math/uint64_avx2_amd64.go	0% <0%> (-100%)	⬇️
go/arrow/memory/memory_amd64.go	28.57% <0%> (-14.29%)	⬇️
go/arrow/math/math_amd64.go	31.57% <0%> (-5.27%)	⬇️
cpp/src/plasma/thirdparty/ae/ae.c	72.03% <0%> (-0.95%)	⬇️
go/arrow/math/float64_amd64.go	33.33% <0%> (ø)	⬆️
go/arrow/math/int64_amd64.go	33.33% <0%> (ø)	⬆️
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 072df89...329f3e4. Read the comment docs.

[wesm]

I think we have a docker-compose target for testing against pandas master. cc @kszucs

[xhochy]

These instructions for the pandas tests can be found at

arrow/docker-compose.yml

Lines 246 to 252 in 5704d8d

	pandas-master:
	# Usage:
	# export PYTHON_VERSION=3.6
	# docker-compose build cpp
	# docker-compose build python
	# docker-compose build --no-cache pandas-master
	# docker-compose run pandas-master

[kszucs]

@jorisvandenbossche a test case would be nice (BTW the warning indeed disappears, tested with the docker-compose snippet above)

[jorisvandenbossche]

Are the docker-compose / crossbow tests also run somewhere on CI, or are you supposed to run them locally? (if I add a test, it could only fail on pandas master, so it wouldn't be catched on travis I think?)

[kszucs]

@jorisvandenbossche You can make the test case dependent on pandas' version

[pitrou]

I'm not sure it's worth writing a test for this, TBH.

[jorisvandenbossche]

Sorry, I will be mainly on holidays the comings weeks, so won't be able to add a test until after (I had some problems compiling yesterday I first need to solve).

A test would be something like adding the following assert to one of the existing tests:

with pytest.warns(None) as record:
    # existing conversion from arrow table to pandas
    ...
# assert pandas raises no FutureWarning about codes being deprecated
assert len([w for w in record if w.category is FutureWarning and 'labels' in str(w.message)]) == 0

I think it can be nice to add it to make sure we don't introduce a usage of labels again, but it's certainly not essential, because the fact that tests are passing both on released pandas and pandas master is the actual test for the functionality.

[xhochy]

I would also be happy merging this without a test, @kszucs do you feel strongly about this and could add one?

[kszucs]

Not necessarily, but if order is the soul of everything then testing is the soul of engineering

[kszucs]

Thanks @jorisvandenbossche

[jorisvandenbossche]

@kszucs Thanks for adding the test!

[kszucs]

You're welcome :)