TST: fail test suite on ResourceWarnings

[jreback]

see comments here

should just require a setting in

pandas.util.testing.TestCase.setUpClass to set when the suite is running.

[jreback]

cc @agraboso

[agraboso]

I just tried simply adding warnings.simplefilter('error', compat.ResourceWarning) to pandas.util.testing.TestCase.setUpClass (and warnings.simplefilter('ignore', compat.ResourceWarning) to pandas.util.testing.TestCase.tearDownClass for good measure). The warnings appear in the output of the test runner, but the tests themselves do not fail.

For example, reverting this change results in pandas.io.tests.parser.common.ParserTests.test_memory_map issuing the following:

Exception ignored in: <_io.FileIO name='/Users/agraboso/Code/pandas/pandas/io/tests/parser/data/test_mmap.csv' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='/Users/agraboso/Code/pandas/pandas/io/tests/parser/data/test_mmap.csv' mode='r' encoding='UTF-8'>

The same thing happens if the responsibility of closing some handle belongs to the test code itself instead of the tested code. For example, commenting out this line gives

Exception ignored in: <_io.FileIO name=7 mode='rb+' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=7 mode='w+' encoding='UTF-8'>

in pandas.io.tests.parser.common.ParserTests.test_temporary_file.

[jreback]

hmm, if what if you do it in set_testing_mode?

[agraboso]

Same thing.

[jreback]

hmm ok

well if can think of a way to make this error would be great!

[h-vetinari]

It's July 2018, and I have unrelated ResourceWarnings appearing in one of the jobs on almost every second Travis run. Sometime they are caught and cause a direct error (see e.g. https://travis-ci.org/pandas-dev/pandas/jobs/403728466)

E           AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed file <_io.BufferedReader name='/var/folders/bb/n7t3rs157850byt_jfdcq9k80000gn/T/tmptvrq0c3stest_file.zip'>",), '/Users/travis/miniconda3/envs/pandas/lib/python3.5/site-packages/pluggy/__init__.py', 139)].

And sometimes they just fail the build without a test failure (see e.g. https://travis-ci.org/pandas-dev/pandas/jobs/403903274)

[...]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.       [100%]sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='UTF-8'>

Thought that this might be the right issue, hopefully.