pytest fails to ignore warnings from netCDF4 C-level Warnings

[dpanici]

• a detailed description of the bug or problem you are having

Unsure if this is a bug or a known issue/limitation, or just something I am doing incorrectly, but I will list as a bug for now. I use pytest in the CI for my project (works very well for us), and we currently have our warnings filtered to errors so we will catch any warnings in the tests. Naturally, it is then important that we can ignore the warnings that are hard for us to avoid but we know are benign. The RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility is one such warning that often can arise but is benign in most cases . So, we want to ignore this warning. However, at least in the case of the netCDF4 package, when this warning is raised, it is NOT raised directly from python code, but rather from a python binding inside C code (this specific warning is raised in this file here around line 52657, which I re-paste here for convience:

    if (!strict && (size_t)((PyTypeObject *)result)->tp_basicsize > size) {
        PyOS_snprintf(warning, sizeof(warning),
            "%s.%s size changed, may indicate binary incompatibility", ! Line 52657
            module_name, class_name);
        #if PY_VERSION_HEX < 0x02050000
        if (PyErr_Warn(NULL, warning) < 0) goto bad;
        #else
        if (PyErr_WarnEx(NULL, warning, 0) < 0) goto bad;
        #endif

in our setup.cfg, I have it set to filter warnings to errors, but to ignore warnings from the netCDF4 package:

filterwarnings=
    error
    ignore::RuntimeWarning:netCDF4.*
    ignore::RuntimeWarning:_netCDF4.*

however, when I attempt to run pytest , I still get an error during the conftest.py when this package is loaded in:

ImportError while loading conftest '/home/dpanici/DESC/tests/conftest.py'.
tests/conftest.py:12: in <module>
    from netCDF4 import Dataset
../miniconda3/envs/desc-diffrax/lib/python3.11/site-packages/netCDF4/__init__.py:3: in <module>
    from ._netCDF4 import *
src/netCDF4/_netCDF4.pyx:1: in init netCDF4._netCDF4
    ???
E   RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

It seems to me that this means that if a warning is raised from C-level code, pytest (or warnings, I don't know exactly how you handle the warnings filtering) cannot detect it as warning from a specific module, and will therefore allow it past.

I will note that if I change my setup.cfg filterwarnings block to instead block ALL RuntimeWarnings:

filterwarnings=
    error
    ignore::RuntimeWarning

I can run fine without error, so it probably is something to do with filterwarnings being unable to realize that the warning is coming from the netCDF4 module, maybe because it is being raised in C code.

• output of pip list from the virtual environment you are using

Package                       Version
----------------------------- -------------------
alabaster                     0.7.13
aniso8601                     9.0.1
ansi2html                     1.8.0
arrow                         1.2.3
asttokens                     2.2.1
attrs                         23.1.0
Babel                         2.12.1
backcall                      0.2.0
beautifulsoup4                4.12.2
black                         22.10.0
bleach                        6.0.0
build                         0.10.0
certifi                       2023.7.22
cfgv                          3.3.1
cftime                        1.6.2
charset-normalizer            3.2.0
click                         8.1.6
colorama                      0.4.6
comm                          0.1.4
contourpy                     1.1.0
coverage                      7.2.7
cycler                        0.11.0
debugpy                       1.6.7.post1
decorator                     5.1.1
defusedxml                    0.7.1
desc-opt                      0.8.2+172.g594c254d
diffrax                       0.4.1
distlib                       0.3.7
docutils                      0.18.1
equinox                       0.10.11
eradicate                     2.3.0
executing                     1.2.0
fastjsonschema                2.18.0
filelock                      3.12.2
flake8                        6.0.0
flake8-docstrings             1.7.0
flake8-eradicate              1.5.0
flake8-isort                  6.0.0
Flask                         2.1.3
Flask-RESTful                 0.3.10
fonttools                     4.42.0
h5py                          3.9.0
identify                      2.5.26
idna                          3.4
imagesize                     1.4.1
iniconfig                     2.0.0
ipykernel                     6.25.1
ipython                       8.14.0
isort                         5.12.0
itsdangerous                  2.1.2
jax                           0.4.13
jaxlib                        0.4.13
jaxtyping                     0.2.20
jedi                          0.19.0
Jinja2                        3.1.2
jsonschema                    4.19.0
jsonschema-specifications     2023.7.1
jupyter_client                8.3.0
jupyter_core                  5.3.1
jupyterlab-pygments           0.2.2
kiwisolver                    1.4.4
MarkupSafe                    2.0.1
matplotlib                    3.6.0
matplotlib-inline             0.1.6
mccabe                        0.7.0
memory-profiler               0.61.0
mistune                       3.0.1
ml-dtypes                     0.1.0
mpmath                        1.3.0
mypy-extensions               1.0.0
nbclient                      0.6.8
nbconvert                     7.7.3
nbformat                      5.9.2
nbmake                        1.4.1
nbsphinx                      0.8.12
nest-asyncio                  1.5.7
netCDF4                       1.6.4
nodeenv                       1.8.0
numpy                         1.24.4
nvgpu                         0.10.0
opt-einsum                    3.3.0
packaging                     23.1
pandas                        2.0.3
pandocfilters                 1.5.0
parso                         0.8.3
pathspec                      0.11.2
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        10.0.0
pip                           23.2.1
platformdirs                  3.10.0
pluggy                        1.2.0
pre-commit                    3.3.3
prompt-toolkit                3.0.39
psutil                        5.9.5
ptyprocess                    0.7.0
pure-eval                     0.2.2
py-cpuinfo                    9.0.0
pycodestyle                   2.10.0
pydantic                      1.10.12
pydocstyle                    6.3.0
pyflakes                      3.0.1
Pygments                      2.16.1
pynvml                        11.5.0
pyparsing                     3.1.1
pyproject_hooks               1.0.0
pytest                        7.4.0
pytest-benchmark              4.0.0
pytest-cov                    4.1.0
pytest-monitor                1.6.6
pytest-mpl                    0.16.1
pytest-split                  0.8.1
python-dateutil               2.8.2
pytz                          2023.3
PyYAML                        6.0.1
pyzmq                         25.1.1
qicna                         0.0.1
qsc                           0.1.2
referencing                   0.30.2
requests                      2.31.0
rpds-py                       0.9.2
scipy                         1.10.1
setuptools                    68.0.0
shapely                       2.0.1
six                           1.16.0
snowballstemmer               2.2.0
soupsieve                     2.4.1
Sphinx                        6.2.1
sphinx-argparse               0.4.0
sphinx-copybutton             0.5.2
sphinx-rtd-theme              1.2.2
sphinxcontrib-applehelp       1.0.6
sphinxcontrib-devhelp         1.0.4
sphinxcontrib-htmlhelp        2.0.3
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.5
sphinxcontrib-serializinghtml 1.1.7
stack-data                    0.6.2
tabulate                      0.9.0
termcolor                     2.3.0
tinycss2                      1.2.1
tornado                       6.3.2
traitlets                     5.9.0
typeguard                     4.1.0
typing_extensions             4.7.1
tzdata                        2023.3
urllib3                       2.0.4
virtualenv                    20.24.2
wcwidth                       0.2.6
webencodings                  0.5.1
Werkzeug                      2.1.2
wheel                         0.38.4

• pytest and operating system versions
  pytest 7.4.0
  OS: Windows Subsystem for Linux, Ubuntu 20.04
• minimal example if possible
  I have not recreated this with a minimal example yet but will update once I do

[The-Compiler]

I believe pytest just passes those things on to Python. Perhaps try without the pytest setting and setting PYTHONWARNINGS=error,ignore::RuntimeWarning:_netCDF4 in your environment first (according to the Python docs, "In -W and PYTHONWARNINGS, module is a literal string that the fully qualified module name must be equal to (case-sensitively)").

If you can't get that to work either, it's probably not a pytest issue. Perhaps the hack from https://stackoverflow.com/a/53222435 will help determining the module name to use.

[dpanici]

When I try that first fix (setting the PYTHONWARNINGS environment variable and removing the filterwarnings part of the pytest setting). I get a different error related to a DeprecationWarning from pkg_resources:

DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

full traceback attached

I was unable to figure out the full module/path name, however I did eventually find another issue that ignores this warning by checking the warning message itself. This is good enough for now to solve my problem, though it would be nice to figure out how to filter based on the netCDF4 module itself instead of the warning message

The filter I ended up using is

ignore:numpy.ndarray size changed:RuntimeWarning

which is not ideal as it will ignore this from any module when I really wanted it to just be from the netCDF4 module, but I think that I am fine with this because from my earlier post link I think this warning is always benign, and when it would actually cause issues it would be raised as an error, not a RuntimeWarning.

Thanks for the help!

[The-Compiler]

Closing this then, as I believe there is nothing pytest can do here.