-
-
Notifications
You must be signed in to change notification settings - Fork 11.2k
Description
Describe the issue:
When using the Python C API, it is possible to close a Python interpreter and then restart it fresh. However, attempting to import numpy when doing this causes an unavoidable RuntimeError. It seems that something is not being properly cleaned up on interpreter exit, resulting in a new interpreter retaining the old state.
Also note there is a typo in the RuntimeError message
Reproduce the code example:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
// clang bug.c `python3-config --embed --cflags` `python3-config --embed --ldflags`
void import_numpy() {
Py_Initialize();
PyObject* numpy = PyImport_ImportModule("numpy");
if (!numpy) {
PyErr_Print();
}
Py_CLEAR(numpy);
Py_Finalize();
}
int main() {
import_numpy(); // this works
import_numpy(); // this hits the exception
return 0;
}
Error message:
Traceback (most recent call last):
File ".venv/lib/python3.12/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show_config
File ".venv/lib/python3.12/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File ".venv/lib/python3.12/site-packages/numpy/_core/__init__.py", line 23, in <module>
from . import multiarray
File ".venv/lib/python3.12/site-packages/numpy/_core/multiarray.py", line 10, in <module>
from . import overrides
File ".venv/lib/python3.12/site-packages/numpy/_core/overrides.py", line 7, in <module>
from numpy._core._multiarray_umath import (
RuntimeError: CPU dispatcher tracer already initlized
Python and NumPy Versions:
numpy 2.2.1
pythom 3.12.7
Runtime Environment:
[{'numpy_version': '2.2.1',
'python': '3.12.7 (main, Oct 1 2024, 02:05:46) [Clang 15.0.0 '
'(clang-1500.1.0.2.5)]',
'uname': uname_result(system='Darwin', node='<>', release='22.6.0', version='Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000', machine='arm64')},
{'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
'found': ['ASIMDHP'],
'not_found': ['ASIMDFHM']}},
{'architecture': 'neoversen1',
'filepath': '.venv/lib/python3.12/site-packages/numpy/.dylibs/libscipy_openblas64_.dylib',
'internal_api': 'openblas',
'num_threads': 10,
'prefix': 'libscipy_openblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.28'}]
Context for the issue:
The Chapel language team is working on Python interop support in general and creating multiple interpreters that interop with numpy is essential. This also prevents us from using other Python modules that rely on numpy. For example, replacing 'numpy' with 'pandas' in the code above hits the exact same error, because pandas uses numpy.