Skip to content

free-threading: PyImport_AddModuleRef race condition #137422

@crusaderky

Description

@crusaderky

Bug report

Bug description:

On a NoGIL interpreter, if two threads call PyImport_AddModuleRef for the same module name at the same time, and that module does not exist yet, it is possible that the two calls to the function return two different objects, only one of which is stored in sys.modules.

The race condition is actually in the helper function import_add_module:

cpython/Python/import.c

Lines 319 to 335 in a50822f

PyObject *m;
if (PyMapping_GetOptionalItem(modules, name, &m) < 0) {
return NULL;
}
if (m != NULL && PyModule_Check(m)) {
return m;
}
Py_XDECREF(m);
m = PyModule_NewObject(name);
if (m == NULL)
return NULL;
if (PyObject_SetItem(modules, name, m) != 0) {
Py_DECREF(m);
return NULL;
}
return m;

if both threads call PyMapping_GetOptionalItem before either has called PyObject_SetItem, you will get two different modules.

Context

Each SWIG module calls PyImport_AddModuleRef to retrieve or initialise a shared global state module with the hardcoded name swig_runtime_data5, and then stores its pointer in a module-local global variable.

SWIG is actually immune from this race condition because the call to PyImport_AddModuleRef always happens inside SWIG_init, which is an alias for PyInit_<modulename> which holds the GIL.

This race condition only affects lazy initialisation patterns, where PyImport_AddModuleRef is executed for the first time after module init.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions