BUG: undefined behavior detected by ubsan in sanitizer CI runs #24209
Description
Once all these issues are fixed the CI job running with the sanitizers turned on can have UBSAN_OPTIONS=halt_on_error=1 in its spin test invocation so that new UB doesn't get introduced in the future.
NULLarray shape passed to memcpy
numpy/array_api/tests/test_creation_functions.py::test_asarray_copy
../numpy/core/src/multiarray/array_coercion.c:1047:13: runtime error: null pointer passed as argument 2, which is declared to never be null
Happening because memcpy is receiving a NULL second argument, ultimately because the array coercion machinery is getting passed an incompletely initialized array object created internally inside numpy as part of ufunc reduction.
There's another similar issue with subarrays:
numpy/core/tests/test_arrayprint.py::TestArray2String::test_structure_format_mixed
../numpy/core/src/multiarray/ctors.c:674:13: runtime error: null pointer passed as argument 2, which is declared to never be null
Another one in TestBool:
numpy/core/tests/test_multiarray.py::TestBool::test_cast_from_void
../numpy/core/src/multiarray/scalarapi.c:276:9: runtime error: null pointer passed as argument 2, which is declared to never be null
This one seems to be from not doing error handling for the scalar_value function.
NULLpassed to qsort
numpy/core/tests/test_datetime.py::TestDateTime::test_datetime_busday_offset
../numpy/core/src/multiarray/datetime_busdaycal.c:234:5: runtime error: null pointer passed as argument 1, which is declared to never be null
- Misaligned cast in boolean indexing
numpy/array_api/tests/test_set_functions.py::test_inverse_indices_shape[unique_all]
../numpy/core/src/multiarray/common.h:288:31: runtime error: load of misaligned address 0x6080000c75a2 for type 'unsigned int', which requires 4 byte alignment
0x6080000c75a2: note: pointer points here
00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^
../numpy/core/src/multiarray/common.h:288:31: runtime error: load of misaligned address 0x6080000c75a1 for type 'unsigned int', which requires 4 byte alignment
0x6080000c75a1: note: pointer points here
00 00 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^
numpy/core/tests/test_api.py::test_copyto_fromscalar
../numpy/core/src/multiarray/common.h:288:31: runtime error: load of misaligned address 0x60200000c6f2 for type 'unsigned int', which requires 4 byte alignment
0x60200000c6f2: note: pointer points here
00 00 00 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 02 11 00 00 10 00 00 00 14 00 80 5b 00 00
^
numpy/core/tests/test_mem_overlap.py::TestUFunc::test_unary_ufunc_1d_manual
../numpy/core/src/multiarray/common.h:288:31: runtime error: load of misaligned address 0x6190020f597f for type 'unsigned int', which requires 4 byte alignment
0x6190020f597f: note: pointer points here
fc fd fe ff 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b
^
The first report is coming from a Python line doing something like:
np.array([False])[np.array([True])]
Somehow this creates a C iterator in numpy's boolean indexing internals that has a stride of 1, ultimately leading to a cast from uninitialized data (I think). I can't reproduce this outside the array API tests so it's something a bit more complicated than just the snippet above.
- Invalid bit shift
numpy/lib/tests/test_mixins.py::TestNDArrayOperatorsMixin::test_forward_binary_methods
../numpy/core/src/npymath/npy_math_internal.h.src:657:18: runtime error: left shift of negative value -1
numpy/core/tests/test_ufunc.py::TestUfunc::test_ufunc_at_basic[a1]
../numpy/core/src/umath/_rational_tests.c:54:24: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
- Int overflows in datetime casts
Quite a few of these, just an example:
numpy/core/tests/test_casting_unittests.py::TestCasting::test_time_to_time[M8[ms]-M8[ns]-Casting.safe-None-1000000-1]
../numpy/core/src/umath/loops_autovec.dispatch.c.src:76:9: runtime error: signed integer overflow: -9223372036854775808 * 1000000 cannot be represented in type 'long int'
../numpy/core/src/multiarray/dtype_transfer.c:865:25: runtime error: signed integer overflow: 9223372036854775807 * 1000000 cannot be represented in type 'long int'
numpy/core/tests/test_casting_unittests.py::TestCasting::test_time_to_time[M8[4D]-M8[1M]-Casting.same_kind-None-None-denom8]
../numpy/core/src/multiarray/datetime.c:478:8: runtime error: signed integer overflow: 4 * 9223372036854775807 cannot be represented in type 'long int'
- Int overflow in einsum tests
numpy/core/tests/test_einsum.py::TestEinsum::test_einsum_broadcast
../numpy/core/src/multiarray/einsum_sumprod.c.src:620:33: runtime error: signed integer overflow: 9223365439786057728 + 13194139533312 cannot be represented in type 'long int'
../numpy/core/src/multiarray/arraytypes.c.src:3789:13: runtime error: signed integer overflow: 9223365439786057728 + 13194139533312 cannot be represented in type 'long int'
PASSED
- Int overflow in int128 tests
numpy/core/tests/test_extint128.py::test_safe_binop
../numpy/core/src/common/npy_extint128.h:21:14: runtime error: signed integer overflow: -9223372036854775808 + -9223372036854775808 cannot be represented in type 'long int'
../numpy/core/src/common/npy_extint128.h:35:14: runtime error: signed integer overflow: -9223372036854775808 - 9223372036854775807 cannot be represented in type 'long int'
../numpy/core/src/common/npy_extint128.h:56:14: runtime error: signed integer overflow: -9223372036854775808 * -9223372036854775808 cannot be represented in type 'long int'
- Int overflow in TestWritebackIfCopy
numpy/core/tests/test_multiarray.py::TestWritebackIfCopy::test_choose_mod_raise
../numpy/core/src/multiarray/iterators.c:1302:44: runtime error: signed integer overflow: -3617008641903833651 * 3 cannot be represented in type 'long int'
- Int overflow in nditer tests
numpy/core/tests/test_nditer.py::test_iter_too_large_with_multiindex
../numpy/core/src/multiarray/nditer_api.c:497:20: runtime error: signed integer overflow: 1152921504606846976 * 1024 cannot be represented in type 'long int'
- Int overflows in NEP 50 tests
numpy/core/tests/test_nep50_promotions.py::test_nep50_weak_integers[i]
../numpy/core/src/umath/scalarmath.c.src:62:14: runtime error: signed integer overflow: 100 + 2147483647 cannot be represented in type 'int'
../numpy/core/src/umath/loops_autovec.dispatch.c.src:76:9: runtime error: signed integer overflow: 100 + 2147483647 cannot be represented in type 'int'
numpy/core/tests/test_nep50_promotions.py::test_nep50_weak_integers[l]
../numpy/core/src/umath/scalarmath.c.src:62:14: runtime error: signed integer overflow: 100 + 9223372036854775807 cannot be represented in type 'long int'
../numpy/core/src/umath/loops_autovec.dispatch.c.src:76:9: runtime error: signed integer overflow: 100 + 9223372036854775807 cannot be represented in type 'long int'
numpy/core/tests/test_nep50_promotions.py::test_nep50_weak_integers[q]
../numpy/core/src/umath/scalarmath.c.src:62:14: runtime error: signed integer overflow: 100 + 9223372036854775807 cannot be represented in type 'long long int'
../numpy/core/src/umath/loops_autovec.dispatch.c.src:76:9: runtime error: signed integer overflow: 100 + 9223372036854775807 cannot be represented in type 'long long int'
- Int overflows in overflow tests
numpy/core/tests/test_scalarmath.py::test_scalar_integer_operation_overflow[--i]
../numpy/core/src/umath/scalarmath.c.src:71:14: runtime error: signed integer overflow: -2147483648 - 2147483647 cannot be represented in type 'int'
numpy/core/tests/test_scalarmath.py::test_scalar_integer_operation_overflow[--l]
../numpy/core/src/umath/scalarmath.c.src:71:14: runtime error: signed integer overflow: -9223372036854775808 - 9223372036854775807 cannot be represented in type 'long int'
numpy/core/tests/test_scalarmath.py::test_scalar_integer_operation_overflow[--q]
../numpy/core/src/umath/scalarmath.c.src:71:14: runtime error: signed integer overflow: -9223372036854775808 - 9223372036854775807 cannot be represented in type 'long long int'
numpy/core/tests/test_umath.py::TestRationalFunctions::test_gcd_overflow
../numpy/core/src/npymath/npy_math_internal.h.src:636:33: runtime error: negation of -9223372036854775808 cannot be represented in
numpy/core/tests/test_ufunc.py::test_ufunc_input_floatingpoint_error[0]
../numpy/core/src/umath/loops_autovec.dispatch.c.src:76:9: runtime error: signed integer overflow: -9223372036854775808 + -9223372036854775808 cannot be represented in type 'long int'
Presumably where we intentionally want overflow to happen we need to check for overflow before actually doing an overflowing operation in C.
- Int overflow in ufunc at inner loops
numpy/core/tests/test_ufunc.py::TestUfunc::test_ufunc_at_inner_loops[multiply-i]
../numpy/core/src/umath/loops.c.src:461:29: runtime error: signed integer overflow: 6058426 * 397 cannot be represented in type 'int'
../numpy/core/src/umath/loops_autovec.dispatch.c.src:76:9: runtime error: signed integer overflow: 6058426 * 397 cannot be represented in type 'int'