Skip to content

BUG: resolve undefined behavior issues currently suppressed by UBSAN CI #29524

@m-clare

Description

@m-clare

This is a tracking issue for resolving the UndefinedBehaviorSanitizer identified issues in numpy. This tracking issue will be closed when the files/modules currently being suppressed for testing with UBSAN have been fixed. Then the suppression for each file/module can be removed from tools/ci/ubsan_suppressions_x86_64.txt or tools/ci/ubsan_suppressions_arm64.txt and the compiler_sanitizers.yml workflow will still run successfully in CI. Some suggested potential fixes are documented in the suppressions files, and repeated here.

To determine where the issue is in the underlying file, you should build numpy with:

python -m spin build -- -Db_sanitize=undefined -Db_lundef=false

and run tests with:

UBSAN_OPTIONS=halt_on_error=1:suppressions=/tools/ci/ubsan_suppressions_x86_64.txt \
spin test -- -v -s --timeout=600 --durations=10

and remove the directive line from the suppressions file while testing.

arm64

  • shift-base:_core/src/umath/_rational_tests.c - use INT_MIN constant
  • shift-base:_core/src/npymath/npy_math_internal.h - check for overflow if signed

add null check before loop:

  • pointer-overflow:_core/src/common/simd/neon/memory.h

  • pointer-overflow:_core/src/multiarray/datetime_busdaycal.c

  • pointer-overflow:_core/src/multiarray/nditer_templ.c

  • pointer-overflow:_core/src/multiarray/nditer_constr.c

  • pointer-overflow:_core/src/umath/loops_arithm_fp.dispatch.c.src

  • pointer-overflow:_core/src/umath/loops_unary.dispatch.c.src

  • pointer-overflow:_core/src/umath/loops_unary_complex.dispatch.c.src

  • pointer-overflow:_core/src/umath/loops_unary_fp_le.dispatch.c.src

  • pointer-overflow:_core/src/umath/string_buffer.h

  • pointer-overflow:linalg/umath_linalg.cpp

  • pointer-overflow:numpy/random/bit_generator.pyx.c

  • float-cast-overflow:_core/src/multiarray/lowlevel_strided_loops.c.src

_core/src/ and numpy/random/ had many modules with call to function through pointer to incorrect function type errors in UBSAN. Some are documented here, but more would need to be identified to disable the general suppressions on function:_core/src/* and function:numpy/random/*

  • function:_core/src/common/cblasfunc.c
  • function:_core/src/common/npy_argparse.c
  • function:_core/src/multiarray/number.c
  • function:_core/src/multiarray/ctors.c
  • function:_core/src/multiarray/convert_datatype.c
  • function:_core/src/multiarray/dtype_transfer.c
  • function:_core/src/multiarray/dtype_traversal.c
  • function:_core/src/multiarray/getset.c
  • function:_core/src/multiarray/scalarapi.c
  • function:_core/src/multiarray/scalartypes.c.src

x86-64

replace left bit shift with LLONG_MIN constant:

  • shift-base:_core/src/common/simd/sse/arithmetic.h
  • shift-base:_core/src/common/simd/avx2/arithmetic.h

use INT_MIN constant:

  • shift-base:_core/src/umath/_rational_tests.c

check for overflow if signed:

  • shift-base:_core/src/npymath/npy_math_internal.h

check that pointer is not null before calling function:

  • nonnull-attribute:_core/src/multiarray/array_coercion.c
  • nonnull-attribute:_core/src/multiarray/ctors.c
  • nonnull-attribute:_core/src/multiarray/datetime_busdaycal.c
  • nonnull-attribute:_core/src/multiarray/scalarapi.c
  • nonnull-attribute:_core/src/multiarray/calculation.c

null check before loop:

  • pointer-overflow:_core/src/multiarray/nditer_templ.c

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions