Skip to content

Assertion failure in get_error_line_from_tokenizer_buffers in pegen_errors.c #112387

@bradlarsen

Description

@bradlarsen

Crash report

What happened?

I found some crashes when adding an additional fuzz target in #111721.

# This script fails in two different ways.
#
# First, it fails with a C assertion failure when assertions are enabled.
#
# Second, it *nondeterministically* gives the C assertion failure or an
# uncaught Python `SyntaxError` when assertions are enabled and pymalloc is
# disabled.
#
# I ran these on ARM64 macOS:
#
#     Darwin dialectic.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000 arm64
#
# For the first case, I ran this through Python built from source with
# assertions enabled:
#
#     ./configure --with-assertions --prefix "$PWD/debugbuild"
#     make -j12 altinstall
#     ./debugbuild/bin/python/crash1.py
#
# For the second case, I ran this through Python built from source with
# assertions enabled and pymalloc disabled:
#
#     ./configure --with-assertions --without-pymalloc --prefix "$PWD/debugbuild"
#     make -j12 altinstall
#     ./debugbuild/bin/python/crash1.py


# Input found via the fuzz target added in https://github.com/python/cpython/pull/111721, then manually minimized
s = b'# coding=latin\r(aaaaaaaaaaaaaaaaa\raaaaaaaaaaa\xb5'

# This line fails nondeterministically with either a C assertion failure or an
# uncaught Python `SyntaxError`, depending on the build configuration:
#
# Outcome 1:
#
#     Assertion failed: (new_line != NULL && new_line + 1 < buf_end), function get_error_line_from_tokenizer_buffers, file pegen_errors.c, line 286.
#
# Outcome 2:
#
#     Traceback (most recent call last):
#       File "crash1.py", line 17, in <module>
#         compile(s, 's', 'exec')
#       File "s", line 2
#         (aaaaaaaaaaaaaaaaa
#         ^
#     SyntaxError: '(' was never closed
compile(s, 's', 'exec')

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

Python 3.13.0a1+ (heads/main:3701f3bc10, Nov 24 2023, 23:05:42) [Clang 15.0.0 (clang-1500.0.40.1)]

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions