Skip to content

Unexpected behaviour of IntFlag with a custom __new__ in Python 3.11.0.  #101541

@picnixz

Description

@picnixz

Bug report

According to the enum documentation, it is possible to customize the enumeration value via a custom __new__ method and the enumeration member (e.g., by adding an attribute) via a custom __init__ method. However, the implementation of the enum.Flag class in 3.11.0 (and probably in 3.11.1) introduces some issues compared to the one in 3.10.3, especially in the case of an enum.IntFlag:

$ read -r -d '' code << EOM
from enum import IntFlag

class Flag(IntFlag):
    def __new__(cls, ch: str, *args):
        value = 1 << ord(ch)
        self = int.__new__(cls, value)
        self._value_ = value
        return self

    def __init__(self, _, *args):
        super().__init__()
        # do something with the positional arguments

    a = ('a', 'A')

print(repr(Flag.a ^ Flag.a))
EOM

$ python3.10 -c "$code"
<Flag.0: 0>

$ python3.11 -c "$code"
ValueError: 0 is not a valid Flag

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 16, in <module>
  File "/Lib/enum.py", line 1501, in __xor__
    return self.__class__(value ^ other)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/enum.py", line 695, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/enum.py", line 1119, in __new__
    raise exc
  File "/Lib/enum.py", line 1096, in __new__
    result = cls._missing_(value)
             ^^^^^^^^^^^^^^^^^^^^
  File "/Lib/enum.py", line 1416, in _missing_
    pseudo_member = (__new__ or cls._member_type_.__new__)(cls, value)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 5, in __new__
TypeError: ord() expected string of length 1, but int found

This also results in the impossibility of writing Flag.a | i for i != 0 (for i = 0, it does work ! and this is confusing !), which IMHO is a regression compared to what was proposed in 3.10.3. It also clashes with the following assumption:

If a Flag operation is performed with an IntFlag member and:

  • the result is a valid IntFlag: an IntFlag is returned
  • result is not a valid IntFlag: the result depends on the FlagBoundary setting

Currently, the FlagBoundary for IntFlag is KEEP, so Flag.a | 12 is expected to be Flag.a|8|4 as in 3.10.3.

In order to avoid this issue, users need to write something like:

def __new__(cls, ch, *args):
    value = ch if isinstance(ch, int) else 1 << ord(ch)
    self = int.__new__(cls, value)
    self._value_ = value
    return self

Neverthless, this is only possible if __new__ converts an input U to an entirely different type V (enum member type) or if args is non-empty when declaring enumeration members. However, this fails in the following example:

class FlagFromChar(IntFlag):
    def __new__(cls, e: int):
        value = 1 << e
        self = int.__new__(cls, value)
        self._value_ = value
        return self

    a = ord('a')

# in Python 3.10.3
repr(FlagFromChar.a ^ FlagFromChar.a) == '<FlagFromChar.0: 0>'

# in Python 3.11.1
repr(FlagFromChar.a ^ FlagFromChar.a) == '<FlagFromChar: 1>'

Environment

  • CPython versions tested on: 3.10.3 and 3.11.0 (compiled from sources with GCC 7.5.0)
  • Operating System: openSUSE Leap 15.2 x86_64
  • Kernel: 5.3.18-lp152.106-default

Linked PRs

Metadata

Metadata

Assignees

Labels

3.11only security fixes3.12only security fixesstdlibPython modules in the Lib dirtype-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions