Skip to content

gh-95245: Document Py_TPFLAGS_PREHEADER #135861

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Aug 25, 2025

Conversation

sergey-miryanov
Copy link
Contributor

@sergey-miryanov sergey-miryanov commented Jun 23, 2025

I see following usages of Py_TPFLAGS_PREHEADER:

  1. Calculate size of preheader (but preheader also contains GC info)
  2. Check if types are support managed dict or managed weakref

I checked all linked PR from #95245 and can't find any explanation. I also checked https://github.com/python/cpython/blob/main/Objects/object_layout.md.

@encukou Please take a look.

@bedevere-app bedevere-app bot mentioned this pull request Jun 23, 2025
39 tasks
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should include documentation on the type object page too.

@encukou encukou changed the title gh-135755: Document Py_TPFLAGS_PREHEADER gh-95245: Document Py_TPFLAGS_PREHEADER Jun 24, 2025
@sergey-miryanov
Copy link
Contributor Author

@ZeroIntensity Thanks for review! Fixed (I hope :)

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if this is internal-only, it would be better to completely drop it from public headers rather than document it as such.

@ZeroIntensity ZeroIntensity added skip news docs Documentation in the Doc dir labels Jun 24, 2025
@github-project-automation github-project-automation bot moved this to Todo in Docs PRs Jun 24, 2025
@ZeroIntensity
Copy link
Member

I think that if this is internal-only, it would be better to completely drop it from public headers rather than document it as such.

(Don't do this yet, by the way. Let's come to a consensus in the issue.)

@sergey-miryanov
Copy link
Contributor Author

As no comments from @iritkatriel or @markshannon I propose to make it internal as @ZeroIntensity suggested and rename it to _Py_TPFLAGS_PREHEADER.

@encukou WDYT?

@encukou
Copy link
Member

encukou commented Jul 7, 2025

Per PEP-387, removing (or renaming) it probablyneeds a deprecation cycle. Or is there evidence that no existing code uses it?

@markshannon
Copy link
Member

Since the tp_flags flags must be public, it seemed sensible to put Py_TPFLAGS_PREHEADER in the same header.
I have no objection to moving it.

It might be better to rename it to Py_TPFLAGS_VM_MANAGED_FIELDS to focus on the semantics, not the layout, and keep it public?

@sergey-miryanov
Copy link
Contributor Author

sergey-miryanov commented Jul 7, 2025

Per PEP-387, removing (or renaming) it probablyneeds a deprecation cycle. Or is there evidence that no existing code uses it?

https://grep.app/search?q=Py_TPFLAGS_PREHEADER shows close to nothing of usage. Also, I'm downloading top-pypi packages to checks usage (as described here https://hugovk.dev/blog/2022/how-to-search-5000-python-projects/), will report as it finished.

@sergey-miryanov
Copy link
Contributor Author

It might be better to rename it to Py_TPFLAGS_VM_MANAGED_FIELDS to focus on the semantics, not the layout, and keep it public?

@markshannon WDYT about wordings in this PR? Can you suggest some improvements?

@sergey-miryanov
Copy link
Contributor Author

sergey-miryanov commented Jul 7, 2025

will report as it finished.

Searched over 13k+ packages:

➜ .\python.exe ../misc/cpython/search_pypi_top.py -q ..\top-pypi\output\  "Py_TPFLAGS_PREHEADER"
..\top-pypi\output\recordclass-0.23.1.tar.gz: recordclass-0.23.1/lib/recordclass/_dataobject.c: tp->tp_flags &= ~Py_TPFLAGS_PREHEADER;

Time: 0:01:10.586519
Found 1 matching lines in 1 projects

➜ ( Get-ChildItem ..\top-pypi\output\ | Measure-Object ).Count;
13961

@encukou
Copy link
Member

encukou commented Jul 7, 2025

It might be better to rename it to Py_TPFLAGS_VM_MANAGED_FIELDS to focus on the semantics, not the layout, and keep it public?

If it's public, it's best not to rename it.

@ZeroIntensity
Copy link
Member

If it's public, it's best not to rename it.

I think it's probably safe to rename it, at least for 3.14. Based on a code search, we're the only ones who use it (all the non-CPython mentions are people copying our headers).

@sergey-miryanov
Copy link
Contributor Author

sergey-miryanov commented Jul 7, 2025

If it's public, it's best not to rename it.

I think it's probably safe to rename it, at least for 3.14. Based on a code search, we're the only ones who use it (all the non-CPython mentions are people copying our headers).

Only recordclass uses it - #135861 (comment)

@sergey-miryanov
Copy link
Contributor Author

@ZeroIntensity Please take a look.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this mostly looks good.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I'm not a huge fan of documenting things that should be private, but I think it's too late to move this to internal headers or rename it. We should consider deprecating this in 3.15.

@encukou Would you like to finish up your review?

@encukou
Copy link
Member

encukou commented Aug 25, 2025

It's hard to deprecate a macro, so adding warnings in docs look best.
Thank you for sticking to this, @sergey-miryanov!

@encukou encukou merged commit 5baa7a0 into python:main Aug 25, 2025
48 checks passed
@github-project-automation github-project-automation bot moved this from Todo to Done in Docs PRs Aug 25, 2025
@sergey-miryanov
Copy link
Contributor Author

Thanks everyone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation in the Doc dir skip news topic-C-API
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants