-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
PEP 796: Relative Virtual Environments [initial draft] #4476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Hello, before we go any further with this PR, has the idea been discussed on Discourse? And after that we'll need a sponsor before assigning the PEP number, do you have one yet? Let's unassign 796 for now. Please see: |
Thanks @hugovk and @StanFromIreland for the early review. I used a draft PR to see the CI results to further clean it up -- my apologies for wasting some of your time, but thank you regardless.
Yes: https://discuss.python.org/t/making-venvs-relocatable-friendly/96177 The criteria of "discussed enough with enough support" is vague, but what gave me enough confidence to begin a PEP and start a (draft) PR at this point was:
Thanks for clarifying that part! Yes, finding a sponsor is my next big step. |
Tip: you can enable GitHub Actions at https://github.com/rickeylev/peps/actions and run the CI on your fork.
Good luck! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks good to me. Thanks for writing this up!
Assorted more detailed comments inline.
Co-authored-by: Alyssa Coghlan <[email protected]>
Thanks @ncoghlan for sponsoring and approving! Let's continue with PEP number 796. Next steps: a PEP editor to review, then merge, and then the PEP discussion can be opened for this proposal. |
Co-authored-by: Alyssa Coghlan <[email protected]>
peps/pep-0796.rst
Outdated
benefits: | ||
|
||
* The closer the development environment is to the non-development environment, | ||
the more reliable software can be achieved, and the easier it is to reproduce |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you may want to rephrase "the more reliable software can be achieved", I find the meaning a bit confusing and it's used both here and in the next bullet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rephrased a bit. My main point is: the more similar two environments are, the less likely there will be issues due to those environmental differences. Hence its more reliable. Rephrased to more directly say it's about environment-specific differences.
still retain predictable and reliable behavior. Thus, the absolute path | ||
requirement should be relaxed and relative path behavior allowed and defined. | ||
|
||
Second, such relative paths are a building block to enable portable virtual |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It'd be nice to hear more about why and how a relative path makes the virtual environment more portable. A Python installation will still need to exist in a specific location on the new host, right? Is the pitch for a relative path that you'd copy a virtual environment and the Python installation together?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, that's the use case, although even the "copying" may be virtual - while an app embedding a single Python runtime and basing multiple virtual environments on it getting installed to different locations is one possibility, other potential scenarios would be a mounted parent folder in a container, or a mounted network drive on a host.
The key is that the position of the venv and its base runtime relative to each other is consistent, but the path prefix leading to their common parent folder may vary across systems.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a sentence to help answer this question. PTAL. Or were you thinking more like a dedicated section?
This comment has been minimized.
This comment has been minimized.
@paveldikov thank you for the comment, please could you re-post it on Discourse? The peps repo / PR discussion is mainly for editorial discussion, rather than substantive comment on the proposal itself. A |
Co-authored-by: Hugo van Kemenade <[email protected]> Co-authored-by: Zanie Blue <[email protected]>
…to relative.venvs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the reviews! I'm out from under $dayJob and vacation backlogs a bit now and addressed comments.
peps/pep-0796.rst
Outdated
benefits: | ||
|
||
* The closer the development environment is to the non-development environment, | ||
the more reliable software can be achieved, and the easier it is to reproduce |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rephrased a bit. My main point is: the more similar two environments are, the less likely there will be issues due to those environmental differences. Hence its more reliable. Rephrased to more directly say it's about environment-specific differences.
still retain predictable and reliable behavior. Thus, the absolute path | ||
requirement should be relaxed and relative path behavior allowed and defined. | ||
|
||
Second, such relative paths are a building block to enable portable virtual |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a sentence to help answer this question. PTAL. Or were you thinking more like a dedicated section?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some little notes
This PEP describes how a relative path for ``home`` in a Python virtual | ||
environment's ``pyvenv.cfg`` is understood by the Python startup process. | ||
Specifically, how it is canonicalized into an absolute path later used | ||
by the runtime. This small detail is a fundamental building block for | ||
virtual environments to be more portable. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I read this, it's not clear to me what exactly is being proposed here, it makes it seem like this is better suited for a doc.
relative paths for ``home`` in ``pyvenv.cfg``. | ||
|
||
Currently, relative paths resolve relative to the process's current working | ||
directory. Because CWD isn't knowable in advance, it makes relative paths today |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence is grammatically incorrect, and this is the only usage of the acronym which is also not explained. I suggest it is just replace with the full spelling, and corrected: "Because the current working directory isn't ..."
parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs | ||
to be checked if it's already absolute. If not, then join it to the directory | ||
name of the ``pyvenv.cfg`` file. The code already knows the directory and has | ||
helpers already present for checking if a path is absolute and joining two |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
helpers already present for checking if a path is absolute and joining two | |
helpers already exist for checking if a path is absolute and joining two |
Belatedly removed the DO-NOT-MERGE label (that was added pending the Discourse discussion and sponsorship of the PEP) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PEP is currently missing a Security Implications section, which I think should be added given that arbitrary directory traversal is permitted -- at the very least explaining why this is fine.
Several editorial notes, I think the Motivation & Rationale sections should be strengthened to focus on the benefits from relative environments, there is currently (I believe) a lot of assumed context.
The PEP also discusses at some length a broader proposal for reloacatable venvs. Is it worth considering making that the proposal here? I don't know the specifics, so it might be that the changes needed for 'relocatable' are too large to tackle in one go.
A
@@ -0,0 +1,268 @@ | |||
PEP: 796 | |||
Title: Relative Virtual Environment Home |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the title of the PEP could be clearer. At the very least I think there ought be an article or preposition between the words.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Title: Relative Virtual Environment Home | |
Title: Relative Home Path in Virtual Environments |
Potentially clearer?
Discussions-To: Pending | ||
Status: Draft | ||
Type: Standards Track | ||
Created: 26-Jun-2025 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the date a sponsor was secured, hence formally the date of number assignment
Created: 26-Jun-2025 | |
Created: 02-Jul-2025 |
This PEP describes how a relative path for ``home`` in a Python virtual | ||
environment's ``pyvenv.cfg`` is understood by the Python startup process. | ||
Specifically, how it is canonicalized into an absolute path later used | ||
by the runtime. This small detail is a fundamental building block for | ||
virtual environments to be more portable. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A PEP describes a change. This paragraph does not state what changes are being proposed compared to the status quo. Perhaps:
This PEP describes how a relative path for ``home`` in a Python virtual | |
environment's ``pyvenv.cfg`` is understood by the Python startup process. | |
Specifically, how it is canonicalized into an absolute path later used | |
by the runtime. This small detail is a fundamental building block for | |
virtual environments to be more portable. | |
This PEP formally specifies the use of a relative path for ``home`` | |
in a Python virtual environment's :file:`pyvenv.cfg` file. | |
Specifically, we will discuss how such relative paths are understood | |
by the Python startup process, including their conversion to absolute | |
paths for use by the runtime. | |
This is a fundamental building block for virtual environments to | |
become more portable. |
It can also be a useful thing to start sentences on new lines, to reduce future diff sizes.
First, it is currently prescribed that the ``home`` value in ``pyvenv.cfg`` be | ||
an absolute path. The behavior of relative paths is unspecified. While |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where? Please provide a cross-reference to the Python documentation or packaging standards
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not documented anywhere, it just doesn't work if you try to do it. I guess we could link to the bug report that precipitated the PEP: python/cpython#135773
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe a link to the source on gh?
Motivation | ||
========== | ||
|
||
There are two main motivations for allowing relative paths in ``pyvenv.cfg``. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence doesn't add much. It's also somewhat confused by the next para which says that the reason they are wanted is because they're prohibited.
I would first (briefly) explain to the reader what the home
field in pyvenv.cfg
is for, then go on to discuss the benefits of relative paths. Assume the reader is technically competent, but doesn't have all the context you do of the history here.
Relative paths are relative to the directory containing ``pyvenv.cfg``. During | ||
interpreter startup (i.e. ``getpath.py``), the relative path is joined to the | ||
directory to form an absolute path. Up-references (``../``) and current |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Relative paths are relative to the directory containing ``pyvenv.cfg``. During | |
interpreter startup (i.e. ``getpath.py``), the relative path is joined to the | |
directory to form an absolute path. Up-references (``../``) and current | |
Relative paths are relative to the directory containing :file:`pyvenv.cfg`. | |
During interpreter startup (i.e. :file:`getpath.py`), the relative path is joined to this | |
directory to form an absolute path. | |
Parent-directory references (``../``) and current |
or
Relative paths are relative to the directory containing ``pyvenv.cfg``. During | |
interpreter startup (i.e. ``getpath.py``), the relative path is joined to the | |
directory to form an absolute path. Up-references (``../``) and current | |
Relative paths are relative to the directory containing :file:`pyvenv.cfg`. | |
During interpreter startup (i.e. :file:`getpath.py`), the relative path is joined to the | |
directory containing ``pyvenv.cfg`` to form an absolute path. | |
Parent-directory references (``../``) and current |
CPython Runtime Changes | ||
======================= |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be a sub-section within Specification
CPython Runtime Changes | |
======================= | |
CPython Runtime Changes | |
----------------------- |
Backwards Compatibility | ||
======================= | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Erroneous whitespace
helpers already present for checking if a path is absolute and joining two | ||
paths. | ||
|
||
A proof-of-concept of this is implemented in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A proof-of-concept of this is implemented in | |
A proof-of-concept of this is implemented in the author's branch, |
Rejected Ideas | ||
===================== |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rejected Ideas | |
===================== | |
Rejected Ideas | |
============== |
Basic requirements (all PEP Types)
pep-NNNN.rst
), PR title (PEP 123: <Title of PEP>
) andPEP
headerAuthor
orSponsor
, and formally confirmed their approvalAuthor
,Status
(Draft
),Type
andCreated
headers filled out correctlyPEP-Delegate
,Topic
,Requires
andReplaces
headers completed if appropriate.github/CODEOWNERS
for the PEPStandards Track requirements
Python-Version
set to valid (pre-beta) future Python version, if relevantDiscussions-To
andPost-History
Work towards python/cpython#136051
📚 Documentation preview 📚: https://pep-previews--4476.org.readthedocs.build/