[WheelVariants] Allow static metadata to override all provider plugin logic

[zanieb]

👋

@DEKHTIARJonathan and I were talking about this at PyCon, so I'm posting a thread here.

Recently, the project has gone from user-installed provider plugins, to package-defined provider plugins. I think this is a great improvement for usability — most users will not want to reason about what providers plugins they need. However, this introduces concerns about attack surface. Now, even if you're only using binary distributions, you must invoke third-party code at install time in order to select an appropriate distribution.

As a solution to this, I think the specification should allow for a manual list of variants to override all of the provider logic. If the user opts-in to providing this list, no provider plugins will be used.

The mechanism for providing this list shouldn't be a part of the specification. This will be a user-experience consideration for tooling. For example, tools could support workflows where:

• You explicitly pip install <plugin> and those are detected at install-time and used to generate the static list
• You enumerate a list of plugins in your pyproject.toml, and those are installed and queried at install time instead
• You enumerate a list of plugins in some system-level file, and invoke a tool once to write the static list to a file on boot — other tools can choose to read this file

As a nice property, something like the last case also addresses concerns about slowing down installation due to provider queries.

cc @charliermarsh @geofft @mgorny

[konstin]

You enumerate a list of plugins in some system-level file, and invoke a tool once to write the static list to a file on boot — other tools can choose to read this file

For docker, it would be great if there was a system level location, so images come pre-made with e.g. the GPU they target.

---

I hope this isn't discussed in another thread already, but it matches the same goal of not running (arbitrary) code: It should also be allowed for the packaging tool to "vendor" variant providers: I suspect that the number of popular variant provider will be small (I expect something like a generic GPU/TPU plugin and one for processor SSE extensions), so by including them in the packaging tool, we can (a) avoid the security implications and (b) solve the performance and reliably issues of subprocess calls.

For pip and other tool written in Python, they could vendor the provider and call it directly, knowing that this provider doesn't need the isolation. For uv and other rust tool, they could use Rust implementations of the provider. This is similar to PEP 517, where some tool know their own build backend and can call it directly.

Implementation-wise, tools can only use vendored providers when they match the requested provider version, and should offer an opt-out to use the real provider.

[DEKHTIARJonathan]

I was quickly chatting with @zanieb on Discord. What about a file like variant-env.lock ?
A sort of lockfile concept - but capturing the "output of plugins".

You could execute uv sync --variant-lock=variant-env.lock - the resolution would still be dynamic (handled by uv or variantlib) but there would be no plugin being called.

This a major feature we need to account for ... for so many reasons.

Any idea on how that format should look like preferably ?
Or a flat TOML file that capture the outputs of the plugins ?

Ideally I would like the file to be easy to read & edit. Humans may want to play with it (unlike pure lock files that are really built by machines for machines)

[zanieb]

I'm not sure if the file itself needs to be part of the specification. It seems easier to just standardize what static metadata looks like during resolution and allow installers to figure out the user experience around the file name and format. I feel like a future proposal could standardize a format, as we have for other things in the ecosystem. But anyway, I'm willing to talk about it.

Why not just a flat text file listing variants? What other information is needed?

[DEKHTIARJonathan]

I'm not sure if the file itself needs to be part of the specification. It seems easier to just standardize what static metadata looks like during resolution and allow installers to figure out the user experience around the file name and format. I feel like a future proposal could standardize a format, as we have for other things in the ecosystem. But anyway, I'm willing to talk about it.

My perspective is that we may end up with "optional PEPs" linked to the primary PEP.
Like extension of the "core proposal" - this would allow us to discuss in the core proposal the stuff abssssollutely necessary to make it work. And in "optional feature PEPs" - optional but nice to have standardized features/behaviors as extension of the standard

So that variant-env.lock would absolutely fall under "nice to have but optional" -making it a standard is really not necessary. Though if we can standardize it - it allows us to have an "installer independent" format - promotes compatibility and portability :)

[zanieb]

I think standardization of a lock file should be out of scope for the first wave, even as an optional PEP. In the sense that we need to understand how the specification is used before it's worth pursuing things like this.

Tell me more about your desired format anyway :D if it's trivial, I may be more into it.

[DEKHTIARJonathan]

@zanieb

A completely original design not inspired by any existing lock file format 😇

version = 1
revision = 2

[[namespace]]
name = "namespaceA"
features = [
    { featureA = "value",  featureB = "value"}
]

[[namespace]]
name = "namespaceB"
features = [
    { featureC = "value",  featureD = "value"}
]

Simple, to the point and capture the output of all plugins into a TOML file. You could use that format as a caching system if you wanted to. Super easy to read and modify by hand 🤘

That essentially allows you to "replay the resolution" but without executing any plugin.

[DEKHTIARJonathan]

I do think we should absolutely include it in the initial PEP design for a few reasons:

• Before people come up with individual ideas and formats

• This is a good way to argue th security criticism. You can use a static file if you prefer - no actual execution.

• This is going to be very important for a lot of large scale deployments for reproducible installs. Run once - deploy at wish in a similar way. Will be crucial to build containers (which won't necessarily actual have the right hardware on the builder).

[zanieb]

Why use an exploded table like that instead of just listing the strings?

I think all of those points are fine, tbh. The latter two are just arguments for static metadata, not necessarily a file format. On the first one, we want people to come up with individual formats so we can learn what users actually need then standardize what's working. The specification should just explain support for static metadata, you're stepping into the domain of user experience by specifying where that metadata comes from. People will be able to do reproducible installs regardless of whether a file format is included in the specification.

[charliermarsh]

My two cents: as a general strategy, I'd like to have good, thoughtful ideas for how things can work (e.g., a variant lockfile) but attempt to cut scope wherever we can for the PEP itself -- and, relatedly, avoid (as much as possible) specifying an interface for tools, but rather, making it possible for tools to explore different end-user designs. Those are actually two risks that I'm trying to keep in mind more broadly: that we grow scope too much, and that we're too prescriptive on tool UX.

This seems like an example of an area in which the PEP should make it possible to provide static metadata, and we should have some ideas for how tools could facilitate it, but where cutting scope could be advantageous for the rest of the PEP (and would leave room for tools to explore different approaches to exposing this to users, which could ultimately be standardized later once a winning approach is clear).

[DEKHTIARJonathan]

@charliermarsh @zanieb very reasonable.

Just to make things clear - this is not related to the pylock.toml / uv.lock. This has nothing to do with the resolver.
This discussion is about "freezing" the output of the plugins for a few reasons:

• Reproducible installs (with or without the plugins).
• Building containers / environments on machine with a different hardware/software than the install target.
• Security reasons => no actual execution, everything comes from a textfile.

That being said, the advantage of specifying a "starter standard" is that most people can start from a good place. Instead of the "difficult situation that was created with pylock.toml".

We could have a variant-env.toml that is essentially a "frozen output of the plugins" part of the initially design and:

1. Make that design evolve in X many years if we think it's insufficient
2. Put it in a dependant-PEP.

---

I don't really care too much how this format will look like ~ however I do think if we don't include in the PEP a way to "freeze the plugin output to disk" we are going to have difficulties both on DPO and when people use variants to build containers.

More on point 2) Dependant PEP - Aka Optional Feature - I was discussing with @warsaw and a few others that we may want to:

1. Publish the "core PEP" as the most minimal trimmed down version of Wheel Variant => remove anything and it collapses.
2. Publish X many dependent PEPs "Assume core-PEP is merged as currently written, this PEP argues that feature X shall also be merged" - this could allow us to discuss separately "nice to have features" without "blocking the core PEP"
   • Locking the variant resolution => Maybe the evolution of pylock.toml ?
   • Locking the output produced by the plugins (variant resolution still dynamic) => This thread
   • Format like: pip install package==1.2.3#abcd1234 ( version 1.2.3 and variant_hash abcd1234)
   • Etc.

I absolutely agree we need to be careful about scope creeping. This github discussion might very well be a good candidate for "an optional feature" of the standard.

[zanieb]

however I do think if we don't include in the PEP a way to "freeze the plugin output to disk" we are going to have difficulties both on DPO and when people use variants to build containers.

I disagree with this premise. If the specification provides a clear way get the output of plugins, and a clear way to override plugins with static metadata, then I think the storage of said static metadata is not going to be a point of contention.

Why is a flat list of features not sufficient? Why do you want to dictate a more complex format and file name? I think you need answers to these questions or there's nothing worth standardizing. We'll have a much clearer picture of these user needs later. The specification should support the simplest possible way to override with static metadata, and more complex file format for that metadata can be introduced once there are use-cases for it.

[DEKHTIARJonathan]

Could you giving an example of what you have mind ? Something like I did above. So that it's easier for me to follow what you mean by Why is a flat list of features not sufficient?

[zanieb]

As in

ns1 :: f1 :: p1
ns2 :: f2 :: p2
ns2 :: f1 :: p1

[zanieb]

(matching the format used for wheel metadata)