`poetry sync -E cpu` downloads multiple gigabytes of data

[dhdaines]

Thanks for this repository! Unfortunately it doesn't resolve the eternal, horrible, no good problem with Poetry and PyTorch, where it insists on downloading every single torch wheel ever published:

$ poetry --version
Poetry (version 2.3.4)
$ poetry sync -E  cpu
Updating dependencies
Resolving dependencies... Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.11.0%2Bcpu-cp310-cp310-linux_s390x.whl ...

The solution is obviously to never use poetry for anything, ever, but sadly, it lives on in a number upstream packages including, crucially, llama.cpp. Do you know if there is any way around this issue? Poetry's insistence that it's PyTorch's fault and nothing can be done rings very, very hollow since uv (and hatch, and pdm, and pip, and ...) seems to have found a way around the problem.

But probably one should just never use poetry for anything, ever.

[lucaspar]

right, I've since switched to uv for most things, and it seems they had to handcraft a parsing solution with uv due to the lack of a final standard:

Since there's no standardized mechanism for specifying these accelerators when publishing or installing, PyTorch encodes them in the local version specifier.

https://docs.astral.sh/uv/guides/integration/pytorch/

So unless poetry writes a custom pytorch handler like that, I think we're stuck until PEP 817 or 825 are final.

[dhdaines]

right, I've since switched to uv for most things, and it seems they had to handcraft a parsing solution with uv due to the lack of a final standard:

Ah! I figured it was something like that, didn't realize this was true for both --torch-backend and specifying it via [tool.uv.sources] ... anyway the solution is obviously to just use uv 😄