Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrating from pip-tools workflow to uv project #11661

Open
mario-at-intercom opened this issue Feb 20, 2025 · 6 comments
Open

Migrating from pip-tools workflow to uv project #11661

mario-at-intercom opened this issue Feb 20, 2025 · 6 comments
Labels
question Asking for clarification or support

Comments

@mario-at-intercom
Copy link

Question

Hey, i'm looking into migrating one of my projects from pip-tools to uv project.

I can't find a guidance on doing that. I'd like my versions in requirements.txt staying the same (and I have two different files for two different platforms) when i move to the uv's lockfile.

Platform

macOS, Windows

Version

uv 0.6.2

@mario-at-intercom mario-at-intercom added the question Asking for clarification or support label Feb 20, 2025
@konstin
Copy link
Member

konstin commented Feb 20, 2025

Can you describe what you tried and what didn't work for your case?

@zanieb
Copy link
Member

zanieb commented Feb 20, 2025

I don't think there's an easy way to use multiple requirements.txt files for different platforms to constrain the versions of a new uv lockfile.

@mario-at-intercom
Copy link
Author

@konstin I haven't tried any approach yet because I have no ideas to try.

My mental model:

  • I have to create a brand new project with uv
  • Have to move all dependencies from requirements.in (no versions defined) to pyproject.toml
  • Have to make sure that uv.lock keeps versions from requirements.txt (will this happen automatically?). This is the piece I have no ideas about even to try.

@zanieb I have two requirements.txt because I need to install a different tensorflow-text package based on the platform. Platform selector (probably not the right term?!) wasn't supported back then so I had to resort to

  • requirements-common.in
  • requirements.in
  • requirements-macos.in

That compile to requirements.txt and requirements-macos.txt. If I can use platform==darwin and platform!=darwin in uv project definition and that get properly translated to single uv lockfile, I don't need multiple requirements files.

@zanieb
Copy link
Member

zanieb commented Feb 20, 2025

You'd need to copy your requirements.txt contents into tool.uv.constraint-dependencies (https://docs.astral.sh/uv/reference/settings/#constraint-dependencies), presumably you'd also want to add the platform markers relevant for the requirements file.

Then, you can uv add -r requirements.in and the lockfile should respect the pinned versions from your compiled requirements files.

If I can use platform==darwin and platform!=darwin in uv project definition and that get properly translated to single uv lockfile,

Yes you can do this with markers, e.g., https://docs.astral.sh/uv/concepts/projects/dependencies/#platform-specific-dependencies

@konstin
Copy link
Member

konstin commented Feb 20, 2025

(zanie wrote their response at the same time, sorry for the overlap)

My mental model:

* I have to create a brand new project with uv

* Have to move all dependencies from requirements.in (no versions defined) to pyproject.toml

* Have to make sure that uv.lock keeps versions from requirements.txt (will this happen automatically?). This is the piece I have no ideas about even to try.

The first two steps are correct, the third one is tricky. There's no 1:1 translation from locked requirement.txt to uv.lock due to the slightly different models (mainly that pip is platform specific and uv's lockfile is universal), but there's some tricks that can help you get close.

One option is using constraints. Constraints use the same syntax as requirements and are passed explicitly, for the uv project interface in tool.uv:

[tool.uv]
constraint-dependencies = ["...", "..."]

You could add ; sys_platform == "darwin" to each entry in the one requirements file and ; sys_platform != "darwin" to the other, then insert them as constraints here. This is the most accurate approach but it my fail with a conflict due to uv's resolver being different from pip's, requiring some manual editing until it works. Once you have your uv.lock, you can remove the constraints, uv updates uv.lock only when required or requested.

Another is doing the first with --exclude-newer and the timestamp of the pip freezing, if that is available, then remove the exclude-newer line from the resulting uv.lock. This trick only works with pypi and simulates a resolution at that time, which may be sufficient to get the right lockfile.

@mario-at-intercom
Copy link
Author

Thanks @zanieb and @konstin. I think I understand what I need to do to migrate us. I don't know when exactly we'll do that but I needed to understand the complexity we might hit. Thanks for that.

It's probably safe to close the issue now since the path to migration is outlined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Asking for clarification or support
Projects
None yet
Development

No branches or pull requests

3 participants