Package reproducibility in Python notebooks using uv isolated environments

https://marimo.io/blog/sandboxed-notebooks

by mscolnick • 2 years ago

37 14 2 years ago

Onawa2• 2 years ago

Seems like an admirable goal and the utilization of UV is great! I'm wondering if there's any possibility for integrating this sort of package management into a project like Quarto which already has a strong ecosystem of plugins and outputs from their scientific notebooks. Quarto notebooks are also markdown based and can utilize Jupyter as a back end for Notebook rendering?

twocupsofcoffee• 2 years ago

Stuff like this is so convenient and intuitive that it makes you wonder why we didn't start here in the first place!

One thing I wonder is how many data scientists will use this feature given that it is not enabled by default (which is understandable, would be messy for every notebook to have a venv), and only via command-line arguments.

I guess this is easily remedied by helpful beginner tooltip UX ("This notebook has several dependencies. Would you like to build it in a sandbox?").

mscolnick• 2 years ago

We would like to make this configurable via a setting so you don't need to pass it through the CLI every time. And I agree, maybe a tooltip/notification to nudge them in that direction.

We would also like to auto-detect a pyproject.toml, and use that when desired.

najork• 2 years ago

How've you found `uv` as a package manager? I've generally been a fan of Astral's tools in the Python ecosystem and I'm considering making the switch from `poetry`.

baggiponte4• 2 years ago

it's an absolute beast.

They need to add a couple minor painful paths that people usually use package managers with (private indexes, binding packages to third party indexes...) but if you don't need fancy corner cases, it's the best thing ever.

odie5533• 2 years ago

For any commercial company, those are not corner cases!

Onawa2• 2 years ago

It is absolutely blazingly fast and surprisingly so. Few small pain points but the speed increase makes it absolutely worth it for many cases.

WCSTombs• 2 years ago

Nice!

I haven't tried this yet, but I love that the functionality of Jupytext is also incorporated, so I guess you get to reproduce the whole end product and all dependencies just from a plain-text script that you can track in Git.

Package reproducibility in Python notebooks using uv isolated environments

Related Stories