Virtualenv to be part of Python 3.3

http://mail.python.org/pipermail/python-dev/2012-May/119668.html

by j2labs • 14 years ago

251 58 14 years ago

mail.python.org

Awesome! I'm hard pressed to think of a third party tool that makes more sense for core-inclusion than virtualenv. Hoping the virtualenvwrapper stuff will also be included (or at least similar commands will be)

lloeki• 14 years ago

I'm personally not fond of the virtualenvwrapper porcelain: cd foo; workon foo is an extremely marginal improvement over sourcing the activate file, especially when routinely opening and closing terminal tabs and windows.

I use a number of zsh functions and hooks as porcelain to virtualenv so that it behaves like oh-my-zsh bundler support [0] (which I've shamelessly ripped of and tweaked too) i.e being inside foo directory is sufficient. This works especially well since new tabs open straight into the current tab's wd.

[0] https://github.com/robbyrussell/oh-my-zsh/tree/master/plugin...

joelhaasnoot• 14 years ago

I've put 'cd foo' in my post-activate hook, saves a command and could ofcourse be widended to do other environment setup tricks...

xnxn• 14 years ago

You might want to consider using the virtualenvwrapper.project extension. You define a "projects" dir (which I had already) and then it binds virtualenvs to projects so `workon foo` automatically `cd`s you to the right place. You can also create a new project dir and virtualenv in one shot with `mkproject`.

timtadh• 14 years ago

I don't use virtualenvwrapper either. I generally have more stuff to initialise for my projects than just the virtualenv. I use swork[1] a tool I wrote for managing your shell environment. It basically dumps all of your environment variables to a file allowing them to be restored later, then it runs a project specific initialisation script.

[1] https://github.com/timtadh/swork

mattlong• 14 years ago

Also a fan of virtualenvwrapper. workon with tab completion (in bash, at least) is so nice when you've got more than a few virtualenvs and can't quite remember your naming scheme :)

j2labs• 14 years ago

Even though I agree, virtualenvwrapper is currently a shell script. That makes me think it probably wouldn't be included, but a pythonic version of the same thing could exist and fill this gap.

kibwen2• 14 years ago

When I first tried to learn Python last year, the frustration of being a first-timer dealing with the Python 2 vs. Python 3 split very nearly turned me off of the language altogether. While I loved the language itself from first sight, Virtualenv was the tool that finally made Python a joy to use, rather than just a joy to write. This is a very good move.

Here's the link to PEP 405:

http://www.python.org/dev/peps/pep-0405/

jcurbo2• 14 years ago

So, as someone just dipping their toes into Python, what did you do regarding 2 vs 3? I have been tinkering with 2 but should I go wholly with 3? Help me before I just go back to Perl 5... :)

sp332• 14 years ago

Python 3 was on a 5-year transition. This was to let the tools develop, give time for major libraries to be ported, etc. We are now more than halfway through, so most of the important libs work fine in Py3. And the syntax is generally nicer, so you might as well start with Py3 today.

ovi256• 14 years ago

As a beginner, picking between 2 or 3 should not be one of your concerns, it's a choice that does not unlock much value either way (and it does not hurt much if you pick the 'wrong' way either).

I would say start on 2.7, as most of the popular Python libraries are still Python 2.x only, and 2.7 backports some Python 3 features, so you (and your code) will be ready for 3 whenever the world switches over :)

verisimilidude• 14 years ago

Unless you need a library that's only available in 2, I'd use 3 at this point. But virtualenv takes the pain away from having to choose, since you can just create a new "environment" with 2 or 3 depending upon your needs for a project.

If you're coming from the Ruby scene, think of virtualenv as Python's take on RVM.

SkyMarshal• 14 years ago

Zed Shaw originally strongly advised students in his Learn Python the Hard Way course to go with Python 2, because there's less opportunity for confusion (eg, if you start with 3 there's a good chance you'll come across libs you need that haven't been ported to it yet).

However, I can't find that warning anymore in the second edition, so not sure if he's changed his mind or not.

kibwen2• 14 years ago

So, as someone just dipping their toes into Python, what did you do regarding 2 vs 3?

As someone who was learning Python for fun personal projects rather than serious business projects, I decided to use Python 3 wherever possible, while familiarizing myself with Python 2.7 just in case. It turns out the two versions really aren't all that different. A lot of people will still recommend starting with Python 2.7, since it really is such a small jump from there to using Py3k; I essentially did the opposite. Both approaches work. Finding tooling for Python 3 might still be a challenge (I remember having to fix one or two bugs in IPython before it would run at all), and the usefulness of these tools to a beginner is really the biggest reason why you might want to stick with 2.7 for the moment. But if you're adventurous, I do highly recommend Py3k. Especially with the upcoming 3.3 release (due later this year), I'd argue that Python 3 finally has enough new features (and enough library support) to justify using it full-time.

(When will it be possible to know, beyond doubt, that Python 3 is ready for prime time? When third-party implementations like PyPy and Jython make the jump. I don't know about Jython (which just recently made it to 2.7), but PyPy's working on it as we speak.)

gecko• 14 years ago

Learn Python 2 first, since it's vastly more supported right now, but know that it's very easy to jump between Python 2 and Python 3, and many people do. The decision of which to use on any particular project is basically:

  1. Are you doing web development?  If so, use Python 2.
  2. If not, do you need or want to use PyPy?  If so, use Python 2.
  3. If not, are the libraries you want available for Python 3?  If so, use Python 3.
  4. Otherwise, use Python 2.

exDM69• 14 years ago

Learn Python 3, it has nice new language features and some of the hairy parts of older versions have been replaced with new and consistent constructs.

If you need to ship and launch a software that absolutely depends on libraries that only work with Python 2, it should be rather easy to not use the newer features and go with Python 2.

crusso2• 14 years ago

That's good news. Last I looked at virtualenv, though, it could have used more oomph. At the time (last Fall), I was jumping back and forth between Python virtualenv environments and Ruby rvms. RVM was a bit more of a pleasure to use. I hope the virtualenv developers have closed the usability/feature gap since then.

monkeyfacebag• 14 years ago

Can you be more specific about what you mean by "oomph"? I ask because I'm not familiar with rvms but am familiar with (and love using) virtualenv.

crusso2• 14 years ago

Virtualenv is a nice tool for managing library sets.

RVM allows you to do that plus it allows you to install and manage multiple versions of Ruby itself. Want to have side-by-side Ruby environments of 1.8.6 and 1.9.3? No problem.

https://rvm.io//

lowboy• 14 years ago

Check out pythonbrew (https://github.com/utahta/pythonbrew).

In my experience, pythonbrew is to virtualenv as rvm is to rbenv.

jcmhn• 14 years ago

Second on that. I find that pythonbrew fits the way I actually work with multiple installs much better than virtualenv (even with virtualenvwrapper) does.

alaithea• 14 years ago

I only skimmed PEP 405, but did anybody get a sense for how this would work with regard to managing different versions of Python per virtualenv? It seems like a tool to do that necessarily has to be higher in the stack than the Python interpreter.

stfp• 14 years ago

You need virtualvirtualenvenv for that.

daliusd• 14 years ago

I guess the same way as with multiple python interpreters. E.g. under Linux/Mac OS X I might have python python2.5 python2.6 python2.7 and etc. Under Windows I have different locations for different python versions (e.g. c:\python26 c:\python27). You will simply have virtualenv, virtualenv2.6, virtualenv2.7.

tomrod• 14 years ago

This is great! I've been wanting to try virtualenv for awhile based on the good things I've heard. Here's to the developers keeping it modern.

zobzu5• 14 years ago

How are you doing system wide security updates with virtualenv?

tvon• 14 years ago

This would have nothing to do with virtualenv, unless I'm misunderstanding you.

zobzu5• 14 years ago

Virtual env installs libraries for your program/web service/etc.

Those libraries will eventually need security updates.

On Linux for example, which hosts most servers, you upgrade python libraries with yum update/apt-get update/you-name-it.

Single central package database, security updates are marked as security, makes upgrades easy and you never miss one.

With VirtualEnv your dependencies are installed outside the score of the package manager. It means you have to keep track of security updates yourself.

If you don't, and one lib has a well known security bug, you'll be hit and you most likely had no chance to update because "you didn't know".

That's why I'm asking. I know a lot of companies hosting python things and using VirtualEnv because "its easy", but they entirely discard updates. I'm not sure that is wise. I was hoping someone had a solution.

IgorPartola• 14 years ago

This. Python's eggs and repos are a far cry from proper software repositories, such as ones available through apt. Tools like pip do not keep track of what eggs are installed, which ones depend on others, etc. It is up to you to keep a requirements file. If you have more than one project that need to share an environment, now you have to find a way to merge two requirements files. If you ever drop a dependency on a package, it's up to you to manually uninstall it and its dependencies.

Don't get me wrong, pip is great for what it does. It just is not anywhere near what we should find acceptable because of the things it does not do.

baq• 14 years ago

oh god yes. a little bit over 8 years too late, but better late than never.

obilgic• 14 years ago

can we do that for rubygems too?

gosub2• 14 years ago

Does anybody else think that the necessity of virtual environments for programming languages and virtual machines for applications is a sign of failure in package management and sandboxing in modern OSes?

Goladus2• 14 years ago

Sort of, but they necessarily approach the problem from different vectors.

An operating system design with a seamless, straightforward, easy-to-use mechanism for handling all potential dependency conflicts on a multi-purpose, multi-user system is a very hard problem. Any solution is likely to be complex and potentially confusing to users, such as debian alternatives or the modules environment system. Every language has its own library management system: cpan, pip, rubygems, leiningen, etc., and they all profit off the assumption that they are managing a single set of requirements and dependencies. Operating Systems do not have that luxury.

On the other hand from the user perspective, they may not even have root on a system and as such may not even be able to install libraries without contacting a system administrator. Maybe it's a shared hosting environment, maybe it's a research compute cluster, maybe there are SOX restrictions on developers, whatever. Virtual environments focus directly on giving the user/app what they need and isolating that user from everything else.

100 different users can use virtualenv to create 100 different, completely not-compatible python environments and they're all happy. How would an Operating System do that, without essentially duplicating what virtualenv already does? (Not saying that OS developers aren't going to try, which Zed Shaw will then ridicule, but that's another story). For sandboxing in general, at what point do you create the sandbox? Usually for each individual app and usedr you don't want to have to answer questions about whether you still want the C++ compiler available or whether a virtualized /etc/passwd is needed. Those are the sort of questions you have to answer when designing OS-level sandboxes.

gosub2• 14 years ago

>> How would an Operating System do that, without essentially duplicating what virtualenv already does?

This, I think, is backwards: virtualenv must do it already because the os doesn't. I concede that creating something now may be too late not to suffer the "15 competing standards" effect.

Virtualenv to be part of Python 3.3

Related Stories