There's no magic: virtualenv edition

https://www.hackerschool.com/blog/14-there-is-no-magic-virtualenv-edition

by akaptur • 13 years ago

134 24 13 years ago

pm90• 13 years ago

Excellent article. Although it is quite obvious how venv worked, I had never bothered to actually look up the source. Consequently, I was always a bit hesitant on using it, because I have a customized environment for installing stuff and didn't want to break it. But venv is doing just what I'm doing, but automatically :)

jacobian• 13 years ago

Related: Carl Meyer's "Reverse-engineering Ian Bicking's brain" talk from PyCon 2011: http://pyvideo.org/video/389/pycon-2011--reverse-engineering...

yesimahuman• 13 years ago

Understanding magic means you understand the concepts. It's a very advanced position to be in relative to your programming education.

I'm guessing it's also why "hard" classes in school start with the concepts rather than the magic (physics is a perfect example). You can't really go far if you don't get the concepts.

I've been doing some training lately for new hires, and I am inclined to start with the concepts and then teach the magic, but I wonder if it's the most effective way to learn in the short term?

rplacd• 13 years ago

Don't fear abstraction - that's as far as we'll get in plenty of cases, especially when we're talking about the epistemology of the sciences; in any case, there's always an element of human control involved - once opinionated humans are put in charge of articulating abstractions, we tend towards elegance, orthogonality, the stuff that makes you grin.

Fear leaky abstractions, though.

ImprovedSilence• 13 years ago

haha, I had no idea "source" and '.' were the same! It all gets a little bit clearer, one step at time....

pyre• 13 years ago

IIRC, 'source' is a bash-ism, while '.' is POSIX, so it might not work in all shells. That said $() isn't POSIX, but works on enough shells, that I'm comfortable replacing `` with it.

mpyne• 13 years ago

Actually $() is in POSIX (or at least, it is now).

You're right about 'source' though, shell scripts that want a semblance of portability must use '.'.

nuxi• 13 years ago

$() has been POSIX for a while, here's the SUSv2 definition from 1997: http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#...

j4mie• 13 years ago

See also "Virtualenv's bin/activate is Doing It Wrong": https://gist.github.com/datagrok/2199506

The activate/deactivate magic is responsible for some real nastiness in deployment scripts, and total confusion for beginners who are thrown by the statefulness. Virtualenvwrapper makes the situation even worse by hiding the virtual environments away.

By far the easiest way to use virtualenv is to forget about activation completely, and directly reference the binary you want to use. For example, rather than typing:

    source env/bin/activate
    python myprogram.py

Just run

    env/bin/python myprogram.py

It's an extra few characters on every command, but it's worth it to totally remove the magic.

kami88452• 13 years ago

No thank you. I will continue using my "magic" that has caused me 0 problems over the past couple of years running python scripts thousands of times. If you hate automation so much, feel free to keep typing "env/bin/python" every time you execute something, but for 99.99% of us, virtualenv works perfectly fine for everyday use.

ProblemFactory• 13 years ago

> for 99.99% of us, virtualenv works perfectly fine for everyday use.

Depends on what you are running and from where. The usual virtualenv activate (although I recommend virtualenvwrapper's workon) works fine for a human interacting with a terminal.

But it is annoying for bash scripts, extremely annoying for Fabric scripts where each command is run in a separate SSH session, and sometimes completely impossible when a server needs to spawn a python subprocess. Used this exact method to solve issues with nginx+uwsgi web server deployments a few weeks ago.

Using the full path to the virtualenv's python works in any context, even when you can't set environment variables or run two commands.

freshhawk2• 13 years ago

> for 99.99% of us,

Really? Most of the time when people say that they mean "for 1 (me) of us,"

Are you sure that isn't what you mean?

achompas• 13 years ago

I think you're overestimating. virtualenv works "fine" for some but it has warts when interacting with the scientific toolkit. IPython gets annoying, for example.

datagrok2• 13 years ago

FWIW I know "doing it wrong" was a bit inflammatory, and generates this kind of reaction from people who think I'm determined to take their easy workflow away.

I've got plans to address this sort of complaint, to let folks who like the current behavior to keep working the way they always have and enable those like me who want a "keep all the state in a subshell" behavior to get my proposed workflow.

As one commenter points out, those plans haven't moved forward since mid 2012. I don't consider the issue "dropped" though. I'll get around to it.

jtchang• 13 years ago

I didn't even know about the activate command. I just remapped my PATH in my shell to the python virtualenv and do it for all the new users.

It makes things easy. The system python is kept clean and I don't have to deal with it.

justinsteele• 13 years ago

Interestingly the author submitted an issue and also posted on the message board, but has not updated anything since April 2012. Seems it simply got dropped.

datagrok2• 13 years ago

No, just busy with other things. I'll get around to it if nobody else does first. ;) Hopefully before PyCon 2013.

clicks2• 13 years ago

(A little bit off-topic, but I see that you're the person who wrote the Flask tutorial, just wanted to say thanks for writing it, I loved it: https://github.com/akaptur/MyFlaskTutorial ).

akaptur• 13 years ago

Glad you liked it, but that's actually my fork of the original, which was by another Hacker Schooler: https://github.com/bev-a-tron/MyFlaskTutorial

I really liked the way she distinguished arbitrary variable names from mandatory class names etc. So often, tutorials don't make that clear.

clicks2• 13 years ago

Ah. Well thanks to you both. :)

There's no magic: virtualenv edition

Related Stories