Creating Python Virtual Environments with Conda: Why and How

https://heartbeat.fritz.ai/creating-python-virtual-environments-with-conda-why-and-how-180ebd02d1db

by Anoetar • 7 years ago

51 21 7 years ago

heartbeat.fritz.ai

bransonf• 7 years ago

This tells me why a virtual environment is good, but not why a Conda venv is good. Am I missing something?

Why not just use pip?

tel• 7 years ago

Conda can manage a few things beyond just Pip. Things like binaries, system libs, R installs.

I'm not entirely convinced that's a good thing, but it's an easy path to managing those sorts of libraries alongside your Python libs.

RobinL• 7 years ago

One reason it can be very useful is that a conda environment gives data scientists a super easy way to Dockerise their code.

Binder (https://mybinder.org/) is a good example of how well this can work - anyone can reproduce your work using e.g. Jupyter hosted on a binder, build from a conda env.

Gimpei• 7 years ago

Conda takes care of a lot of other useful things. For example, it will install cuda and cudnn for you. It will also install mkl. So much easier than pip + many other steps.

jhrmnn• 7 years ago

Conda virtual environments and Pip are orthogonal. Pip can be used to install into venvs. But as to why use conda rather than just the standard library venv module (python3 -m venv venv): conda can easily manage multiple python versions. In my experience, it's the most hassle-free option compared to something like pyenv. I personally use conda just for managing venvs, and don't use its package manager functionality at all.

dfsegoat• 7 years ago

FWIW our use case is the same:

- Conda to manage multiple python versions on 1 box:

  $ conda create -n env27 python=2.7
  $ conda create -n env37 python=3.7

- Pip to manage packages in the environment:

  $ source activate env27
  $ pip install -r requirements_27.txt

qwerty456127• 7 years ago

But why would anybody need a Python version other than the latest? Aren't they backward-compatible within a major version? Won't everything written for Python 3.7 work on 3.8 and everything written for 2.6 work on 2.7?

meritt• 7 years ago

I'm by no means a Python expert but I recently had a small project where I needed to convert the output of an SQL query to a parquet file, and I settled on using fastparquet to do so.

I first tried using pip under a venv to install it, and I ran into dependency hell. I needed various dev libs and build-essential on my ubuntu box and I ultimately couldn't get it to compile llvmlite correctly (linker issuers and way too many LLVM versions).

I finally gave up and tried miniconda on a fresh box per some advice here on HN. I simply installed it, ran "conda create -n myenv fastparquet" and it just worked first try. No extra effort involved.

phenkdo2• 7 years ago

Not sure, why this article is on HN front page.

airstrike2• 7 years ago

Because people are upvoting it.

phenkdo2• 7 years ago

Thanks for the snark, but my point was is this article front-page worthy?

jknoepfler• 7 years ago

I mean, if you've ever hosed your ubuntu (etc) installation by replacing your system's python2 with python3, you already get the why.

ggm• 7 years ago

bring back stow!

kevin_thibedeau• 7 years ago

It never left.

mulmen2• 7 years ago

Is there a global or user config that overrides the environment config file?

It seems less valuable if the prefix is hard coded to the specific user. To a lesser extent even specifying the name may be too limiting.

jawilson2• 7 years ago

Do you mean the condarc file? You can do `export CONDARC=./project_condarc`

mulmen2• 7 years ago

No I mean in the config file example the PREFIX is set to a static path including a user directory. This means if anyone checks out the config file and tries to run it the path will not exist. Is there an override somewhere that provides the desired prefix at a user level? Something like ~/.condarc or /etc/condarc?

Ideally I would set the prefix for Conda for myself only once (~/.condarc?) and the source controlled config file would then extend that config.

I would expect the conda config file in a project to say nothing at all about the prefix unless it is relative to an existing prefix I set elsewhere.

This way I don't have to manipulate my environment and edit config files every time I run conda.

I should probably just RTFM.

jaspergilley• 7 years ago

Or, if you actually know how to use the Unix command line in non-trivial ways, you can just use pip.

talentedcoin• 7 years ago

Although the article doesn’t really go into it, the best thing about using conda is that it obviates the need for pipenv or poetry.

burkesquires• 7 years ago

As already stated conda and pip both enable you to create separate environments. A major advantage of using conda over pip is that conda has additional channels, such as bioconda and conda-forge, that enable software besides python or R to be installed. Through bioconda a user can install bioinformatics unix software with a single command. See https://conda-forge.org/ and https://bioconda.github.io/.

Also, you can use pip to install software if you have the anaconda distribution installed. As far as I know it does not work the other way around.

Creating Python Virtual Environments with Conda: Why and How

Related Stories