Starting a Django Project the Right Way

http://www.jeffknupp.com/blog/2012/02/09/starting-a-django-project-the-right-way/

by Brajeshwar • 14 years ago

111 49 14 years ago

lloeki• 14 years ago

    $ git add .

Wait, what? No!

This will add the whole venv, which contains symlinks, scripts with shebangs, and potential binaries, and as such is totally linked to your system, so this definitely breaks if your python ends up in another location or you're entirely on another OS.

What should be done is

    virtualenv env --no-site-packages
    echo "/env" >> .gitignore
    pip freeze > requirements.txt

So when you want to restore/deploy you'd do

    git clone foo
    cd foo
    virtualenv env --no-site-packages
    source ./env/bin/activate
    pip install -r requirements.txt

I'm not even considering the issues regarding the presented git workflow. If one wants to semi-automate a git workflow, one would rather use git-flow instead of this prepare_deployment hack.

kiwidrew• 14 years ago

A hook that I've found useful (though not perfect -- esp. when working on many branches) is to check that "pip freeze" and requirements.txt match before allowing a commit. My hgrc has the following line for this:

    [hooks]
    pretxncommit.pip = bash -c "diff -u requirements.txt <(source env/bin/activate && pip freeze)"

slig3• 14 years ago

Better yet, use virtualenvwrapper and don't worry about cluttering your project folder.

pyre2• 14 years ago

While I agree with the virtualenvwrapper suggestion, the idea that a single directory is going to 'clutter' your project folder is pushing the definition of 'clutter' to me.

redouane• 14 years ago

just add env to .gitignore

tocomment2• 14 years ago

So do you use virtualenv in production? Is there a good tutorial on this for my developers?

pyre2• 14 years ago

I don't think there is any 'best practice' as of yet. It ranges from just running virtualenv, and then using pip + requirements.txt on deploy, to packaging up your virtualenv in a .rpm/.deb and installing via your distro's package manager.

irahul2• 14 years ago

> So do you use virtualenv in production? Is there a good tutorial on this for my developers?

Using virtualenv in production mostly boils down to `pip -r reqs.txt -E virtual_env`(in place of pip install -r reqs.txt) and making sure virtualenv path is the first in sys.path http://code.google.com/p/modwsgi/wiki/VirtualEnvironments

You can also execfile the activation script, but I prefer changing sys.path.

tocomment2• 14 years ago

What's the benefit of using virtualenv in production? And actually what's the benefit in general?

veeti• 14 years ago

FYI, --no-site-packages is default nowadays.

ehutch794• 14 years ago

or better yet, keep anything no part of the project out of the project's directory. if you don't intend on distributing it, it doesn't belong there.

parham• 14 years ago

I wrote a script a while ago that takes care of setting up a similar structure to the django project described and also takes care of issues such as the one you've described. https://github.com/skinnyp/djan-n-go

5h3• 14 years ago

I did only scan the article, but using having a remote repo hosting service (is it really developing on the server while using git?!) configured, branches for feature-dev/bugs/staging/qa/production, vm configuration via chef/puppet, separated settings files, fault reporting etc etc are most (for me) all a part of doing it the "right way" before writing a single line of my own code.

Terretta• 14 years ago

Sounds like you have a great blog article just begging to be let out.

5h3• 14 years ago

Heh, at some point soon[1]... once i've migrated the rest of my websites into rackspace cloud & their "next generation" offering stabilises i'll be writing a "This is how we do it now, it might work for you" type article.

[1] for a yet to be determined value of soon

ehutch794• 14 years ago

somehow i don't think this article was targeted at you. i'm betting if you already know how to run puppet/chef, and have all the above, you already have an opinion on project setup.

5h3• 14 years ago

True, the bit about developing on the server is straight up odd though (if i skimmed it correctly that is) as I find the debug mode exception screens rather helpful, and I wouldnt want debug mode running on a publicly reachable machine

yummyfajitas4• 14 years ago

Don't use Fabric. Seriously, just don't.

It's easier to set up than puppet/chef, but you lose a huge amount of flexibility/robustness. All fabric does is runs commands on host machines. Dependency management is your responsibility.

arocks2• 14 years ago

Interestingly most posts which recommend using Puppet/Chef depend on Fabric for deployment. Is a mutually exclusive or pure Puppet/Chef approach better in any way?

yummyfajitas4• 14 years ago

If you want to use fabric (or a shell script) to run puppet, go ahead. I'm just suggesting that you really want to use a deploy system with proper dependency management.

The issue I ran into with fabric is that I often got stuck in dependency hell. The following is fabric's simplest method of dependency management:

    def install_foo():
        install_foo_dependency()
        ...

Unfortunately, you don't want to do this every time you deploy because install_foo_dependency() might take a while to run. You can work around it by checking inside install_foo_dependency whether it's already there. In practice, you probably won't always do this. Puppet usually has recipes which already do this for you.

So you typically have functions like:

    def full_deploy():
        install_foo_dependency()
        install_foo_without_dependency()

In theory, you can do things right with fabric. In practice, you have to do a lot of work to replicate what puppet (together with assorted easy to find recipes) gives you out of the box.

herrwolfe• 14 years ago

Eric Holscher also has an excellent blog post, http://ericholscher.com/blog/2010/nov/8/building-django-app-... , describing how to deploy using both fabric and chef. There are certain instances where one tool works better than the other, and in that situation that tool is used.

eli• 14 years ago

I agree that puppet/chef is better... But easier? That has not been my experience.

edit: Doh, I can't read

yummyfajitas4• 14 years ago

Reread what I said. Fabric is easier to set up, puppet/chef works better after it's no longer a small project.

mdehaan• 14 years ago

This is why I created http://ansible.github.com

slig3• 14 years ago

He's using fabric to automate the deployment of his app, not to create and manage a new VM.

hcarvalhoalves• 14 years ago

I wouldn't be so harsh. Both Puppet and Chef come with their own (enterprisey) overhead and it's Yet Another Tool to understand and maintain. Chef in particular feels like an over-engineered solution for anyone managing less than hundreds of servers, it adds a lot of cruft (centralized server, authorization, protocols, etc.) that most people wouldn't need. I believe a lot of people like it for the sole reason they are not experienced managing servers, so they can just use pre-made recipes and call it a day.

Anyway, you can go a long way with just a bunch of scripts leveraging Fabric's API. I have setup ~10 servers for a news portal I run from the ground up in just a few lines of code. Managing dependencies is not ridiculously difficult as you make it sound, package managers (apt-get, pip) already handle that for you without any overhead.

unohoo• 14 years ago

completely agree. Chef and puppet could be overkill for several small to mid sized environments. The learning curve for both is relatively steep as compared to fabric. With the parallel exec feature, fabric more than meets our requirements for a small setup (<10 instances).

ehutch794• 14 years ago

i donno, i mean, how often do you think my server gets its dependencies out of sync? (note the singular there)

tbatterii• 14 years ago

I think buildout addresses the deployment problem better than virtualenv + fabric. Only "problem" is it's zope heritage and thus not cool enough for bloggers to use.

sergiotapia• 14 years ago

I really dislike this trend Rails has brought that a Model maps directly to a database table.

I've coded in CakePHP, Rails and ASP.Net MVC3, and out of all three MVC3 was the cleanest one for me just because any Model you created was just a simple POCO class. It didn't map anywhere and prevented you from shooting yourself in the proverbial foot. Problems inherent in Rails and CakePHP if you aren't careful.

I even asked a question on SO about this issue, ZERO responses if you can believe that. I guess the silence is answer enough. ;)

http://stackoverflow.com/questions/11424719/where-do-viewmod...

danso• 14 years ago

I read your SO post but wasn't quite sure what you were trying to get at, because I wasn't quite sure how you perceive the MVC structure in Rails to be.

jspiral• 14 years ago

I read it as a statement that he doesn't like the Active Record pattern

scott_w2• 14 years ago

We find using --system-site-packages on your deployment is a better solution for deploying Django - especially when it comes to dependencies that need to be compiled.

It gives you flexibility on your environment and lets you deploy onto machines without a C compiler, or strange setup requirements e.g. psycopg2 on OS X, and PIL on Ubuntu.

You then install Django, and other Python-only dependencies inside your virtualenv, which stops you polluting the global Python path.

It requires some discipline in your development setup, but it means you're able to develop across whatever platform you choose, and simplifies deployment.

dominicrodger• 14 years ago

What's wrong with PIL on Ubuntu? If you're running into needing to repoint JPEG_ROOT/ZLIB_ROOT - I've started avoiding that by symlinking where PIL expects things to be to where they actually are). Works a charm, and avoids using --system-site-packages.

scott_w2• 14 years ago

I used to do this, but it feels like a suboptimal solution. I could use Fabric to ensure the links are there but it adds more complexity to the setup:

a) Another step to remember b) Requires a compiler on the production system c) Requires you to manually resolve the image library dependencies

Multiplying that across multiple packages can quicky become a headache. It's much easier to let the OS package manager deal with this, and will make your system more robust over time.

zalew• 14 years ago

> If you do a lot of Django development, just dump all of the commands above into a fabfile and make creating a proper Django app a one step process.

if you do a lot of django development, you probably already got a kick ass project template with a requirements file, so you have a basic working website with all the commonly used modules set up and running in 5 seconds.

DanielN2• 14 years ago

I would highly recommend using http://code.google.com/p/django-evolution/ over South

arocks2• 14 years ago

It seems the "What doesn't work?" section lists several significant gaps like ForeignKeys and other constraints. Isn't South much more stable?

jimray• 14 years ago

Any particular reason why? South appears to be under much more active development.

DanielN2• 14 years ago

To be honest I haven't used South in about a year so I don't really know how it has changed. But with South I always seemed to be running into issues of it allowing you to modify existing elements of your schema but not add new elements or modify types in your schema.

ehutch794• 14 years ago

i'm pretty sure south is the only schema migration app recommended at this point.

Starting a Django Project the Right Way

Related Stories