Django's goals are probably not our goals for our web application

https://utcc.utoronto.ca/~cks/space/blog/python/DjangoGoalsNotOurGoals

by andyjpb • 7 years ago

36 58 7 years ago

utcc.utoronto.ca

cardamomo• 7 years ago

I'm aware of the well-established movement to static websites for (mostly) static sites, but what work is happening right now to counter this problem in rich web applications? Are there frameworks that take a stance that favors stability and maintenance?

SwellJoe• 7 years ago

I think the move toward micro-services (and serverless and whatever other buzzwords you want to use for a distributed loosely coupled design) is an answer to that problem, if not exactly the answer you're describing.

The idea being that each small piece is easier to leave alone for long periods of time, and, should it ever become impossible to leave it alone, whether for security reasons or something else, updating it or replacing it is much easier than doing the same for a big tightly coupled pile of components.

freshhawk• 7 years ago

Frameworks aren't the design choice that favours stability. You take the inversion of control to get all the convention/plug-and-play/just-works features that come with it. Those are focused on developer productivity, which is a constantly shifting goal.

Avoid that design and using a "libraries over frameworks" approach solves this problem but has other tradeoffs obviously, although a good ecosystem of libraries that have designed for interop solves most of them. These library ecosystems aren't the norm though, and the lure of "I'm going to package these together in a framework, that does things the way I like, and then brand it and help my career" for the library devs is always there (see Flask in Python for example, it was the "unDjango" until it got popular and now it's just another framework)

JustSomeNobody• 7 years ago

Those frameworks would be considered dead by most devs these days. If the last change log is dated older than a week they start to panic.

GuiA• 7 years ago

This is a need I’d like to see addressed more broadly in software. The computing needs of many in my non-technical entourage have not changed meaningfully in the past 10 years - they crop photos, write letters, put together videos of family holidays, do basic spreadsheets, etc - and yet every few months I field support requests because their software has updated and some feature has gone missing, or the UI has been redesigned, or it’s not compatible with the new OS update, etc.

Bug/security fixes and new format support (eg a photo editor probably wants to support HEIC in the 202x) are fine, but we don’t really need the user facing part to change. This is a niche that seems open source could fill nicely.

(The big edge case in all this is the web browser)

torben-friis• 7 years ago

Open source software has always had a major problem with user experience (as well as aesthetics).

Most people working in those projects come from a strong technical background, and in many cases users come from that background as well - so there's not much of a push for improvements in areas like ux, nor a business need to improve to boost sales.

reaperducer3• 7 years ago

I feel their pain.

One of the big sites that I maintain was built upon a framework that was fine for its purposes five years ago. But the developers of that framework have decided to take it in a direction that is incompatible with the site's goals.

Once I'm done with my current project, my next task is to rebuild that site without its current framework, and there's a 95% chance I'll go without any framework at all. Just plain HTML5 + PHP + MySQL + CSS3 + a very small amount of javascript.

I know it sounds primitive for a 700+ page site with interactive elements supporting English, Spanish, and Chinese; but earlier this year my boss gave me permission to de-framework a similarly-sized site, and it worked out really well.

Frameworks are great for many projects, but not all. It's important to weigh the risks of being at the mercy of someone else, and that organization may eventually adopt a different vision of the future. Their pivot will not make your life easier.

Nextgrid2• 7 years ago

Would a microframework (no idea what's the equivalent of Flask or Sinatra is on PHP) help? I still have nightmares from my beginner days of doing everything in plain PHP.

reaperducer3• 7 years ago

Would a microframework (no idea what's the equivalent of Flask or Sinatra is on PHP) help?

I have no idea. Maybe. I've just been coding straight PHP. With the way IDEs are these days, once your environment is set up the way you like and you have your favorite macros bound, it's not a problem.

I'll look into Flask and Sinatra before I start the next site. Any other ones you recommend?

crispinb• 7 years ago

This is endemic to the tech world. So much of it is invested in a reflexive forward momentum that is inimical to at least some of the products it facilitates. This is very much true for example of many pretty slow-moving small-scale line of business mobile & web apps. So many of them end up being a waste of money after initial launch as the platform outruns them, and further work becomes, as the author here suggests, pure overhead.

It's among the reasons I tend to be deflationary when being consulted with about extending web or mobile presence by small business. Often I just suggest they don't bother, or do something far more minimal than their first impulse.

jsmeaton3• 7 years ago

The author writes that they have a particularly big piece of maintenance coming up.

The latest change we need is an especially large amount of work, as we will have to move from Python 2 to Python 3.

This sounds like the transition from Django 1.11 to (probably) Django 2.2, if they're moving from LTS to LTS, and it definitely is a big piece of work if you also have to change python versions.

Django defined a release schedule DEP[0] a few years back to try to preference stability and reduce the amount of maintenance when moving between LTS versions, which is approximately every 3 years.

If your application runs on an LTS version without any deprecation warnings, then you should be able to move to the next LTS version without any changes. Which is nice in theory, but 3rd party packages in particular aren't always aligned in such a nice way. If the stars align though, it is 6 years (LTS + LTS) of no changes.

This is all to say, I think the author has a point, but that Django has recognised the maintenance burden as an issue, and has tried to help developers with that as best they can, without just shutting down all progress. I think the balance is mostly right.

Moving from 1.11 to 2.2 should be mostly painless. Unfortunately, moving from Python 2 to Python 3 is going to make that more painful. From 2.2 the maintenance burden should be much less than in previous years. Stick with Django if the features meet your needs.

[0] https://github.com/django/deps/blob/master/final/0004-releas...

1337shadow3• 7 years ago

Every once in a while someone opens an issue on a repo i maintain asking for "Django 1.11" support: 2 days ago, 20 days ago ... For me, "LTS" is a mistake indeed and and goes against any Continuous Delivery / Agile philosophy that I know of, it serves as an excuse for users to not update their software which in reality accumulates tech debt. I mean, if you're going to do very little customizations to a system then LTS might be fine, but if it serves as a base to something that's custom development from scratch, like Django, it really doesn't look like a choice that's going to pay off in the long term.

jsmeaton3• 7 years ago

As a library author that’s well within your rights to determine what you’re willing to support or not.

Django has a very big user base. There are many users working on “very large sites” where the cost of “churn” is dramatic. There are threads on Django-developers right now complaining that the current model is too aggressive. Striking that balance is difficult for a heavily used framework.

cj• 7 years ago

On the frontend, our company uses Backbone.js.

My favorite part of Backbone.js is that its development as a framework is essentially “complete” (it works as designed, functions reliably / predictably, and the maintainers aren’t running in circles trying to reinvent the framework with breaking changes)

That said, I feel like I’m losing “street cred” for admitting that we still use Backbone.

It would be great, at least in my opinion, to see more frameworks like Backbone.js (with respect to reaching a “framework is complete” state).

Backbone.js Changelog: https://backbonejs.org/#changelog

Edit: All of the above also applies to Underscore.js (https://underscorejs.org/#changelog)

reustle• 7 years ago

We did some client work up until about 2 years ago with Backbone, too. We loved it. The entire team knew it very well and it got the job done. At least a few developers at the client would also be familiar with it every time, so it was a win win.

andrewstuart2• 7 years ago

I don't really understand the point being made here. Can someone reinterpret for me please?

resoluteteeth• 7 years ago

To get security fixes they have to update to new versions, which requires them to change their code, but they don't want to have to change their code once it's done.

swanson• 7 years ago

The author has opted to use Django and would like to get to a state where their app is essentially "completed" -- where it will work fine without having to touch it again or make major changes.

The conflict arises because Django has a stated mission to always be improving (like a perfectionist). This causes issues for the author when they have to re-write parts of their application in order to keep up with updates to the mainline framework.

The author would like be able to get things like security fixes and performance improvements without breaking changes -- whereas the Django team is willing to require breaking changes in pursuit of improvements.

Ultimately, the author may be better served by using tools/frameworks that are considered "completed" (examples might be things like jQuery) or have a maintenance philosophy that prioritizes backporting and compatibility over improving the features/experience.

andrewstuart2• 7 years ago

Oh I see.

Software is a living thing - it needs to keep up with the technologies it is built on for a whole bunch of reasons.

Some software of course can be "complete" but that tends to be pretty specific stuff with a very defined scope.

txcwpalpha2• 7 years ago

The author essentially states that he just wants to build a website and "be done with it" and never have to touch it again. He seems to acquiesce that security updates are necessary, and I suppose his greatest complaint is that old versions of software (eg python 2, django 1.11) are no longer receiving security updates, which necessitates him to update to python3 and django 2.

I feel that. But on the other hand, without trying to sound too mean: what exactly does he expect? Is he also complaining that "Windows's goals are not our goals" because Microsoft stopped supporting Windows XP? Does he complain that Ubuntu version 11 is no longer getting updates? Java SE 9?

For better or for worse, this is simply how software works. Things have to be updated. Unless you want to eschew security updates (which is not smart), there is never such a thing as a fully "finished" product. Software maintenance is part of the job responsibilities.

This post is particularly targeted at Django, but I feel that it applies to almost every web framework or piece of software, and I'm afraid this is unfairly casting Django in a bad light. I'm also afraid that we're going to see a lot more posts like this in the second half of 2019 as we move towards python 2's EOL date.

reaperducer3• 7 years ago

The author acknowledges that some maintenance is necessary, especially for security updates. My reading of it was that his complaint is that in keeping up with the security upgrades, Django breaks other things.

This is true of a number of frameworks I've worked with. If you want security, you have to constantly stay on top of the other changes that the framework makers implement, often "fixing" things that aren't broken, removing features, or changing the way things work.

I don't know how practical it is, but it would be nice to see a framework where security updates and feature updates were separate things. One could keep one's site secure through updates without having to debug formerly working things every few months.

txcwpalpha2• 7 years ago

I don't know how practical it is, but it would be nice to see a framework where security updates and feature updates were separate things. One could keep one's site secure through updates without having to debug formerly working things every few months.

This is more or less what an LTS release is. The problem as I see it is that, as the codebases diverge, it becomes more and more work for the dev teams to support what is now essentially two different products. And especially in the open source world, it seems futile to expect unpaid or low-paid contributors to continue to do extra work simply because someone else's dev team doesn't want to put in the work doing updates.

aeternus• 7 years ago

Unfortunately it's not practical because the test matrix explodes. They would have to develop and test every security update on every previous version which is quite costly, especially if the codebase has diverged.

The best strategy is probably what many large distros like Ubuntu have adopted: designate only a few long term support (LTS) versions, and commit to supporting only those versions.

orf• 7 years ago

My reading of it was that his complaint is that in keeping up with the security upgrades, Django breaks other things.

Django takes great pains to ensure that this isn't true. We don't backport anything other than bugfixes and security updates to releases. It's not like we suddenly introduce a new feature into a previous release that's backwards incompatible with your current code. If you're on the LTS release then you can be happy in upgrading Django to the latest point release with all the latest fixes.

And the Django project has finite resources. It cannot backport security fixes indefinitely which is why we have the LTS release in the first place. If you want something stable: sit on that for several years at a time, or spend your time backporting any security fixes yourselves.

Unfortunately software is a moving target, but Django tries to make it easy to upgrade from one LTS release to the next. The upgrading difficulty often comes from third party packages, which is hard to avoid in any framework or language.

svnpenn• 7 years ago

Yep. I was interested in the post as I've always felt Django is python software "done right". I stopped reading when they started complaining about python 3.

Python 3 has been out for over 10 years now. Did OP really think Django was going to use python 2 forever? Frankly that's a selfish outlook, and I'd go as far as saying it's a cancer of software development.

Backward compatibility needs to be a thing, but not forever. Things need to be deprecated eventually.

faitswulff• 7 years ago

Backward compatibility needs to be a thing, but not forever. Things need to be deprecated eventually.

That might not need to be the case. I guess we'll see: one of Rust's tenets is perpetual backwards compatibility. New features (and potential breaking changes) are opt-in with a per project configuration setting.

philwelch5• 7 years ago

Backward compatibility needs to be a thing, but not forever. Things need to be deprecated eventually.

This mentality is the true cancer of software development IMO. Can you imagine the utter insanity and chaos if any other form of engineering worked this way? Programming languages are like protocols and standards: they are a common platform that people build on; simply deprecating them with no plan for backwards compatibility requires everything to be rebuilt, which represents a tremendous cost. Sometimes these changes are worth it, but outside of software, these are usually once-in-a-lifetime changes.

I can't think of any fundamental technical change between Python 2 and Python 3 that make this scale of disruption worthwhile. If the maintainers of Python don't want to maintain Python 2 anymore, they are completely free to make that decision, but it's a decision worth criticizing and complaining about. It is not a good thing for all software developers to be forced to rebuild the same sand castles every time the tide goes back out.

orev• 7 years ago

For better or for worse, this is simply how software works.

This post is more or less a lament about that fact, not necessarily a complaint or gripe. Chris is a Sysadmin, and there’s a big difference in perspective between this and a developer of an app whose reason for being is THE APP. A Sysadmin needs to provide a bunch of services that are not the core revenue generator, and multiples of them, meaning that every internal app has this same kind of updating overhead. It becomes tiresome to have to keep all of them up to date all the time. We know that is a fact of life, but not all facts of life are fun and people like to blow off steam about that.

mikekchar• 7 years ago

There is a saying amongst growers of bonsai trees: A tree isn't "finished" until it is dead. In other words, it keeps growing and so you constantly have to keep working on it. You can't just think of a design, prune the tree and then say, "Oh, I'm done! Let it stay like this forever".

Software is exactly the same. Your needs may never changes, but the world around you does. You have no control of that and so you must constantly maintain your code -- even if there are no changes that you want.

It's not just Python or django. It's hasn't even possible to compile Ruby 1.8.7 with recent versions of GCC for at least a year (don't ask me how I know... :-P ). Starting a software project -- any project -- is a long term commitment. If you want it to survive, you need to care for it.

verisimilitudes• 7 years ago

I feel that. But on the other hand, without trying to sound too mean: what exactly does he expect?

It's not unreasonable to expect to be finished with software.

For better or for worse, this is simply how software works.

This is false. The author has already fallen into this trap by using a programming language which is also subject to this, however.

Things have to be updated. Unless you want to eschew security updates (which is not smart), there is never such a thing as a fully "finished" product.

There is such a thing as finished software, but I can understand why something such as a WWW framework wouldn't be. The WWW is ugly and misguided and inspires disgusting systems to accomodate it.

Software maintenance is part of the job responsibilities.

When you have a project with no real design goals, apparently, I agree. Otherwise, you can remark on how a program achieves its goals and so is finished. That doesn't stop minor improvements or reworkings, but those would be entirely unnecessary.

This post is particularly targeted at Django, but I feel that it applies to almost every web framework or piece of software

I agree, because, again, the WWW is a disgusting tangle of systems that encourages similarly disgusting systems.

One language I enjoy is Common Lisp. The standard hasn't changed in decades and there are many implementations. I can finish a library and then never need to touch it again. Python can't compare.

carwyn• 7 years ago

Different frameworks and language run-times move at different paces. It's very often the case that projects don't match their product pace and life-cycle to their toolset pace and life-cycle. "Standards" be it ABIs, APIs, data formats or markup languages all have different paces and life-cycles. Then there are the paces and life-cycles of the stacks underneath, the OS, the hardware, networks and data sources.

Then there's the security question. If this is a concern at all, ultimately the system may need to be air-gapped (e.g. like many systems depending on Windows XP are these days) to safely keep it running in lieu of fixes.

Another approach is common in the gaming scene with painstaking work on timing and bug-accurate emulation to keep old software running.

Ultimatley every system has a cost per annum, not a static capital cost. The end game being to maintain the hardware and "firewall" or update the software.

quickthrower22• 7 years ago

Vanilla PHP is your friend then. Don't use frameworks.

smacktoward• 7 years ago

Vanilla PHP isn’t any “safer” in this regard. There’s lots of old PHP code out there that stopped working when PHP 7 removed a bunch of old, long-deprecated legacy cruft. That was good for the platform overall, but if you wrote a vanilla PHP app in 2008 and haven’t touched it since, odds are it complicated your life.

TFA is asking for a platform that never moves. The problem is that the only platforms that never move are ones that have died.

quickthrower22• 7 years ago

That's a good point. You need a language that doesn't move and always remains backwards compatiable. But I presume C is not a great choice as it's too low level.

I think JS will have this property for a long time, because of it's browser ties. And Haskell is another language I think won't change dramatically if you use the fundimentals. However in both cases you will want to roll your own web framework or at least pin to one you like, ideally something minimal.

However it does raise the problem of security patching, or lack of if you go down such a route.

The other mitigation, which you can do in any language is Inversion of Control. Have your code specify what sort of things it needs rather than use hard dependencies. E.g. "I need something that takes a string and a function and routes it" rather than "I need Express.js", with a layer that maps this up to actual implementations.

This way if frameworks change the mapping layer should be all that needs to change. Again pick a language like JS, Haskell, probably C#/Java etc. that is likely to be backwards compatiable for a few years to come.

jb827• 7 years ago

I think complexity here is migration to Python 3. Seems like the author picked Django and Python 2 in 2011. Python 3 was released in 2008. So by 2011 should have been clear that you might be facing a migration project sometime in future. Having made that choice getting 8 years out of Py 2 is pretty good. (Not saying he made the wrong choice. Would have made the same choice myself.)

fatbird• 7 years ago

The author isn't considering an alternative strategy, which is to address security issues in-place (i.e., without upgrading Django). This means keeping up on them and substantively addressing them through configuration or possibly custom code, but for something they want to freeze in time with respect to a platform, it's a reasonable choice.

Every time I've had security issues to address, there's almost always a short term fix (close a port, disable a particular eval method). In the cost/benefit analysis of maintaining a legacy app, it's a legitimate option to simply rely on those.

1337shadow3• 7 years ago

Take a day or two to do your upgrades once in a year and you'll be just fine. Start upgrading your own plugins as soon as possible, so that you can learn how to practice the upgrade steps. Python 2 to 3 was hard for me for the first few codebases, but then piece of cake for the other dozens of codebases I have upgraded.

tptacek• 7 years ago

There's a business opportunity here that Rails people already caught onto: you can start a 3rd-party "Django LTS" service that backports security fixes, and charge money for that.

jsmeaton3• 7 years ago

I’ve previously suggested this as something the DSF could do as a money maker, but I don’t think the operational complexity makes sense for a non profit foundation.

I’m surprised no one else has picked this up as a business idea though. The actual technical work likely wouldn’t be very hard.

Grue3• 7 years ago

That's why you develop in Common Lisp. The language specification is 20 years old and is never getting updated. Your software will work forever!

Django's goals are probably not our goals for our web application

Related Stories