A List of Useful Backend Architectures

https://gist.github.com/ragingwind/5840075

by yarapavan • 13 years ago

98 25 13 years ago

gist.github.com

asperous• 13 years ago

Completely dominated by open source projects.

This helps my theory that when you release open source code, you are improving the baseline technology for the entire world.

thejosh• 13 years ago

Yep.

And when companies release the products THEY use and find helpful, everyone improves it and releases it back.

Memcache springs to mind.

zimbatm• 13 years ago

Not necessarily. I have tons of github repos where nobody contributes to. I don't see why it wouldn't be the same for a company. It depends if your project is useful to others and if you're able to market it properly.

krenoten• 13 years ago

There is an extreme case of the availability heuristic at work here. There are a ton of incredibly successful companies that use little or no OSS.

presty• 13 years ago

architectures != stacks

jaggederest• 13 years ago

Agreed. This is a list of buzzwords, not a list of architectures.

An architecture would be something like 'all major work done via job queues distributed across cheap machines, all pages statically built and pushed to CMS', or 'a dozen bare-metal machines with serious hardware running one-machine sharded app/cache/db slices and hot backups of each shard', fun stuff like that. relatively technology agnostic.

gizzlon• 13 years ago

Agreed, but it's still a nice list with a lot of information.

Check out the first chapter of "The Architecture of Open Source Applications (Volume Two)":

"Scalable Web Architecture and Distributed Systems" http://www.aosabook.org/en/distsys.html

camus• 13 years ago

A a lot of these start-up (except for Facebook maybe ) are using python in their stack, why is that ? all I hear about around me is "Java Java Java". What makes python better ? ( it is just a question ). Or is python just for mvp then the stack moves to Java ( like Twitter did with rails )?

kaoD• 13 years ago

I'd advise against Python. There are too many caveats in it for a language that's supposed not to surprise me.

My biggest gripe: it's easy to shoot yourself in the foot using Unicode. The system is cumbersome and the lack of typing makes it easy to slip a regular ASCII string somewhere, which will jump at your face when you expect it the least morphed into a stack trace.

The data structures' API is the worst and most inconsistent API I've ever seen. Why is 'dict' not 'Dict' ?

A comma ending a line actually makes a tuple, that's asking for trouble. I've witnessed more than one bug caused by this.

It is slow and resource hungry. Django is too[1]. GlobalInterpreterLock[2].

The package system is horrible, and pip won't save your day either. I love Clojure and Node.js just because of Leiningen/NPM. I couldn't use PyOpenGL in Windows from some unknown reason. I gave up!

Discourages functional programming, while encouraging ugly inline list comprehensions. The solution to the half-assed closures in Python2.7[3] was to introduce 'nonloncal'. Lists are mutable. Object Orientation feels clunky to me and using super is awkward.

"Batteries included" but core libraries won't take you too far. Unfortunately they're still bloating the language.

Concurrency is hard. Twisted solves it a bit but... meh...

What I like:

- Proper nested comprehensions (CoffeeScript completely blew this one...)

- I like the minimal syntax, it helps me visualize the program better.

- PyLint/PyFlakes highlights lines that don't conform to PEP8.

- Generators.

[1] http://www.techempower.com/benchmarks/#section=data-r5

[2] http://ynniv.com/blog/2007/08/closures-in-python.html (section Caveats)

[3] http://wiki.python.org/moin/GlobalInterpreterLock

jzwinck• 13 years ago

What would you use instead of Python? It should be roughly comparable in terms of having a wide array of well-used libraries, and of course a good REPL.

I've been using Python for a couple years and it's mostly joyful. It is unfortunate that not even the various containers within "collections" use a uniform naming convention, I'll give you that. And some of the scoping stuff is weird (nonlocal which nobody really uses? and seemingly-but-not constant default function arguments). But Twisted is a pretty good evented programming system, and NumPy plus multiprocessing let you process a lot of data efficiently, GIL or no. I cannot agree that "the core libraries won't take you too far"--they've taken me farther than the comparable bits in any other language.

w0utert• 13 years ago

There's nothing in your list that makes Python less suitable for web development than [insert favorite language]. Many of the issues you describe (I agree with most but not all) are either irrelevant for web application development (e.g. the GIL, I/O bound code such as web applications do not suffer from it at all, especially not when using something like gevent), easily avoided or worked around (unicode strings: either use Python 3, or a web framework that defaults to always unicode + do static source analysis for non-unicode string literals), already have pretty good solutions (package management: always use virtualenv and pip), or a matter of style/taste (dict vs Dict, functional aspects, etc).

I could make my own list of reasons why I think Java sucks for web development, but someone who's proficient in Java web application development will disagree with those just the same. Every language has its faults, and Python is no exception. Personally I simply hate writing Java code, of all the languages I've used, it's by far the least enjoyable to write and maintain.

darkarmani• 13 years ago

A comma ending a line actually makes a tuple, that's asking for trouble. I've witnessed more than one bug caused by this.

Spurious symbols at the end of a line is really a complaint? Have you ever forgotten a semi-colon in c/c++ code?

dagw• 13 years ago

This is like discussing the architecture of buildings by listing the principal building materials

wmblaettler• 13 years ago

It's at least a good jumping off point to read more about how to use the materials to build the type of thing you want. I haven't even heard of many of them, but I work in the .NET world these days.

pietro• 13 years ago

That wouldn't be a bad thing, actually. The choice of building material has implications far beyond what you'd imagine.

Duelist• 13 years ago

Agreed. At the very least, it would be helpful to know the specific applications of the languages in projects that use more than one.

davidradcliffe• 13 years ago

What's with "my-sql". Shouldn't it be "MySQL"?

arturhoo• 13 years ago

http://highscalability.com/ is also a great blog for those interested in scaling backend architectures, they have great use cases

hellopat• 13 years ago

While these aren't so much architectures as they are a list of tools, it's fascinating to see which ones successful companies are using.

I like that Trello has such a short list and still runs incredibly fast with a high number of users.

twic• 13 years ago

You say that like picking a small number of decent building blocks and then putting your energy into building high-quality code rather than wrangling a circus of services is a bad way to build scalable systems.

anuragramdasan• 13 years ago

This was disappointing. I came here actually expecting to see something about architecture but ended up looking at product stacks.

having said that, I am somewhat glad to see django there. for all the inconsistencies and speed issues that have already mentioned about python and django, it is great to see that the framework still gives pretty good performance on large systems.

A List of Useful Backend Architectures

Related Stories