What's Going On (on "Why Python, Ruby, and Javascript are Slow")

by ANTSANTS • 13 years ago

283 192 13 years ago

cjh_3• 13 years ago

Very interesting read.

I have only written a small amount of Go, I have been through tour.golang.org, just waiting for the right project for me to use it on.

Currently whenever a new project comes along I reach for C or Perl, hoping to find a niche between them for Go.

berntb4• 13 years ago

The feature with scripting languages compared with system languages (C, Java, etc) is that they are fast -- not to run code, but to develop functionality.

The article argued that idiomatic Go gave at least as good runtime speedup as a normal scripting language (Perl, Ruby, etc) with the (small) time critical parts rewritten in C.

But it didn't touch the important question -- is the development time for Go faster than for scripting languages with critical sections in C?

Go looks fun, but I'd personally want information on that metric before trying it on a project. [Edit: Yeah, as the article noted, Go excels re parallelization etc. I might Go for it in those use cases.]

(What I'd really want is to find a place which use Lisp, since development speed is roughly as fast as for scripting languages and there has been good compilers for decades.)

Edit: Spelling, must start going over the room for the keyboard when using the iPad.

voidlogic5• 13 years ago

"But he didn't touch the important question -- is the development time for Go faster than for scripting languages + C?"

That is a good question, but- Isn't that going to vary a lot by the person/team involved? Calculating "effort" is far more difficult than run-time or complexity characteristics.

Doesn't that question also assume that people writing in the scripting language can write (correct/decent/secure) C. Ironically, I have a strong feeling that many more Go developers can write C than say Ruby or PHP devs.

cjh_3• 13 years ago

I see this as a devtime-runtime tradeoff, If you follow the link to the mentioned article [1] there is some discussion about how to improve on this tradeoff.

FWIW: I use C for many of my personal projects as I often enjoy solving problems at that level, however at my job I am 90% Perl with the other 10% being various web things (css, js, html, etc.).

I too would be very interested in articles looking at performance of some of the older dynamic languages.

[1] https://speakerdeck.com/alex/why-python-ruby-and-javascript-...

mrgordon• 13 years ago

Yeah, I've been enjoying using Clojure for this reason. The strengths of the JVM and Java libraries with the development time and functional style of a Lisp.

szopa• 13 years ago

> But it didn't touch the important question -- is the development time for Go faster than for scripting languages with critical sections in C?

At YouTube (which is a predominantly Python shop) we use Go for some elements of our infrastructure, and I spend approximately half of my time working in each language (main project in Go, integration tests in Python). In my experience easy things (say, split and massage some strings) are more complicated/annoying in Go than in Python (and much more verbose). However, difficult things usually end up being much simpler in Go than in Python – especially if they involve concurrency (Python threading is a trainwreck). The difficult parts of our code consume most of our brainpower, so that's a net gain from our point of view. What is more, for some weird reason Go code seems to be less susceptible to bitrot (go fmt FTW).

pifflesnort5• 13 years ago

> But it didn't touch the important question -- is the development time for Go faster than for scripting languages with critical sections in C?

"Writing critical sections in C" is a terrible cop-out of the Python community, by which they try to make the engineer do the compiler/VM's job.

Compare:

1) Write algorithmically efficient code.

2) Profile for hotspots.

3) Lift out the hotspots by rewriting them in C.

4) Repeat throughout the lifetime of your project.

Versus:

1) Write algorithmically efficient code.

2) There's no step two.

What winds up happening is that A) nobody does steps 1-4 with any regularity, and B) The bar at which a C rewrite is set very high, while Python is very slow across the board. The whole system winds up slow in aggregate, with only a few hotspots optimized, if any.

voidlogic5• 13 years ago

Since the author uses Go as a basis of comparison. I thought these benchmarks from The Computer Language Benchmarks Game might be interesting:

Go vs. Ruby: http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...

Go vs. Python: http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...

Note: These results are from the 64-bit single core machine since it has already been updated to Go 1.1 beta 1. There is also a faster Go regex program (#7) that hopefully can be made to work again after 1.1 is released (in place of #1).

Although still using Go 1.0.X ,the recent TechEmpower benchmarks might be interesting if you have not seen them: http://www.techempower.com/blog/2013/03/28/framework-benchma... (It looks like Go 1.1 might be 3x faster: https://gist.github.com/errnoh/5320784)

P.S. As always your own program is always the best benchmark to use, your millage may very, take your gain of salt, etc etc.

yew• 13 years ago

Probably worth noting that the presentation that the author is discussing is less about 'which language is faster' and more about what sort of abstractions different types of languages encourage.

The presenter explicitly takes the position that Go and Python (actually PyPy) should be equally fast given the same sort of operations. Just that Python's idioms tend to be less efficient. He's a PyPy developer too, so I would think he would know something about what's going on under the hood. (He also talks a bit about why the VM doesn't/shouldn't try to optimize these cases automatically. Worth a look, it isn't long.)

ealloc• 13 years ago

Good summary. But I disagree with the author that immutable strings and ubiquitous hashes are merely unfortunate idioms of python - to me python is about simplicity, and those things are what make python conceptually simple. I don't have to worry about clobbered buffers, nor about datastructure semantics besides simple hashes.

acqq• 13 years ago

On the same site, the advantages of Go vs good dynamic language implementation like JavaScript V8 aren't so impressive:

http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...

melling• 13 years ago

Being 2-10x faster and using a lot less memory isn't impressive? I'm sure the Go optimizer still needs a lot of work. It's going to get much better. The V8 optimizer is great but we're probably closer to the limit of how much more can be done than with Go.

voidlogic5• 13 years ago

Those results are from the x86-64 machine with one core (because the multi-core 64-bit machine is still running Go 1.0.3). Once Go 1.1 is realized and the multicore machine is updated, run the comparison again there. Javascript fairs even worse against Go when multiple cores are available. Gos support for parallel processing blows JS away.

joe_the_user2• 13 years ago

"Programmers of elegant scripting languages desire similarly elegant looking code with flexible data structures even where no flexibility is in play. Some who know better will understand that technical limitations make the difference between hashes and other types of named lookups essentially nil for many popular language runtimes."

OK, if no flexibility is in play in many usages of hashmaps, should not an intelligent interpreter/compiler be able to detect this and modify the use of the hashmap accordingly? Maybe even use a different lookup system in those cases? It seems like that would be ... elegant.

The reason VMs don't need to be slow is that you don't need to treat an eval call literally, you don't always need to create a full environment to parse a twenty character string, right? But if you can get around that, should you not be able to get around hash-map's problems? Am I missing something.

cjh_3• 13 years ago

This question is partially address in the articles linked to at the start [1]

I linked to the slide that begins to mention it, rather short though.

The answer is that we can add heuristics, but that we should also be able to solve this somewhat on a language/library level rather than only at a vm level.

[1] https://speakerdeck.com/alex/why-python-ruby-and-javascript-...

joe_the_user2• 13 years ago

Yeah,

Perhaps if I saw/heard the talk I would get more out of it than "data-structure heuristics are the worst". It would be nice to know why they are the worst.

makomk• 13 years ago

Python already has a feature called __slots__ which replaces the dictionary normally used to store object attributes with fixed, well, slots.

jes51997• 13 years ago

Doesn't the V8 javascript engine do something like this? Where it builds C-structs instead of hashmaps if it can detect that your objects are being built in a consistent way

maggit• 13 years ago

Yes. They call it "hidden classes": https://developers.google.com/v8/design#prop_access

gngeal2• 13 years ago

Self-93, the language that pioneered this approach for dynamic native compilation, celebrates it 20th birthday this year.

gsnedders5• 13 years ago

So does PyPy, but only for objects (and not for hashmaps, i.e., the dict object).

Too• 13 years ago

IIRC the Dictionary<TKey, TValue> in .NET does this. If you have less than 8 items in it a lookup is simply iterating over the keys like a plain array instead of keeping a hashmap.

jeltz• 13 years ago

So does ruby Hash in 2.0 if I recall correctly.

d0m• 13 years ago

Time of "Getting that shit done" and cost of long-term maintenance are two of the main reasons why people use Ruby/Python, not for the raw performance. I thought that was an old debate. And anyway, from my experience, Python has never been the cause of the slowness.. It's more about tweaking that DB query, smartly packaging the assets or using a better algorithm. (I know it's not always the case. I.e. If my program has to run in the space where bits could be randomly shifted, some additional thoughts would be needed, obviously.)

jacobmarble• 13 years ago

True, the author doesn't really address "effort to get shit done", but Others have commented on this and there seems to be consensus around "Go is a very human readable/writable language".

Anyone know of a human read/write-ability comparison of programming languages?

zachrose• 13 years ago

Depends on the human! (One end of the spectrum is a DSL customized to the concepts and patterns of domain experts.)

antiterra• 13 years ago

Isn't that a useless comparison? Chinese, Tamil and Estonian are all extremely human read/writeable, so much that children can write them. However, it's likely at least one of them is indecipherable to you.

Perhaps the metric you're more interested in is the difficulty in learning to read/write decent code in a particular language. Of course, this difficulty can vary based on what other languages the learner is already familiar with.

jes51997• 13 years ago

Yeah: your real bottlenecks are going to be 1) getting data out of your database and 2) sending packets across the internet. These are 100 times and 1000 times slower than your program, the 10x penalty to use ruby/python instead of C/C++/go/rust isn't going to matter.

theoj2• 13 years ago

You conveniently forgot a key bottleneck: the amount of RAM on your server. That's what determines how many concurrent sessions you can keep running before performance degrades and you need to add another server.

anoncow2• 13 years ago

Is Go fast AND human friendly?

ansgri• 13 years ago

Well, you're talking about making easy things even easier (CRUD web apps connected to DBs; the inherent complexity is low). That's very important for horizontal technology expansion, i.e. 'easy IT everywhere'.

For vertical progress, i.e. making complex things possible, these micro-bottlenecks are important. These same DBs with their query processing and other domains with extremely high inherent complexity would greatly benefit from high-level languages yet still the most difficult parts have to be written in the already difficult C(++).

coldtea10• 13 years ago

>Yeah: your real bottlenecks are going to be 1) getting data out of your database and 2) sending packets across the internet. These are 100 times and 1000 times slower than your program, the 10x penalty to use ruby/python instead of C/C++/go/rust isn't going to matter.

Only if you're talking about the typical CRUD, website, etc program.

Why should we? People use Python for numerical analysis, scientific computing, etc. Why not let them be able to do that, while still writing in Python (instead of using libraries that delegate to C/Fortran/etc)?

And even a website can do intensive computation work. Why HAVE to write it in 2 languages?

pifflesnort5• 13 years ago

> Time of "Getting that shit done" and cost of long-term maintenance are two of the main reasons why people use Ruby/Python, not for the raw performance.

Plenty of other languages "get shit done".

> And anyway, from my experience, Python has never been the cause of the slowness.

Then you're not actually performing performance analysis. Python is S-L-O-W.

I've written large systems in Python. I've profiled them to find the hotspots. I've micro-benchmarked them against implementations in other languages (C, Java). Python lost. Always. By a lot.

So then I had to rewrite critical sections of the Python in C (or accept that we'd need 32 servers instead of 2). Eventually I stopped bothering, and used a different language, because the aggregate cost of a thousand small performance issues (and a few big ones) is actually quite high.

scott_w• 13 years ago

I think you missed the point. There are plenty of use cases where Python isn't the cause of slowness.

It doesn't matter if C is 100x faster than Python if your app is running 49/50 seconds running SQL queries.

nostrademons3• 13 years ago

"So then I had to rewrite critical sections of the Python in C"

I think that's kinda the point; rewriting critical sections in C is pretty much Working As Intended. That's why it's a scripting language. The point is that you didn't have to write the whole thing in C.

vidarh5• 13 years ago

> Then you're not actually performing performance analysis. Python is S-L-O-W.

You totally miss the point. There are large classes of systems where CPU time in your app is not a significant factor. E.g. my go-to example of this is the first Ruby project I deployed in a production environment:

A queueing middleware server. If we served the queues entirely out of memory we could serve a few million messages a day out of 10% of a single 2005-era Xeon core (and while Ruby doesn't scale across cores, this system could easily spread load by simply running multiple queue processes).But of of that 10%, 90% was spent in kernel space. So even if switching to C (or whatever) were to speed up the user-space part of the code 90%, it would only cut CPU time spent from 10% to 9.1%.

This is fairly typical. If your code is CPU-bound, and it's not CPU bound because of stupid algorithm choices, then things can look very different. The first thing to do is find out if you're CPU or IO bound.

pointyhatuk6• 13 years ago

Actually I find the opposite. Python is easy for quick wins but from a maintenance, performance and reliability perspective it rapidly falls off a cliff. If your code propels your business then these concerns need to be considered up front.

Even off the shelf functionality doesnt have the same time/complexity properties as say c#, java or go resulting in a fairly basic algorithm reaching ridiculous time constraints very quickly.

As for CRUD type apps, you are probably right but its the bits that aren't CRUD which add the most value to your product.

nilliams3• 13 years ago

If there really are business-critical parts of the app that are too slow a good developer will drop down to C (as noted by others). That doesn't negate the reasons for using Python for the rest of the app.

Your mentions of maintenance and reliability in the opening sentence are unfounded. What makes a Python app less so than the same app written in any other language? Surely it comes down to the developer. A well-factored app written in Python (with tests) is going to be far more maintainable than a big ball of mud written in C. The choice of language has little to do with it.

danenania2• 13 years ago

"As for CRUD type apps, you are probably right but its the bits that aren't CRUD which add the most value to your product."

I'm sympathetic to your point, but I don't think this is necessarily true. What adds the most value to a product is determined by economics. It has nothing to do with whether an app is doing basic crud or nlp wizardry underneath, just whether it does something people find useful. There are tons of basic crud apps out there making people millionaires.

coldtea10• 13 years ago

>And anyway, from my experience, Python has never been the cause of the slowness.. It's more about tweaking that DB query, smartly packaging the assets or using a better algorithm.

That's because you constrain your use of Python in I/O bound problems, like webpages and such. If you had any CPU bound problems, then you would have found Python is quite slow -- and you can get 2-20-100 times the performance in another language.

Python is very slow (compared to what it can be), which is what necessitates tons of workarounds like: C extensions, SWIG, NumPy and SciPy (C, Fortran, etc), Cython, PyPy, etc. A whole cottage industry around Python's slowness. Which is nice to exist, but not an ideal solution: suddenly you have 2 codebases to manage in 2 different languages, makefiles, portability issues, etc etc. All of those are barriers to entry.

pointyhatuk6• 13 years ago

This.

Integration is hell. This is a universal fact.

I'd rather pick a tool which doesn't add integration requirements!

pwang2• 13 years ago

There are definitely limitations of the implementation of CPython itself, which do not need to be there if the language were willing to ratchet down the dynamism in certain cases. (Even without that, there are potentially implementation avenues that would have permitted better concurrency than the current codebase.)

However, implementation aside, I think it's invalid to assert that the existence of a cottage industry is inherently a sign of weakness; it's also a sign of just how pervasively Python is used. Find me another language that's used as widely, as popularly (i.e. not just two random guys using it that way), and in as many different kind of settings as Python. (Java might be a contender, except that it's not really used for the really high-performance scientific workloads like Python.)

RegRegReg2• 13 years ago

Exactly, if you want shit done you need to use Ruby/Python. If you want actual stuff done you use something else.

YesThatTom2• 13 years ago

Wouldn't adding goland-style slices to Python be awesome?

kkuduk• 13 years ago

well, there are memoryviews, but they could have been designed better

travisoliphant5• 13 years ago

Memoryviews are the step-child of NumPy arrays. The buffer-protocol was the real intent and the memory view the forgotten "example" until a few brave and noble Python devs rescued it from obscurity. Memory-views are not that useful to one who will always have NumPy installed.

almost• 13 years ago

It's not quite the same but some of the use cases (and a bunch of other stuff) would be covered by numpy arrays. You can create multiple views on the same in memory data with different start and end points and different strides.

dagw• 13 years ago

Agreed. I think numpy arrays a rather overlooked by most python programmers. They allow all kinds of useful tricks like that that are quite handy even if you're not doing strictly numeric work.

alexchamberlain• 13 years ago

On the to garbage collect or not issue briefly mentioned at the end, the best reason to use an unmanaged language is predictability of performance. You know the GC won't step in at a critical moment. I regularly get frustrated by Eclipse becoming unresponsive for several seconds.

pjmlp5• 13 years ago

There are real time GCs for many languages that compile to native code.

The way Eclipse is coded is not a good example of GC performance.

pjmlp5• 13 years ago

Overall nice article, it just looses a bit by focusing too much on Go vs Python issue.

1 - The are quite quite a few strong type languages with native code implementations since the mid-80's, even GC enabled, Go is not the only one.

2 - There are dynamic languages like Self, Dylan, Lisp even the young Julia that have implementations which achieve C like speeds when compiled to native code, especially when the developers make use of type annotations and use the right data structures.

Self is specially important given that the research work, in a dynamic language, ended up being the heart of Sun's JIT compiler.

batgaijin• 13 years ago

Yeah I don't sympathize with ruby/python/go people. CL can be interpreted & compiled. Why the fuck is it okay to literally choose the worst options just because people don't like seeing an explicit AST?

pekk7• 13 years ago

Why specifically are these "literally the worst" options?

mratzloff2• 13 years ago

Because you can hire for those languages, but good luck hiring for CL.

mwcampbell• 13 years ago

Yes, there have been many other languages with ahead-of-time compilation and garbage-collected runtimes, but Go is one of the few that might be considered popular today. At this point, one might invoke the old chestnut, "If you make technology choices based on popularity, you might as well go with (Windows|PHP)." But there are degrees of popularity, and there are advantages in choosing something that's at least moderately popular, e.g. a large selection of ready-to-use libraries and a pool of developers who don't need to learn a new language on the job.

Moto7451• 13 years ago

I've noticed this in some code I've been refactoring at work recently (a feed generation system). It's easy in Perl to suck in a large amount of data into one array and map/grep it into another array of hashes and output filtered/formatted data in a few lines. The problem of course is that your business'/project's own growth overwhelms the algorithm as the data structure balloons. This is further compounded by libraries which are designed to accept an array as input (reinforcing the idiom causing the first problem) since you end up with thousands of additional iterations.

I refactored the code to lazy load each item in the collection and map/filter/generate the XML at the end of each node's iteration. The end result is a nice flat memory space (save for the odd leak here and there) that's about 1/10th of the original program's, at the 'expense' of only a few more lines of code.

RegRegReg2• 13 years ago

The biggest confusion is seeing Python/Ruby/JS as a programming languages. They are not. They are scripting languages. Something you put on top of your core code.

pekk7• 13 years ago

Does the phrase "scripting language" have any hard technical meaning or is it just another subjective label?

voidlogic5• 13 years ago

I've always informally thought of a scripting language as anything the executes by interpreting source code or bytecode (lacking JIT compilation).

doktrin• 13 years ago

This might make sense if you contrasted "scripting" to something a little more specific, like "system" development. However, creating a "programming" v. "scripting" dichotomy makes no sense IMHO. Ruby & Python are programming languages that can also be considered, informally, scripting languages.

corresation3• 13 years ago

Both the submission and the target of the submission dismiss dynamic typing as a source of slowdowns (in the unfortunately haughty, completely unsupported manner that is too common in the industry. It is, in effect, the head-in-the-sand approach: If you don't want something to be true, simply keep saying it isn't and somehow that will become reality), yet I guess we need to define what "slow" is then, using as an example an empirical test (ASM.JS) that certainly serves better than some people waving their hands about innovations in JIT compilers.

The primary optimization that ASM.JS brings is its static type system. With that, and the obvious benefits to the code path, intensive activities can be run at 1/2 native speed, versus the 1/12th of so native speed available with dynamic typing.

Is 1/12th native speed "slow"? In most cases -- where the code is the duct-tape that glues together native code (which is the case on web servers, admin scripts, etc), no, it isn't slow at all. But from a basic optimization perspective, asm.js sure seems to indict dynamic typing if you really care about squeezing every cycle's maximal output.

gsnedders5• 13 years ago

No, the primary optimization asm.js brings (at least in OdinMonkey) is AOT compilation.

Fully typed code should produce the same code from IonMonkey without OdinMonkey: the problem is you have to get up to 10k calls (at least last I looked!) of a function before you get there; but once you are there it is just as good. It's the build-up, through the interpreter and the less efficient code-generators (in terms of code generated, that is; they're more efficient in terms of their own runtime), which really hurts.

corresation3• 13 years ago

No, the primary optimization asm.js brings (at least in OdinMonkey) is AOT compilation.

But it can do that precisely because of the static typing, which loops back to exactly what I said. Looser, more dynamic language always have a cost associated with that flexibility.

munificent• 13 years ago

> The primary optimization that ASM.JS brings is its static type system.

Well, that and not having objects, user-defined types, closures, strings, polymorphism, or even garbage collection. asm.js is fast because it's essentially BCPL, not just because it has a static type system.

NateDad3• 13 years ago

Everyone keeps saying "Use Python and write the slow parts in C"..... except that writing GOOD (secure, correct, safe) C is really hard. That's actually the main reason why the Go authors came up with Go. Not to replace Python etc, but to replace C/C++.

I'm kind of surprised that all the "Python: let's get stuff done" people are suggesting writing a whole bunch of code in C, one of the least "Getting stuff done" languages there is.

For those who have asked what the productivity of Go is like vs. Python + C: Go is very productive. When newbies to Go say it's "verbose" they mean "I have to write out some 3 line loops that in other languages are one line". If that's the major cause of slowness in my development cycle, I'd be ecstatic.

Go is also very easy to write "correct" code in... no buffer overflows, no memory leaks, no access of random memory, no unnecessary allocations, and a very clear and defined behavior from clear, concise code.

pekk7• 13 years ago

Nobody said you have to write any C. If you are freaking out about performance, the option to write some parts in C is valuable.

gsnedders5• 13 years ago

Given the original presenter is one of the major contributors to PyPy, I'm fairly certain that wouldn't be his first advice: I'd expect it'd be to try PyPy (and file a bug if it's slower than CPython!).

Also note on the last slide: "Stop writing C extensions, use something like cffi". There's two (really) good reasons you should do this: firstly, your code is no longer tied tied to CPython, so using alternative Python implementations (like PyPy) becomes viable; secondly, you don't have to use C — you can trivially use anything that exports functions using the C ABI.

officemonkey• 13 years ago

>"I have to write out some 3 line loops that in other languages are one line"

I think every vim, emacs and decent IDE user would say "I has a text editor. Your argument is invalid."

stouset• 13 years ago

On the flip side of this argument, code should be optimized for reading, not writing. Short, idiomatic code IMHO optimizes for reading. Code like

  array.map(&:to_s)

parses instantaneously. Whereas the procedural equivalent

  strings = []
  for (element in array)
    strings.push element.to_s
  end

requires scanning through the loop to extract out the logic specific to this case. And it's easy to accidentally overlook something when mentally ignoring the "boilerplate".

Not to make a mountain out of a molehill, and not commenting on Go specifically, but the extra "cost" of this style of code adds up when the language itself gets in the way of using high-level abstractions. Methods that are three meaningful lines long are inherently more understandable than equivalent methods that are twelve. Multiply this cost across an entire class, file, and/or project, and it can have a significant impact on one's ability to grok the expected behavior.

lobster_johnson• 13 years ago

The problem with redundant code is usually not editing, it's bugs. More code statistically corresponds to a higher number of bugs.

MostAwesomeDude• 13 years ago

Please don't say that "all" of us endorse writing C. Many of us gave up on writing C a long time ago and want to see pure-Python solutions to things.

klrr• 13 years ago

Go is by far a replacement for C. C was made as a language for writing operating systems. C++ tried add more abstraction making it a better choice for more regular applications. Go was made for writing distributed systems.

hogu• 13 years ago

I've had pretty easy success moving my bottlenecks into Cython. With the development of Numba, https://github.com/numba/numba, this is going to become even easier(I work for the company that produces Numba, but I do not work on Numba). I think one thing the scientific community does right, is that we keep our data in NumPy arrays. This makes it really easy to write extensions in other langauges because NumPy arrays are very transparent and easy to access in other languages (they're just chunks of memory with dtypes, and a shape)

quantgsm• 13 years ago

> I work for the company that produces Numba, but I do not work on Numba

In experience Cython and C++ extensions are still the best way to speed up existing Python algorithms. Initially Numba showed great promise but after attempting to jit compile several simple hotspots ( cryptography and hashing functions primarily ) in my application I discovered lot of bugs and apparent numeric instability particularly in the trigonometric and bit functions. Have these issues been resolved? For instance for Diffie–Hellman calculations?

kalkin• 13 years ago

It's too bad that this discussion is ending up mostly in language wars, because Alex' talk was about how to improve the situation, not how to prove that your favored technology is always the right tool for the job and everybody else is wrong.

Let's say that you're doing some NLP and you want to pre-process your strings in Ruby because its easy and elegant. Idiomatically you might write something like "my_objs.map(&:to_s).map { |s| s.gsub(/\W\S/+, '') }.reject(&:blank?)" - perhaps broken up into multiple lines - but you're allocating three arrays and re-allocating every string. And in real life you're probably doing more pre-processing steps which involve additional arrays and possibly additional string re-allocations.

You could switch to gsub! which does the non-word stripping in place, but it "Performs the substitutions of String#gsub in place, returning str, or nil if no substitutions were performed", so you're adding an extra conditional. And it's going to be harder to avoid re-constructing a bunch of arrays.

There's no intrinsic reason you couldn't have basic Ruby language APIs which let you write code which chains operations just as naturally and obviously, but don't wastefully re-allocate. Think of how ActiveRecord relations defer the actual SQL execution until it's unavoidable.

(Haskell-style laziness might solve the main issue above, but I've run into analogous unavoidable-allocation problems with Haskell. If I'm depth-first searching a board game state-tree I shouldn't have to re-allocate the whole state representation at each step, but it's pretty hard to avoid while writing elegant Haskell since modifying memory in-place has side-effects.)

I don't know Go, but maybe instead of talking about why people who still use Python or Ruby are or are not stupid, we can talk about what it gets right that other languages can adapt?

bm1362• 13 years ago

I'm currently hitting some performance issues trying to use Python to make a naive physics engine- does anyone have any reading recommendations on best practices for python performance?

travisoliphant5• 13 years ago

I'd love to see how Numba helps you. Give it a try: http://numba.pydata.org

Slide deck: https://speakerdeck.com/pyconslides/numba-a-dynamic-python-c...

IPython notebook examples: https://www.wakari.io/nb/travis/numba https://www.wakari.io/nb/travis/Using_Numba

plg• 13 years ago

the endless debate continues...

choose two:

a. fast execution of the code on the machine

b. fast code generation by the programmer

c. fine-grained control over every aspect of the code

I know which two I'd rather have, esp in the long run.

travisoliphant5• 13 years ago

Numba is trying to let you make a different choice for different sections of the same program so that you can do different styles of programming in code meant to be a library than you might do in other sections of code meant to be more "ocassional programmer" facing.

Some of you might find the roadmap particularly interesting. It's an open-source project eager for input from others...

http://numba.pydata.org/numba-doc/dev/roadmap.html

Xcelerate• 13 years ago

This thread has got me cracking up. I'm sure all of you are very smart, yet you all treat each other as though the other person is a stupid, incompetent programmer. It's a trend I observe everywhere in online programming communities.

teeja• 13 years ago

But all the others are stupid, incompetent programmers. It started at MIT.

Just kidding! (sort of). But have a look at the Jargon Appendix (http://catb.org/jargon/html/pt03.html) - especially The Story of Mel (http://catb.org/jargon/html/story-of-mel.html).

g2e• 13 years ago

The video has been posted for anyone interested:

http://vimeo.com/61044810

ilaksh3• 13 years ago

I am sure I am just going to have this comment buried because people don't like to have Python and Ruby compared unfavorably, but I have to try to help people out.

Looks like a lot of people STILL haven't caught on to how big of a performance advantage Node.js's V8 engine gives you over Python and Ruby. Look at benchmarks like this one: http://fengmk2.github.io/blog/2011/fibonacci/nodejs-python-p...

Most of the time modern JavaScript is compiled to native code. Its much faster than you realize.

The cleanest code is a new language called CoffeeScript which compiles to JavaScript. Its more Pythonic than Python. Its the closest to pseudocode that you can get (especially if you avoid a few Rubyisms).

The best APIs available are in Node.js modules (see npmjs.org). Best meaning most advanced API design in that the modules are focused, decoupled, very straightforward. This is also by far the best design for package management and dependencies since it is very open and flexible.

On top of that, Node.js is built from the ground up to support highly concurrent input/output. It just amazes me that people haven't figured out what a big advantage Node.js and CoffeeScript provide over Python or Ruby.

I actually think that a big thing holding people back is that Node.js and CoffeeScript make things so much easier, people think its cheating, and are afraid their programmer friends will think less of them if they take advantage of something that really simplifies their jobs.

TazeTSchnitzel3• 13 years ago

CoffeeScript is horribly... stabby? Its ambiguous syntax makes it easy to hurt yourself.

And node.js, while great, is not magic.

I would also much rather use Python's classes than do JS OOP or use CS classes. That said, I use node.js more often than Python.

ilaksh3• 13 years ago

How much time have you spent using CoffeeScript? It might take a week or two to get used to it. I think if you just stick with some basic stuff and write it like Python with less punctuation then it is pretty straightforward and CS classes are very straightforward, what's wrong with them?

Just leave out the outer parenthesis and put them in after that.

kragen2• 13 years ago

I've been programming in JS since the previous millennium (compilers, parser generators, 3-D engines, browser extensions, web servers, Comet in 2000) and I've used Node on and off for a few years now. V8 is pretty awesome, but outside of a relatively narrow niche, the advantage of Node doesn't seem that huge to me.

* Yes, your code runs a lot faster in V8 than in CPython or Ruby, maybe a factor of 3 to 10 slower than C (http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...) instead of the factor of 30 you get with CPython or YARV (http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...) or the factor of 100 you get with MRI Ruby. But much of the time, if a factor of 30 is a problem, so is a factor of 10! And there are plenty of language implementations that get closer to C than V8 does, without being as error-prone and verbose as C (Java, SBCL, Ada (just kidding), Haskell, OCaml, Scala, occasionally LuaJIT). PyPy has similar performance overall to V8, but a different performance profile. See http://speed.pypy.org/ for details.

* JS is a pretty reasonable language, and CS is even better, despite its lack of getters and setters, its stupid Ruby-derived inclusive-bounds ranges, and a few other minor warts. But it's not nearly as clean as Python or as flexible as Ruby.

* Node makes you rewrite your program into explicit continuation-passing style every time you do I/O, in order to maintain responsiveness. Also, every time you do a potentially long computation, unless they've finally gotten around to adding Web Workers to Node. I've done this in order to get massively scalable I/O in high-performance systems in C, Python, and JS, and it is my considered opinion that it is sometimes not the best way to do things. In cases where this is the best option, Python has Twisted and Tornado, which give you the massively-scalable nonblocking goodness while they still let you run other threads if that's what you think best.

* npm is a pretty amazing ecosystem, but the Python, Ruby, and C ecosystems are orders of magnitude bigger. On my netbook right now, among other things, I have Python bindings to 0MQ, D-Bus, GConf, TidyLib, DNS, GObject, CUPS, Cairo, and the GNOME keyring. Of these, 0MQ, D-Bus, DNS, and Cairo have bindings in npm, while GConf, TidyLib, GObject, CUPS, and the GNOME keyring do not. In realms like natural language processing, symbolic mathematics, and numerical computation (e.g. DSP), there's nothing in the JS world that comes close. Note that this objection applies also to PyPy, since many Python bindings do not yet work with PyPy.

PyPy also offers massive multithreading (a la Erlang, Golang, or rust) and sandboxing (the ability to run untrusted code safely, at least in theory); it contains a generalized JIT compiler generator that allows you to write JIT compilers for other languages very easily; and the PyPy people are working on software transactional memory and dynamic code generation for numerical algorithms.

In summary, I agree that Node is pretty awesome, especially, as you say, coupled with CoffeeScript, and also when you want to share code between browser and server. But there are still valid reasons for choosing other alternatives.

ilaksh3• 13 years ago

I don't see Python as being cleaner than CoffeeScript. CS is very similar but with less punctuation. How do you figure that one?

Have you tried using CoffeeScript indented with two spaces, or using the async module? You don't always have to do continuation passing, and I feel like Node.js's way is a lot more straightforward than anything like Twisted.

What's Going On (on "Why Python, Ruby, and Javascript are Slow")

Related Stories