Ask YC: What Language Do You Use for Multiple CPUs?

https://news.ycombinator.com/item?id=81707

by bcater • 19 years ago

5 22 19 years ago

news.ycombinator.com

bcater• 19 years ago

I was hoping that Python threads split across processors, but it seems that they don't. Is there a language that does it "automagically" without having to explicitly code it?

jsjenkins168• 19 years ago

Your OS process scheduler should do it to some degree at runtime, but beyond that there is not a lot that can be achieved without explicitly handling synchronization and locking of shared resources. I know there are some startups working on parallel processing optimizations though, so there could be some new developments in the future. Google has been acquiring a few of these startups recently..

cpinto• 19 years ago

See if the processing package is useful to you: http://pypi.python.org/pypi/processing

jey• 19 years ago

No, the standard Python interpreter kind of sucks at this. You might want to look at Stackless Python though, it has all sorts of neat concurrency primitives. http://www.stackless.com

euccastro• 19 years ago

Stackless is great, but it doesn't help much with multiple processors.

astrec• 19 years ago

Have a look at Erlang. Actually, have a look at functional languages in general as being (mostly) side effect free they often have primitives for parallel processing.

Another pretty exciting language you might to look at is F# - presently running under Mono on my macbook.

Finally, if you love your python (who doesn't? ;)) - see http://www.parallelpython.com/ for some interesting work.

marcus• 19 years ago

I love python and use it for many of my prototyping needs but if you really need that extra juice from your extra processors you are probably using the wrong language for the problem.

From my experience average increase in speed gained from porting to C/C++ (even without using multi processors) is about x60-x100.

nevinera• 19 years ago

unless your application is very unusual, you could've just ported your < 10 bottlenecks and made the same gain. (and x60? right..)

icky• 19 years ago

> I was hoping that Python threads split across processors, but it seems that they don't.

Understanding concurrency in Python: They're called process-ors, not thread-ors... ;-)

boucher• 19 years ago

Java, C/C++/Objective-C. Many others.

vmcodes• 19 years ago

I believe Erlang does.

sohail• 19 years ago

I've used C++ for multiple CPUs and multiple machines. Quite easy really. Most people just make it too complicated I think.

downer• 19 years ago

C and C++ using shared state, but minimizing locking by being clever (e.g., subdivide first).

dfranke• 19 years ago

Haskell is excellent for parallel processing -- probably as good as Erlang if you're just talking about multiple cores on one system and not distributed processing.

carpal• 19 years ago

Ruby/Rails/Mongrel/Pound does a great job of utilizing all those CPUs that I have in a production environment. There's a bunch of overhead, but it is easier than having to deal with locking.

It really depends on your application, though. If it isn't a web application, and state matters, then you're going to have to go elsewhere.

ivankirigin• 19 years ago

Design software to run in multiple processes & threads, and you'll be fine for both multi-core and multi-cpu.

Most interesting systems have multiple dynamically interacting processes, but this might be a bias of mine from the robotics world.

Elfan• 19 years ago

I just want to second what what several others have implied about Erlang. It makes the actor model feel right to the point that the fact that your program can run on multiple cores/cpus/computers is seems incidental.

crucini• 19 years ago

The real question is, "what process model do you use for multiple CPUs?" Assuming a Unixy OS, you can use threads or multiple processes.

If you choose multiple processes, you have to intelligently use IPC to coordinate them. But you can happily use pretty much any language.

If you choose threading, scripting languages are problematic. For example, Python has a Global Interpreter Lock. In that case, I think the only "safe" choices are C++ and Java. Java's safe because its "synchronized methods" enforce locking, often in a wasteful manner. C++ leaves the entire synchronization problem in your lap, giving you infinite room for performance and for subtle and maddening bugs.

Multi-process is generally a better approach than threading. I've seen too much time spent debugging threaded apps to ever recommend that approach. Every hour poured into debugging a deadlock, or worse yet, a mysterious cross-thread heap trampling is an hour taken away from improving the product.

yariv• 19 years ago

Erlang

queensnake• 19 years ago

Intel has made a library (TBB) and language (Ct) (both based on C++) that tackle different subsets of how you might want to use parallelism.

http://blogs.intel.com/research/2007/11/q4_itj.php

ayc• 19 years ago

SP-Forth (http://spf.sourceforge.net/) to program multithreaded applications, such as acWEB, acSMTP [.sf.net], Eserv/3 ( http://www.eserv.ru/ ) internet servers.

zurla• 19 years ago

lolcode

Ask YC: What Language Do You Use for Multiple CPUs?

Related Stories