What languages and tools do you use for cluster computing?
https://news.ycombinator.com/item?id=309300I've recently come into possession of an account on a research computing cluster. This is for bioinformatics work at a university. For the past few years I've been interested in parallelism-friendly functional languages like Haskell, but in my actual work I haven't come across anything that C, Python and Bash can't handle easily. So this seems like a great opportunity to try the weird stuff.
Most scientists seem to get by using MPI with C, C++ or Fortran. Naturally I'll learn how to do that, too, but I strongly suspect that this traditional route tends to result in sprawling, complex code that's difficult to debug. There must be a better way. Ocaml and Python have MPI interfaces, too; Scala seems to be designed for distributed computing on existing platforms.
And the platform is a limitation, too. The interactive node on this cluster shows a few stable packages to work with: GCC 3.4.6, Java 1.6, Python 2.3 (with a broken numpy), and of course the MPI suite. Nothing exotic; CVS and SVN for version control. I think this means Erlang and Haskell are ruled out, unless they're compiled to C or JVM bytecode first -- any VM must be available on all the nodes if the code's going to run. Right?
So, what do you use? Or, what would you use in this situation?