Julia: A Fast Language for Numerical Computing

https://sinews.siam.org/DetailsPage/tabid/607/ArticleID/744/Julia-A-Fast-Language-for-Numerical-Computing.aspx

by boromi • 10 years ago

142 78 10 years ago

sinews.siam.org

sdegutis• 10 years ago

See also, Julia's home page: https://julialang.org/

I toyed around with it for a while, and really liked it. I wondered to myself "what makes this language specific to numerical computing? it seems generic enough to be used wherever I'd want to use Ruby or Haskell..."

But then I tried using it. And even though the language is generic enough, the standard library just plain assumes you're only doing numerical computing, and makes everything else needlessly hard by trivial omissions.

That was like 2013 though, so maybe things are different now?

gravypod2• 10 years ago

I was really REALLY interested in the language due to the clustering ability and their JIT they build into it. But it does seem like the language was build without any systems-usage in mind. How sad. Nothing will ever quench my thirst for clustering.

vegabook• 10 years ago

so... it was built for clustering but it doesn't quench your thirst for clustering? Please could you clarify.

KenoFischer6• 10 years ago

While numerical computing is the first target, that's definitely not the sole goal of the language, so if there's something simple that you feel is missing, I would be very curious to know what that is.

skybrian• 10 years ago

It seems to be offline.

KenoFischer6• 10 years ago

HTTP site is up. It's hosted on Github Pages so HTTPS may not work.

noam87• 10 years ago

Interesting, I've had the opposite reaction. I find it to be about the nicest programming language to work with. I even started writing a little toy database in Julia, and find it really easy to reason about.

Package management system is its only weak point IMO. It makes a lot of counter-intuitive decisions (having to build a package from the packages directory? that's just strange). Cargo / Bundler to me feel like the gold standard in package management... most alternatives I've seen would be better off just copying those workflows unless there's a really good reason to try something new.

jernfrost• 10 years ago

I haven't used Julia for any big projects but I routinely use it for small projects or utilities. I have used it for: 1) Obfuscating source code 2) Reading and presenting iOS provisioning profile data 3) Simple social science related simulations ported from Ruby or Python. 4) Code completion plugin for TextMate etc

So no huge projects, but I found Julia to be a very nice and versatile language to work in. I find I can write code which is clearer than its corresponding Python and Ruby version and which has less bugs because type info is a bit more explicit, but without getting too much in the way as is typical with statically typed languages.

I tried Haskell for some of the same projects, but I got to say Haskell takes way longer time to get proficient in. For the stuff I did I also found the way the Julia type system worked, made code reuse much easier than Haskell.

E.g. with the Julia Union type it was possible to define individual function across multiple specific types. E.g. I could make a particular implementation of f apply to type A, B and C, while another particular implementation of f applies to D and E. Function g could have a specific implementation of A and E. While another implementation of g is used for B and C.

bitL• 10 years ago

Just a question - I completely lost interest in Julia when I read it adopts many of the ugly language hacks of Matlab - why should I care about Julia when I can feel frustrated with Matlab already?

kafkaesq4• 10 years ago

Such as? I'd be curious.

rspeer2• 10 years ago

The literal syntax that you'd think would make an array of arrays concatenates the arrays instead. Sometimes. It depends on types. (Or have they fixed that now?)

Also, although this is of course a preference, 1-based indexing. I would rather have compatibility with C and Python and I don't care one bit about compatibility with Matlab.

These decisions make it seem that Julia just aspires to be "open source Matlab" when that already exists. It could instead aspire to be the good programming language that Matlab isn't.

CyberDildonics2• 10 years ago

It was made to be specifically better than matlab and was made in part by people frustrated with matlab. Have you taken a look at the language yourself?

kafkaesq4• 10 years ago

BTW is it just me, or do other people find that the interpreter is very slow to start under older versions of OS X?

That's one of my current obstacles to diving further into Julia.

KenoFischer6• 10 years ago

It used to be a big problem. It's still not super fast (~1s or so), but we've been doing quite a bit of working at making it faster. If it's significantly slower than that, something else must be going on.

kafkaesq4• 10 years ago

Thanks -- I'll definitely be giving it another shot, at some point.

vonnik3• 10 years ago

This is super. Fwiw, Java and Scala aren't bad for numerical computing either: http://nd4j.org/ https://github.com/deeplearning4j/nd4j/

namelezz• 10 years ago

How is nd4j compared to Breeze and jblas, which are used in Spark MLlib[1]?

[1] http://spark.apache.org/docs/latest/mllib-data-types.html

vonnik3• 10 years ago

ND4J supports n-dimensional arrays, while Breeze does not. jblas is great but it's not fast enough. We've relied on netlibblas in the past, and we're moving to faster computation libs now. We'd love for Spark MLlib to plug into ND4J at some point.

agibsonccc6• 10 years ago

See my post here: https://news.ycombinator.com/edit?id=11275071

We have this concept of a backend where we had net lib java and jblas, but both are horribly slow/limited and need some updating which is why we added our approach to this.

If a new matrix framework comes out, I will just write a backend for it and allow people to keep the same dsl.

Breeze also isn't usable from java: http://stackoverflow.com/questions/27246348/using-breeze-fro...

FWIW, we also support both row and column major (you can specify the data as well as ordering) very similar to numpy.

If you come use nd4j, it will mainly be for a dsl that encourages vectorization just like any other numerical language.

noobermin2• 10 years ago

Java certainly has obtained a bad reputation in the sciences, however.

vonnik3• 10 years ago

Just curious: Why is that?

agibsonccc6• 10 years ago

I agree. A lot of that is because no one has taken the lessons hard learned from every other numerical language and just replicated it (hardware acceleration,BLAS,ndarrays,..)

The JVM has a lot of matrix library fragmentation.

ubasu• 10 years ago

You seem to be conflating "numerical computing" with machine learning. However, numerical computing typically involves solving PDEs via e.g. finite elements or finite differences, or solving large systems of linear equations associated with such methods.

The difference between the two is that when solving PDEs, accuracy is paramount, so even using single precision is a bit of a compromise, whereas in machine learning, the trend seems to be to use half-precision or lower, sacrificing accuracy for speed.

For classical numerical computing, e.g. solving PDEs or linear equations, Java may not be the best choice, e.g. see the following paper:

How Java's Floating Point Hurts Everyone Everywhere:

https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf

agibsonccc6• 10 years ago

FWIW, we use blas just like matlab,numpy,r, and julia.

The speed depends on the blas implementation.

Nd4j has a concept of "backends". Nd4j backends allow us to sub in different blas implementations as well as different ways of doing operations.

We have a data buffer type that allows people to specify floating point or double. Those data buffers then have an allocation type that can be javacpp pointers,nio byte buffers (direct/offheap) or normal arrays

We are also currently working on surprassing the jvm's memory limits ourselves via javacpp's pointers. That allows us to have 64 bit addressing which people normally have access to in c++.

Every current jvm matrix lib that uses net lib java or jblas is going to have problems with jvm communications as well. The reason for this is passing around java arrays and byte buffers is slower than how we handle it which is via passing longs (raw pointer addresses) around that are addressed via unsafe or allocated in jni where we retain the pointer address directly. We expose that to our native operators.

We are solving this by writing our own c++ backend called libnd4j that supports cuda as well as normal openmp optimized for loops for computation.

We also offer a unified interface to cublas and cblas (which is implemented by openblas as well as mkl) FWIW, I more or less agree with you, but it doesn't mean it shouldn't exist.

JVM based environments can bypass the jvm just like python does now.

The fundamental problem with the jvm is no one just took what works on other platforms and mapped the concepts 1 to 1.

Our idea with nd4j is to not only allow people to write their own backends, but also provide a sane default platform for numerical computing on the jvm. Things like garbage collection shouldn't be a hindrance for what is otherwise a great platform for bigger workloads (hadoop,spark,kafka,..)

In summary, we know the jvm has been bad till now - it's our hope for fixing that.

samuell12• 10 years ago

If Julia had Go-style concurrency primitives (channels and lightweight threads), and M:N thread multiplexing, it would be the perfect language to implement my upcoming data flow based scientific workflow language (http://github.com/samuell/scipipe).

Now I'm instead trapped with Go, which lacks a REPL, and leaves a lot to wish in terms of metaprogramming capabilities needed to create a nice programmatic API.

The lack of the mentioned features is my biggest concern with Julia.

StefanKarpinski• 10 years ago

Threading is already an experimental feature and Go-style concurrency will be a standard feature in the future. Well, technically it may be more like Cilk or TBB but it will be very similar to Go.

kkylin• 10 years ago

Cilk-style concurrency would be a really nice addition to Julia.

samuell12• 10 years ago

Interesting.

elcritch3• 10 years ago

I implemented a test of a micro-benchmark posted on HN several weeks back. It took some extra tries but eventually the performance using the existing 'Task' feature was acceptable. Not Go level but decent and give the rate of development on Julia it'd probably be worthwhile to experiment.

samuell12• 10 years ago

That is interesting to know. But, so the Tasks (which is implemented with light-weight threads IIUC) will all run in the same os process/thread, unless you manually create new ones?

wyager• 10 years ago

Sounds like Haskell may be appropriate for your use case. It has green threads, a great REPL, and extremely powerful polymorphism (which provides a safe and clean alternative to e.g. macro/template-based metaprogramming, although Haskell has that too if you need it).

samuell12• 10 years ago

Interesting, didn't know about green threads in haskell (I'm not too familiar with it over all).

erikpukinskis2• 10 years ago

What kind of API do you need that requires metaprogramming?

samuell12• 10 years ago

For example I'd be happy if I could generate structs dynamically, based on string input.

This would mean that we could automatically create true struct-based components with channel fields from the shell like syntax used in the examples in the README (like "echo > {o:foo}" to write to an out-port named "foo"), so that connecting an out-port to an in-port would go like:

Process2.InFoo = Process1.OutFoo

This is not possible with Go's reflection though, so right now, based on these shell like patterns, we can only populate maps (InPorts and OutPorts) of the process, such that the above code example becomes:

Process2.InPorts["foo"] = Process1.OutPorts["foo"]

pjmlp• 10 years ago

You don't need language specific primitives, when a language is feature rich enough, that those features can be done in a library.

samuell12• 10 years ago

Yep, but this adds complexity and fear of maintenance problems, depending on how "close to the core" the library is.

SixSigma2• 10 years ago

You may be interested to read this section on Julia's parallel computing facilities.

http://docs.julialang.org/en/release-0.4/manual/parallel-com...

Which to me as Limbo programmer, look like channels, even across machines.

freyr• 10 years ago

Go is really great for "workflow," but I find it completely lacking for "scientific." Have you attempted this latter part of the project yet? I don't see many examples in your project that attempt scientific or numerical operations.

samuell12• 10 years ago

The thinking is to use external tools for the main scientific parts - hence the big focus on shell support.

This is common practice in bioinformatics already, because of the large plethora of tools which would be too much to rewrite for any particular language.

Then, Go will probably be OK for more mundane tasks such as data pre-processing, filtering, etc etc.

Otherwise, there are in fact some scientific Go libraries already, including BioGo [1] and GoChem [2].

[1] https://github.com/biogo/biogo

[2] http://gochem.org

agentgt• 10 years ago

I don't have knowledge of the "Common Workflow Language" but I am not sure Go's concurrency is really that much of a selling point for composition. Particularly for composition of evaluation since I would imagine how Go does concurrency would bleed through into your implementation code (probably not what you want.. or maybe it is... or maybe it doesn't for you library?). That is I'm not sure Go is really good at composition compared to functional programming languages.

For example one could use a monadic style data structure such .NET's reactive Observable (which has an analog in many different languages) and allow you to compose a stream like definition independent of how it runs. You can then feed this definition (Observable) to an evaluator which could run it on Go using channels or it could run it on Go using a cluster of machines with ZeroMQ.

I think the selling point to your library is that it compiles to native code but my question is with out changing code can I switch to running it on a cluster instead of local (something I could do with a language with an abstract form of concurrency using monads or Observables or even just using the Actor model)? It looks like I can right?

samuell12• 10 years ago

The idea for cluster support has so far been to implement connectors for resource managers like SLURM, and basically keeping scipipe as an orchestrator.

That is, for multi-node jobs. As long as you can stay within the 16-32 or so cores on a typical HPC node (in our cluster at least), scipipe should be great for that. I think some means of simple resource management (to not overstress these 16-32 cores) is needed, but that can be done in a simple way by e.g. using a central go-routine that lends out "the right to use a cpu core" on demand.

Thanks for interesting feedback. I will think about this!

knlje• 10 years ago

Have they already fixed the need to "devectorize" the code to make it fast? This was the major drawback for me since I wish the code to be concise AND fast.

elcritch3• 10 years ago

Found this to be pretty handy at times: [Devec](https://github.com/lindahua/Devectorize.jl).

chrispeel2• 10 years ago

I agree, this would be good. Intel's ParallelAccelerator.jl [1] is making some progress in this area.

[1] http://julialang.org/blog/2016/03/parallelaccelerator

irremediable• 10 years ago

What's the best way to run a debugger through Julia code? I value an interactive debugger much more for this kind of nitty gritty numerical programming than I do for lots of other tasks.

ced• 10 years ago

Work on the debugger is underway, it should be in the 0.5 release in a few months.

KKKKkkkk12• 10 years ago

Does Prof. Edelman have a financial stake in Julia? He is such an enthusiastic booster of the language, that either it must be truly amazing, or there is an undisclosed motive.

KenoFischer6• 10 years ago

Alan is one of the people who started the language (which is mentioned in the article). He is also one of the cofounders of Julia Computing (the company we formed to provide support and other services for the company). So from my biased viewpoint, the language is both truly amazing and Alan has a motive to promote the language, if only since he has a significant chunk of time and effort into making it a reality. I don't think it's fair to call it undisclosed however.

KKKKkkkk12• 10 years ago

I think it's great that Prof. Edelman is dedicating his time to Julia and I hope he will be rewarded for it. However, I think that a statement of the sort you provided above should have appeared below the SIAM News article. This is something that would have been done in any major newspaper or scientific journal.

SixSigma2• 10 years ago

I recently started using Julia as my main platform as a replacement for Octave in my engineering classes.

I have found it an excellent language, as easy as Python to pick up and some great numerical primitives.

Using Juliabox in class means no installation woes and means whatever I do during lessons is available wherever I can get online. The ability to mix in markdown and code on the same page is brilliant for keeping notes. I realise that is not a feature of Julia but still, it is a benefit.

I've not done any heavy work in it but I'm really looking forward to doing so. I can dump Python, Octave and Limbo all at the same time.

Julia: A Fast Language for Numerical Computing

Related Stories