Julia Impressions

http://eyeballtrees.com/posts/julia-impressions.html

by a0 • 12 years ago

82 59 12 years ago

idunning4• 12 years ago

Great stuff. I think the author may be using the release version 0.2, as the boot time issue has been rectified now for the most part in the next release, 0.3. The next big slow-load issues is packages, that'll be post 0.3 though.

srean3• 12 years ago

Assuming that this was fixed by caching the machine code, cant the same approach be used for packages ? Or to turn my question around: what is different about packages is it that they are not necessarily written in Julia ?

I have still not taken a serious plunge with Julia, testing waters.

Any plans for doing the same to individual scripts ? It would be handy to start off from the last known set of JIT'ed functions, may be retiring paths that are no longer frequented.

ihnorton4• 12 years ago

Yes - the base image is compiled to a shared library.

There is a mechanism for including package/user code in the system image if you compile Julia yourself. It works well with a wide range of packages, but the goal is to generalize this and make it usable with binary releases, and probably to compile packages to separate shared libraries.

eyeballtrees• 12 years ago

OP here. I'm currently using 0.2, I'll have to upgrade and see if that fixes the issue. Thanks!

ViralBShah2• 12 years ago

There are two separate issues, I believe you refer to. There is the boot time, which has been considerably improved, and is less than a second. The second thing is the slight pause that you refer to (if I understand the post correctly), when executing stuff the first time. That is the JIT compiling and warming the code cache.

DennisP2• 12 years ago

Is the error reporting improved for 0.3? That was my main problem with 0.2 (on linux). I ran into situations where all I knew was that something went wrong somewhere in my program. I've seen other people make similar comments.

(Aside from that I really love the language, and I'm looking forward to doing more with it.)

idunning4• 12 years ago

Not completely. Not an expert on this, but I believe issue at least partially stems from the interaction with the older LLVM JIT we are using. We are moving to the newer LLVM JIT in the 0.4 release cycle and I understand this will address the issue.

CyberShadow• 12 years ago

How is multiple dispatch different from function overloading, as found in C++/Pascal? Is it the same thing?

brandonbloom• 12 years ago

Dispatch is the process of selecting and executing the matching overload. Multiple-dispatch refers to the ability to select the best overload based on more than just the first argument. C++ can do arity and type-based multiple-dispatch, but only statically. Julia supports dynamic multiple dispatch, where as C++'s virtual methods are only single-dispatch dynamically.

StefanKarpinski5• 12 years ago

Precisely. This may sound pedantic or academic, but in practice when static overloading and dynamic dispatch differ, the static behavior is usually not what you want. In my experience, this makes it painful to make code that is polymorphic in multiple arguments behave correctly in C++ (double dispatch [1] is one of the necessary but awkward tactics that's used), whereas multiple dispatch just works. And of course, almost working and actually working are a world apart even though they sound so close.

[1] http://lostechies.com/derekgreer/2010/04/19/double-dispatch-...

epidemian• 12 years ago

From playing a bit with the Julia REPL (http://forio.com/julia/repl/), it seems that the dispatch of the method is dynamic, as opposed to the dispatch of overloaded functions in C++:

  julia> function foo(s::String) "$(s)!!!" end
  foo (generic function with 1 method)
  julia> function foo(n::Int) n + 1 end
  foo (generic function with 2 methods)
  julia> foo(rand() < 0.5 ? "Hello" : 41)
  42

I'm not used to this kind of dispatch for bare functions, but it seems to me that it could offer similar capabilities to what subtype polymorphism provides in OO†, without having to define new types for this methods to live in or polluting the existing types.

† Probably more, as the dispatch can be resolved based on many arguments.

ihnorton4• 12 years ago

Stefan Karpinski wrote a very nice overview of multiple dispatch with many examples and some discussion of design implications:

http://nbviewer.ipython.org/gist/StefanKarpinski/b8fe9dbb36c...

(he gave this as a talk at Strange Loop 2013, but the video seems to still be embargoed)

StefanKarpinski5• 12 years ago

This may be my fault for not submitting the slides. I haven't figured out how to turn the IJulia notebook into a PDF :-\

ninjin2• 12 years ago

I really want to get into Julia. As I have been using an increasing amount of numerical code NumPy, SciPy and even my own Cython code has started to fail me performance-wise. The Julia community is really friendly and I tried rolled out a few implementations of the inner-most loop of a Recursive Neural Net in a variety of languages.

https://github.com/ninjin/ppod/tree/master/hck/rnn_perf

Disappointingly enough I was unable to push the performance of Julia beyond that of pure (and ugly) Matlab. I tried asking over at #julia, but it seems that my implementation should be reasonably canonical. This is really a pity for me since I would love to remain highly productive in terms of code and use something scripting-esque rather than have to re-consider going back to C. I have also had a look at Nimrod, but the scientific library support appears somewhat lacking at the moment.

idunning4• 12 years ago

I used Julia's in-built profiler to look at your code. Almost all the work is done, as you'd hope, on lines 20, 22, 33, and 35. I added @inbounds to lines 20 and 35, figuring that line 22 is mostly just the effort of calculating tanh. This dropped the running time on my computer from 10.7 seconds to 6.4 seconds, so about 60% of original running time. If I compile with gcc (gcc rnn_perf.c -O2 -o rnn_perf -lm) and run that, I get 6.2 (O3 is worse). That might not be how you compiled the C version, but regardless it looks like your C time was about 46% of the original Julia time, so its getting close. Three-fifths of the work now is in the tanh line. I'll submit a PR.

StefanKarpinski5• 12 years ago

Having people like Iain – with this kind of knowledge of numerical programming and optimization – help you out with your code is one of the major benefits of Julia programming :-)

ninjin2• 12 years ago

Thanks a ton Iain, it pushed it down by 60% on my end as well. About the C code, I used -O2, the specific compiler flags are in the rnn_perf.sh file.

https://github.com/ninjin/ppod/commit/ce7665a2cfd045332e9861...

Julia is really tempting for my next project, the benefits of scripting without all the OOP cruft and a community that just shines. Also, did I mention the awesome meta programming?

JPKab• 12 years ago

Wow. If this is what the community in Julia is like, count me in.

I've been focusing on mastering the scientific libraries of Python and waiting until I hit performance walls before I took Julia on, but perhaps I'll start now.

andreasvc• 12 years ago

my own Cython code has started to fail me performance-wise.

I think this is an odd statement, seeing as how Cython allows you to write pure C code without making any use of the Python C API, so by definition Cython can be as efficient as C minus a single function call from Python to C.

I think your benchmark is way too small to be interesting. You're only testing raw numerical performance, and of course C wins there. Any of these languages allow you to wrap a C library with more or less convenience. I think the more interesting questions are how well these languages allow you to build larger applications with performance-sensitive components, how extensive the ecosystem of libraries is, and the quality of debugging and profiling facilities.

bachase• 12 years ago

I also starting working on a neural network implementation for my first foray into Julia:

https://github.com/bachase/nnadl-julia.

The notes file on the optimization branch is a very rough outline of things I discovered while iterating on performance. Outside of Iain's great advice and without profiling your code, for a larger sized network, you might benefit from reordering loops for column major storage, or doing normal matrix multiplication (which uses BLAS) followed by a devectorized tanh.

As for the language, I love being able to get something working quickly while writing in a Matlab/"mathy" style. The built in profiler and timer tools are then excellent for zeroing in on hotspots and memory allocations. I'm just amazed at how simply I can drill down from scripting style code to LLVM IR to native asm all in an ijulia notebook browser window.

On the downside, I'm not a fan of having to manually devectorize to avoid temporaries, especially given the success of c++ libraries like Blitz that figure it out for you. I'm sure Julia will improve in this area as it matures.

Serow225• 12 years ago

Just curious, is it possible to vectorize your MATLAB implementation? In your test code you don't use the loop index in the calcs, so that makes it hard for me to tell if it is or is not possible to vectorize it for your actual application.

mandor• 12 years ago

I was curious to know how modern C++ would compare. Here is a version (not 100% sure about the result because I was not 100% sure of the meaning of some of the numpy constructs when applied to matrixes).

http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc

It seems pretty fast!

zmmmmm3• 12 years ago

What are the libraries like for Julia at this point? The big thing holding me to R is the breadth of libraries for a) various statistical methods and algorithms and b) plotting.

I'm kind of desperate to get away from R at this point, but reluctant to move to Python (not that I have anything against Python, but I would really like to add a high performance language to my toolkit than something inbetween).

StefanKarpinski5• 12 years ago

It depends on what you need. DataFrames [1] is a pretty good replacement for R's data frames. Distributions [2] is pretty amazing. Gadfly.jl [3] is a great ggplot-style plotting library that integrates nicely with IJulia [4]. If you have questions about specific things you need to do and want pointers, post on the julia-users list [5] and you should get some good answers.

[1] http://harlanh.github.io/DataFrames.jl/

[2] http://distributionsjl.readthedocs.org/en/latest/

[3] https://github.com/dcjones/Gadfly.jl

[4] https://github.com/JuliaLang/IJulia.jl

[5] https://groups.google.com/forum/#!forum/julia-users

zmmmmm3• 12 years ago

Thanks!

izyda2• 12 years ago

Out of curiosity, why are you so desperate to get away from R? Is it for the speed of a high performance language?

As you mention, the vast selection of libraries (and the continual development of these libraries) make even the python module universe look small (in reference to statistical methods, statistical algorithms/machine learning, plotting).

Further, not only are there a lot of libraries.. but many of them are bleeding edge written by the same people who originally invented the given algorithm. Most similar libraries in other languages (like Python) have incomplete ports of these libraries to them often not written by the same people. Not to mention, the documentation behind these algorithms is often excellent - besides the usual man files, there's "vignettes" (detailed instruction manual) as well as an entire journal that now has become a place to publish papers detailing the usage and implementation of many of these algorithms.

So I'm curious, why leave?

zmmmmm3• 12 years ago

I love a lot about R, and I totally acknowledge the benefits. I don't think I would ever stop using it for data exploration and ad hoc analysis.

But after trying really hard over the past 3 years I'm sort of giving up ever being able to use it in a productive reliable fashion for building more complex software. It's a failing on my own part in many ways, but the loose typing and poor support for structuring code interact very badly with my personal style. When I'm working in R I literally spend 80% of my time fighting with the type system, debugging weird and wonderful features of R. I'm at the point where whenever I write an R function the first 50 lines of the code are type checks to make sure the data coming in is what I expect. I came to this point after I started systematically tracking why I was wasting so much time, and nearly every time it would come back to the type of my data being something other than what I had expected or assumed.

So I'm sort of figuring if I'm at the point where I'm writing manually statically typed code in R ... I should look around for a language that's just like R but has at least slightly stronger data types built in.

srean3• 12 years ago

I cannot speak for zmmmmm but I can think of two reasons (i) performance, not that R cant be fast but you have to pull out heavy machinery such as parallelization much earlier and (ii) its inconsistent and confusing language semantics, and silent and unexpected type promotion/demotions that make it very error prone to develop and debug code in. To me the one redeeming quality about R is ggplot and, ok, availability of push button libraries

I am not so sure about the scope of the 'bleeding edge' part. It is popular among old school, statisticians. Another population (no pun intended) that R is popular in is one where one knows neither statistics nor machine learning but just wants to try out a laundry list of canned methods without a need to understand: pretty plot goes up, yay awesome; pretty plot goes down, ok try the next algorithm (or the other way round). Here I think R is pretty unbeatable in its breadth.

It is not that popular among machine learners. Part of it is cultural, a typical machine learning person will be coming from a CS background and then R grates more.

It has been claimed that R is lisp meets stats, I think that would be Julia now and Lush then. http://lush.sourceforge.net/. To quote Yann LeCun:

"Lush combines three languages in one: a very simple to use, loosely-typed interpreted language, a strongly-typed compiled language with the same syntax, and the C language, which can be freely mixed with the other languages within a single source file, and even within a single function."

onalark• 12 years ago

This is one of the misconceptions about Python, that it's "slow". Python has several type annotation systems with JITing capability, and can easily achieve C-speeds (as well as interface easily to C libraries). We've given two tutorials at Supercomputing in the last two years, and the trend for high performance computing in Python is on the rise. Don't count it out!

throwaway13qf85• 12 years ago

I'm really excited by Julia. We use Matlab heavily in my team (as in, we probably have 300-600 Matlab instances running at any given time not counting the ones on personal workstations) and I'd love to use a language with similar support for numerical programming and linear algebra, but with better typing and functional programming support.

Do you have any suggestions for slowly introducing Julia to the team?

Bootvis2• 12 years ago

May I ask what you're doing with all those instances?

And to be extremely bold: what are you paying for all those licenses?

irremediable2• 12 years ago

What's the deal with the lack of conditionals in comprehensions? Is there a good reason that the Julia devs didn't include that, or is it just something that's not been implemented yet?

Edit: after a little searching, I found this.

Filters and guards don't mix with multidimensional comprehensions. For 1-d comprehensions, I'm not convinced that it's really worth the additional syntax. What's the case for making it part of the syntax instead of just using a filter function?

omaranto• 12 years ago

Julia's comprehensions are multidimensional array comprehensions so allowing filtering doesn't make much sense because the result needs to be rectangular and filtering can create arbitrary holes.

For a one-dimensional array comprehension it does make sense to allow filtering and this can be easily expressed in Julia by making a coroutine that produces the values and collecting its output in an array.

Multidimensional (here multi=2) array result:

    julia> [i+j for i=1:3, j=1:3]
    3x3 Array{Int32,2}:
     2  3  4
     3  4  5
     4  5  6

One dimensional array result, with filtering:

    julia> collect(@task for i=1:6; if i%3>0; produce(i); end; end) 
    4-element Array{Any,1}:
     1
     2
     4
     5

Notice that this last technique is more powerful than typical comprehensions, since it doesn't just add the possibility of filtering but arbitrary code to generate the elements.

irremediable2• 12 years ago

Thanks for the further info! I'd managed to figure out that it was because of multidimensional arrays (that snippet I quoted) but I didn't immediately see a nice way to achieve the same thing. And now, because you posted it, I do.

I'll have to give Julia a go. :)

SeanDav• 12 years ago

If I can ask a stupid question. Why Julia? It seems to me that if you are not going to be using Python, then why not use a language like Java?

jey• 12 years ago

Huh, what upside does Java have?

Julia is specifically designed to be a great high-performance high-level language for scientific computing. See http://julialang.org/blog/2012/02/why-we-created-julia/

pepon• 12 years ago

public class Answer {

public static void main(String[] args) {

   System.out.println("That's why");

 }

}

GFK_of_xmaspast• 12 years ago

A lot of people doing scientific programming are not, in many senses of the word, 'programmers', and java doesn't really fit in the kinds of workflow they want. (That's not a condemnation, it's just the way it is, see also the ghosts of fortran 77)

xioxox2• 12 years ago

I was interested in Julia until I discovered that it doesn't allow useful things like object-oriented inheritance (unless you go down some complex metaprogramming route). Multiple dispatch is indeed cool, but for some types of systems modelling with inheritance and classes is a great or the best approach (e.g. GUI widget libraries). I thought about how you might wrap something like Qt, gave up, and decided to continue with lovely Python (and its lovely 0 based indexes).

ihnorton4• 12 years ago

Single inheritance is possible and used extensively in Julia. Multiple inheritance is not (currently) supported, and there are a few places where it would be useful, but it is surprising that this would be a show stopper.

xioxox2• 12 years ago

I'm not sure it's the single inheritance people from other languages might understand. Perhaps I'm misunderstanding things as I've hardly used the languages, but there are links like http://thenewphalls.wordpress.com/2014/03/06/understanding-o... and http://grollchristian.wordpress.com/2014/01/22/julia-inherit... and https://github.com/JuliaLang/julia/issues/4935

SandB0x• 12 years ago

Hmm, I was hoping for some impressions of using Julia for numerical computing. The mechanics of list comprehensions and uppercasing some text seem a little tangential.

thinkpad203• 12 years ago

That's an important part of Julia, but it's also important to talk about the language as a language. One of nice things about Julia is that it has things to talk about other than its numerical computing capabilities :) Those are fine and dandy, but as a non-numerical-computer, I find lots to like about Julia that has nothing to do with its ability to multiply matrices.

ihaveqvestion• 12 years ago

People who are using Julia or similar tools to do "scientific computing, machine learning, data mining, large-scale linear algebra, distributed and parallel computing" - what sort of problems are you solving? How did you end up doing this sort of work?

hderms• 12 years ago

Julia looks like a wonderful mix of Ruby, Python and ML (or something). I'm really eager to see this adopted by more people.

matheusbn• 12 years ago

At the beginning of this article: "I spent most of last night programming in Julia and I've got some thoughts on my first impressions. TL;DR: It's like Python with less OOP, more FP and heavier typing. Also, it's awesome."

--------------------------------

Then I clicked on the site's about: "Welcome to Eyeball Trees. My name is Stephen Malone and I cannot be trusted."

I'm really confused!

thinkpad203• 12 years ago

I've been following Julia for a while now, not really using it, but playing with it a bit and keeping up with it. I think it's very promising and I really look forward to seeing it fledged out more.

It's a very small nitpick, but I wish Julia had a postfix function application syntax. It would be nice to be able to say `foo = 'hello'.reverse` and have it be syntactic sugar for `foo = reverse(hello)`. Or some other operator since `.` is in use. This reads nicer for long function chains, and especially when a function modifies its argument, it's nice to have that argument come first, which gives an OO-impression. E.g. for me this:

    app = make_app(options)
    app.route("/", _ -> "hello, world!"),
    app.route("/submit", POST, req -> "hello, $(req.params["name"])!")
    app.start(3000)

than this:

    app = make_app(options)
    route(app, "/", _ -> "hello, world!"),
    route(app, "/submit", POST, req -> "hello, $(req.params["name"])!")
    start(app, 3000)

idunning4• 12 years ago

There is this, which is similar but not quite the same:

    julia> "hello" |> reverse |> uppercase
    "OLLEH"

doctoboggan• 12 years ago

I too am intrigued by julia based off of my brief interactions with it. My only issue is the lack of namespaces. Is there a good reason why they aren't included?

thinkpad203• 12 years ago

They are: http://docs.julialang.org/en/latest/manual/modules/

Or is that not what you were referring to?

Julia Impressions

Related Stories