Why I am betting on Julia (2014)

http://www.evanmiller.org/why-im-betting-on-julia.html

by sean_the_geek • 10 years ago

142 109 10 years ago

evanmiller.org

ones_and_zeros• 10 years ago

Any thoughts on how it compares and will overcome the dominant enviromnent for technical programming, matlab?

2miles4chicken• 10 years ago

It's certainly faster: http://julialang.org/benchmarks/.

It is free/open source. And, it benefits from being a younger language and therefore has the hindsight of the quirks in Matlab's design. Coming from Python, the 1-indexed arrays are annoying but it's great for math.

sghodas• 10 years ago

The 1-indexed arrays can sometimes be nice, but I've run into bugs so many times when translating the code into a language with 0-indexed arrays.

ninjin• 10 years ago

What Julia really lacks at this moment is something along the lines of the Matlab IDE. The community is aware of this though and there is something in the works that even I, someone that lives happily with a ssh/tmux/vim set-up, would deem as interesting.

Since we are on Hacker News, maybe I should also spread a rumour. MathWorks, allegedly, in a recent settled lawsuit added a condition stating that the other company was not to create any Julia bindings for its product. If this is true, which I have many reasons to believe, it is a major recognition of how at least MathWorks sees Julia as a real threat to the Matlab market share.

bsilvereagle• 10 years ago

MathWorks sees Julia as a real threat to the Matlab market share

I think Mathworks is feeling a lot of pressure from both Python & Julia. In the r2016a release of MATLAB, Mathworks essentially released a Jupyter (one of the first environments for Julia, and formerly iPython Notebooks) competitor with the 'Live Notebook'. Mathematica has had notebooks for quite a while, and the best Mathworks had was their MuPad suite. I think the prevalence of Jupyter definitely led to this development.

http://www.mathworks.com/products/matlab/live-editor/

etrautmann2• 10 years ago

Certainly using Julia with Jupyter gets you much closer to having a reasonable IDE, though you'll have to be comfortable with cell-mode debugging instead of breakpoints and single stepping. I don't personally use it but I'm intrigued and would love to switch away from Matlab if there were reasonable feature parity or an alternate model that seemed reasonable for most basic tasks.

I'd really like to have a replacement for cell arrays and struct arrays, that have some of the convenient features of those structures. I know you can get this with lists of dictionaries in python, for example, but it's not the same.

phantom_oracle• 10 years ago

When I read this, all I can think about is the enormous amount of man-hours people have put in to make Python the go-to for Julias target market.

As the author says, the biggest drawback is libraries, and that is (1 of) the big attraction(s) towards languages.

Nobody wants to have to switch between C, Python, Assembly, etc. to make their code faster, but even fewer people want to write bindings to underlying C code. This also goes exactly against what the author doesn't want to do, which is to yak-shave around the language to get things working.

sdenton4• 10 years ago

It's extremely common to prototype in a language like python or R to be able to bang out a draft idea before diving into making it fast. If I'm not sure how awesome something is going to be, there's no sense in wasting weeks getting a C implementation working. And prototyping in a higher level language often reveals a lot of the architectural complexity you'l have to deal with in a lower level implementation. So I think this guy's use-case is more common than you think.

Python addresses a good deal of the problem with Cython, which lets you declare variables and compile to get ~90% of the speed gain you would get from a pure C implementation. And then you can just link to a C implementation if you still find you need to build one...

rm9992• 10 years ago

It's extremely common to prototype in a language like python or R to be able to bang out a draft idea before diving into making it fast.

I think what's far more common is prototyping something in Python or R, and then realizing that 99% of its CPU time is already spent in wrapped C code. And then you call it a day and move on to other things :)

This is why Julia hasn't taken off nearly as quickly as I initially expected - Python/R/etc are highly optimized for their common use-cases.

marmaduke• 10 years ago

Numba accelerates Python loopy code using an LLVM backend in the same spirit as Julia, so there's not much reason to look at Julia just for speed reasons

jernfrost2• 10 years ago

You can use python libraries in Julia. And it is really good at binding to C and Fortran.

Python is old and stuck with some serious design flaws which can not easily be fixed. The question is, can Python get fixed in shorter time than it takes to give Julia a competitive set of libraries.

eva19845• 10 years ago

I would love to know what fundamental design flaws are there for Python in your opinion...GIL? No problem with Cython. Unicode? It is ugly but it could be done. Other stuff like static type system, is about language itself, and I don't think it needs to be addressed as flaw to be fixed or rather just being different preference.

For me, there is not something I can think of being such a deal breaker for me to switch to a another language. More likely it might not be ideal but could addressed by the ecosystem of Python itself.

baldfat2• 10 years ago

Python the second best option.

I love Python (despite the community). Sorry to be that guy but Python is always the second best and for me that usually means I end up using Python less and less. There isn't one area that Python is the best tool and sadly that is why it just isn't taking over the programming world.

riscy3• 10 years ago

Julia breaks down the second wall — the wall between your high-level code and native assembly [...] you can take a peek behind the curtain of any function into its LLVM Intermediate Representation as well as its generated assembly code — all within the REPL

While I agree with the author in "betting on Julia", I do not think inspecting LLVM IR or assembly of individual functions in the REPL is as useful as a feature as the author thinks, because it gives an inaccurate picture of what the actual output will be.

Peeking at the code at the level of functions means a non-inlined version of the function must be produced at the REPL, and anywhere else this function is used in your actual code, it will probably be inlined (the author's example certainly will) so it can be further optimized/eliminated by later passes. Furthermore, for a non-trivial function's dump, fundamental optimizations that increase code size (inlining, unrolling loops etc) would make the output confusing to compare with the Julia equivalents.

Thus, I imagine a lower optimization level is used to produce readable REPL dumps. For a performance minded person (the only type of person who would care about ASM/LLVM dumps at a REPL) even this is not helpful.

3JPLW3• 10 years ago

Yes, when a function is inlined, the caller's context can dramatically affect optimizations. And inspecting varargs functions can be tricky.

But beyond that, it generally displays exactly what gets executed. You can verify this yourself by looking at functions with different `-O` or `--inline=no` flags. And if you're interested how a function optimizes with inlining under typical conditions, you can just inspect the outer function.

mynegation• 10 years ago

It needs (2014) in the title if for nothing else than for the fact that a lot could have changed since then, as Julia is pretty fast developing

dang• 10 years ago

Added.

ProAm• 10 years ago

For me, code is like a car. It's a means to an end. The "expressiveness" of a piece of code is about as important to me as the "expressiveness" of a catalytic converter.

Im a sculptor, but all I make is coffee cups and ashtrays so I dont need to worry about the minor details of things..... This is what abstraction gets us.

tanlermin10• 10 years ago

I think it will all come down to web assembly. Why?

Python has blaze, numba, dynd and dask going for it. These all ameliorate (and exceed Julia in some respects) many of the disadvantages of python (including fast user defined types). Then there are the libraries that while some can be used in Julia, you will never get full reliability, ease of use and compatibility.

On the other hand, Julia has amazing metaprogramming and much cleaner scientific syntax.

I think it will come down to whether Python can compile to fast LLVM regular standard lib programs.

Once (if?) Julia can be run in the browser with web assembly (using ahead of time compilation to produce small binaries), python has no chance if it doesn't follow suit. Python has no advantage that can permanently match Julia's potential ability to run on mobile, front end web, back end etc all from one beautiful codebase.

What does HN think?

riscy3• 10 years ago

Once Julia can be run in the browser with web assembly [...]

That will never happen for Julia, because the language is designed to be fast for scientific computing... targeting web assembly is a pointless additional layer of abstraction that only negatively impacts performance.

tanlermin10• 10 years ago

Yea... I don't think so:

https://github.com/JuliaLang/julia/issues/9430 http://juliacomputing.com/blog/2016/02/09/static-julia.html

tavert6• 10 years ago

including fast user defined types

Can numba do this yet? Without having to write special "numba classes" code?

Blaze has a giant serialization wall between it and the JVM which it needs to solve to ever be competitive in its target market. Ibis is the much more compelling effort here.

Dynd is an overly complex C++ disaster that is trying way too hard to solve problems that aren't what anyone really struggles with. The number of real world problems for which homogeneous dense multidimensional arrays of floating point numbers is the solution isn't growing much. Real problems have interesting structure that your data structures need to be able to exploit on a problem specific basis. Doing the array layer all in templated C++ with Python bindings is not a recipe for making something easy to use and effective at problems it wasn't previously designed and compiled to solve. Displacing numpy is not going to happen in the Python space.

Dask has good ideas, but the insistence on implementing everything in pure python means you're stuck with the GIL and lousy performance of your scheduler. This needs to be low overhead if they want anyone to actually use it.

tanlermin10• 10 years ago

Granted special Numba class syntax, and nowhere near as elegant or integrated as Julia.

Dask distributed interfaces directly with HDFS...no need for serialization.

Dynd is specifically designed to deal with arrays of custom types...criticism sounds like it would be more aptly directly towards numpy.

once it gains Numba binding it will lose the Python overhead.

Re Dask...is 1ms per task too much? It has a threading scheduler that can work with numpy arrays to release the GIL....so not bound by that with numerical code.

Regarding regular list and text processing... Julia's poor performance with heterogenous data actually makes it slower than python for cleaning the corresponding dirty and text datasets.

simonster2• 10 years ago

WebAssembly is a neat idea, and could be useful for creating cool ways of interacting with scientific data. But a lot of code scientists write only needs to run once on one machine. And in that case, there's no reason to take the inevitable performance hit associated with WebAssembly.

ww520• 10 years ago

Web assembly would be great. Most browsers support WebGL now. Julia can utilize the GPU via the browser; the gain would be huge. Second, browsers are perfect for SETI-style distributed computation. Imagine a beowulf cluster of these browsers running downloaded Julia snippets...

username2232• 10 years ago

Julia's potential ability to run on mobile, front end web,...

Julia is designed for interactive, exploratory scientific programming, like Matlab and R. I don't see any reason someone would want to do that on a phone or in a web browser when it already runs natively on a PC. Then there's the matter of Julia being designed to interface efficiently with optimized native C/Fortran libraries.

KingMob2• 10 years ago

This. Scientific computing is not interested in running in web browsers.

As a former neuroscientist, I occasionally had connectivity analyses that would take 3 weeks to run. No way would I risk slowing it down with anything unnecessary. A 25% slowdown for using VMs in the browser would mean milliseconds for users and days for me.

ZenoArrow• 10 years ago

I understand why some may see Julia as 'designed for scientific computing', but this is mostly a reflection of the work that's been put in to build up the library ecosystem. Julia could end up being a good general purpose language too, the only thing holding it back from this is the lack of library diversity and size.

rdtsc• 10 years ago

2014. Wonder if anything changed since then. How did author find using and developing in Julia since then. Has their opinion changed?

mhd• 10 years ago

http://wizardmac.tumblr.com/post/104019606584/whats-wrong-wi...

The author writes that Julia still isn't ready to replace his current development setup.

mahmud3• 10 years ago

  CL-USER> (defun f (x) (* x x))
  
  F
  CL-USER> (f 2.0)

  4.0
  CL-USER> (disassemble 'f)
  
  ; disassembly for F
  ; Size: 37 bytes. Origin: #x1002B10BDC
  ; BDC:       498B4C2460       MOV RCX, [R12+96]               ; thread.binding-stack-pointer
                                                                ; no-arg-parsing entry point
  ; BE1:       48894DF8         MOV [RBP-8], RCX
  ; BE5:       488BD3           MOV RDX, RBX
  ; BE8:       488BFB           MOV RDI, RBX
  ; BEB:       41BBB0020020     MOV R11D, 536871600             ; GENERIC-*
  ; BF1:       41FFD3           CALL R11
  ; BF4:       488B5DF0         MOV RBX, [RBP-16]
  ; BF8:       488BE5           MOV RSP, RBP
  ; BFB:       F8               CLC
  ; BFC:       5D               POP RBP
  ; BFD:       C3               RET
  ; BFE:       0F0B10           BREAK 16                        ; Invalid argument count trap
  NIL
  *

username2232• 10 years ago

That "GENERIC-*" points to just one of the problems.

mahmud3• 10 years ago

[Edit: Julia prides itself in being a fully generic language. From its paper:

"Generic functions have appeared in several object systems in the past, notably CLOS [15] and Dylan [28]. Julia is distinguished from these in that it uses generic functions as its primary abstraction mechanism, putting it in the company of research languages like Diesel [10] and Cecil [9]." :-) ]

Behold!

  CL-USER> (defun f (x) (declare (single-float x)) (* x x))
  
  F
  CL-USER> (compile 'f)
  
  F
  NIL
  NIL
  CL-USER> (disassemble 'f)
  
  ; disassembly for F
  ; Size: 37 bytes. Origin: #x10048EACEC
  ; CEC:       498B4C2460       MOV RCX, [R12+96]               ; thread.binding-stack-pointer
                                                                ; no-arg-parsing entry point
  ; CF1:       48894DF8         MOV [RBP-8], RCX
  ; CF5:       0F28D1           MOVAPS XMM2, XMM1
  ; CF8:       F30F59D1         MULSS XMM2, XMM1
  ; CFC:       660F7ED2         MOVD EDX, XMM2
  ; D00:       48C1E220         SHL RDX, 32
  ; D04:       4883CA19         OR RDX, 25
  ; D08:       488BE5           MOV RSP, RBP
  ; D0B:       F8               CLC
  ; D0C:       5D               POP RBP
  ; D0D:       C3               RET
  ; D0E:       0F0B10           BREAK 16                        ; Invalid argument count trap
  NIL

tomp• 10 years ago

Julia does it an order of magnitude better, though.

A generic `f` will probably be similar to the Common Lisp one. However, as soon as you use `f` in a context where types are known (e.g. if you use multimethods to overload a function that calls `f`), it will be specialized and JIT-compiled for the specific type (int, float, matrix, ...), with no extra work for the programmer.

lispm2• 10 years ago

In the 'Lisp World' JIT compiler are mostly not used. The main approaches are:

* program compilation, where the object system can declare parts as non-extensible

* a runtime compiler, which can compile methods

See for example for a recent attempt to statically compile methods and handing the types to the optimizing compiler (which also uses type inference):

https://github.com/guicho271828/inlined-generic-function/

junke7• 10 years ago

If I understand correctly, functions are inlined[0] (not saying this is bad, just different). If you declare functions as inline then you can get rid of the dispatching code in contexts where the type is known in advance.

[0] https://github.com/JuliaLang/julia/issues/265

tombert• 10 years ago

All of Evan Miller's programming reviews are good. I'm pretty sure it's been posted on HN before, but I quite liked his take on Rust as well.

muktabh• 10 years ago

This was in 2014. Here is a more recent perspective (from a different blogger who used Julia in production) https://blog.staffjoy.com/retro-on-the-julia-programming-lan...

philip1209• 10 years ago

I wrote that article. I still bet on Julia. We just used pre-production software in a production environment and the tooling wasn't ready. I still love the language and believe in its future, and I can see us building future services in the language.

sklogic2• 10 years ago

They tried to use Julia for things like networking instead of confining it within purely numeric parts. No idea why would anyone use a DSL for doing some "general purpose" stuff.

IshKebab• 10 years ago

I've used Matlab quite a lot for "general purpose" stuff. Ok not networking, but you'd be surprised how good it can be at non-numeric things.

Most of that was probably due to the slick GUI which nothing else comes close to (I've tried them all; trust me), but that is a legitimate factor.

max_• 10 years ago

The founders of julia are geniuses. The only problem the language is facing is, marketing.

Juila is one of those great things that everyone should use, but are simply bot promoted enough.

dmpk2k• 10 years ago

Julia is getting plenty of attention. It's pretty obvious at this point that Julia is destined to eventually be a major player in some fields.

The main problem facing the language is maturity. That will come with time. Until it has fully stabilised, has a mature runtime, has a mature library ecosystem, and has strong debug and post-mortem tooling, it is not something everyone should use.

eggy2• 10 years ago

They're smart alright, but I think Julia has gotten a lot of play on other channels besides the Matlab/tech world. The stopper is for me is use cases. I play with ideas in Mathematica (notebook IDE for decades) because it has everything at my fingertips, keep J programming language opened on my desktop for quick prototypes (not just as a desk calculator), and if I need technical libraries and speed, I just use C with the relative libraries (even though I should learn Fortran :) ) I like Julia's syntax, and I think it will be more general purpose as it matures, but it is already the wild mistress of the Matlab playboy.

tanlermin10• 10 years ago

What other channels?

lmm3• 10 years ago

Language design is vital for a language that can evolve. If you build a language for "cowboys" with the immediately-useful functionality that you can see, but without the "language geeks" insisting on the underlying principles, you end up with a language like perl - one that can dominate a particular use case for a while, but will be virtually impossible to evolve, and ultimately fall out of favour as it's overtaken by languages with firmer foundations.

That's why I'm betting on the strongly-typed functional tradition - OCaml/Haskell/etc. These language already have the killer feature from this article: they can be (almost) as expressive as Python and (almost) as performant as (naive) C++. But they also have really solid design with a lot of thought put into it that will let them evolve to meet the needs of the future.

junke7• 10 years ago

Julia stands definitely on firm foundations, has a solid design and is strongly-typed too.

9981721212• 10 years ago

Strongly typed and dynamically typed, which means that you can get runtime errors. Also, the "type system" with "dependent types" is really just equivalent to an ML datatype that's dynamically type checked.

I'm a bit mystified with respect to what passes as research in the field of type systems for dynamic languages.

lmm3• 10 years ago

Then why's it positioned as a numerical/datascience language rather than a general-purpose language?

vfclists• 10 years ago

Spoiler alert. Use Nim.

bleair• 10 years ago

This review was really helpful to me when I was looking for more insight into Julia. http://danluu.com/julialang/

Why I am betting on Julia (2014)

Related Stories