Open Big Data Computing with Julia

http://istc-bigdata.org/index.php/open-big-data-computing-with-julia/

by astrieanna • 13 years ago

70 21 13 years ago

pjmlp4• 13 years ago

Quite nice to see Julia taking off.

glifchits• 13 years ago

As a beginner in scientific/data computing, is it worth building a foundation in Julia or do with some other tool instead? I'm comfortable in Python but I don't know if its best to teach myself how to use NumPy and friends when I have the opportunity to immerse myself in this brand new shiny technology.

acidflask• 13 years ago

It depends on what kind of user you are. If you are comfortable with the risks and rewards of being an early adopter using bleeding-edge tools, then by all means try Julia. If you need the stability and functionality of a mature platform, then you may not want to learn Julia as your first tool.

burntsushi• 13 years ago

It depends. There is a certain allure to working with a shiny new technology. It's sort of like an adventure, because not many have paved a strong path forward yet. But not everyone likes an adventure. Sometimes there will be huge roadblocks that don't have obvious solutions; especially for the beginner.

On the other hand, the scientific computing crowd for Python/NumPy is well established and there will be plenty of people who have probably come across any problem that you will see. There is already an established ecosystem of tools that you'll be able to draw from, and it's likely that many of them have been well tested and are reliable.

The decision is really up to you. Do you have the time and desire to go on adventure? If so, maybe Julia would be a nice pick. Are you short on time or weary of adventure? Then maybe Python/NumPy would be a better pick.

trthatcher• 13 years ago

I don't think it will be a problem to do both.

It doesn't take long to learn Julia well enough to write a Julia package that others will use. For me, that was about 3 weeks @ 10-15 hours a week, and I'm no programmer. It's a small language and it's quite readable. I'd reccomend going through the Julia manual; it doesn't take long and only requires some diligence.

The problem is the lack of packages/libraries and lots of documentation is still missing. That, and the packages that do exist with multiple contributors are still in disagreement about standards and consistency.

sanskritabelt• 13 years ago

If you're a beginner, stick with python, or matlab.

RivieraKid3• 13 years ago

I hope that the web libs will improve, I'm trying to create a very simple web app, but the libs are in alpha stage.

adambard2• 13 years ago

I'm not sure Julia is the most appropriate for this application. It's nominally general-purpose, sure, and you can run a web server from it, but you can do that with any language with a socket library and strings.

Would you make a website with MATLAB?

pjmlp4• 13 years ago

Would you make a website with MATLAB?

The world is a nail. :)

RivieraKid3• 13 years ago

I think that if you ignore libs, Julia is a better general-purpose language than other dynamic languages like Python or Ruby. Which is definitely not true for MATLAB :)

This is an ultra simple web app, that does some calculations on the backend that would be slow in Python (or I would have to preprocess the data).

zhemao• 13 years ago

Julia uses libuv, the same library used by Node.JS, for its networking functions. Therefore, given better library support, Julia could be quite suitable for writing web applications. If you wanted to provide a web frontend to some numerical computation (as I think the Sudoku solver mentioned in the article does), such functionality would be quite useful.

astrieanna• 13 years ago

Hi, I'm one of the maintainers of Julia's webstack packages. If you're having problems, please make an issue(s) on github. We would definitely appreciate feedback. I'll try to make sure to respond.

RivieraKid3• 13 years ago

Hi, sorry, I currently don't have time / energy for opening issues so I hope it's ok if I sum it up here:

1. I didn't manage to make Mersel serve static files (advice in comments under github issue didn't help).

2. The server doesn't send the full response to the browser, it's cut-off at the end. I had to add many white-space characters at the end of the html. I think this related to non-ascii characters and content-length.

3. If I run the server in the REPL, than stop it (Ctrl+C) and then start again, it complain that the port is in use. So I change the port, but the server runs on the original port. Related: autoreload would be very nice.

EvanMiller• 13 years ago

I like Julia because in many ways, it's a better C than C:

* You can inspect a function's generated LLVM or (x86, ARM) assembly code from the REPL. (code_llvm or code_native)

* You can interface with C libraries without writing any C. That let me wrap a 5 kLoC C library with 100 lines of Julia:

https://github.com/WizardMac/DataRead.jl

* You can use the dynamic features of the language to write something quickly, then add type annotations to make it fast later

* Certain tuples are compiled to SIMD vectors. In contrast, the only way to access SIMD features in C is to pray that you have one of those "sufficiently smart compilers".

* Like C, there's no OO junk and the associated handwringing about where methods belong. There are structures and there are functions, that's it. But then multiple dispatch in Julia gives you the main benefits of OO without all the ontological crap that comes with it.

For me, Julia feels like it's simultaneously higher-level and lower-level than C. The deep LLVM integration is fantastic because I can get an idea for how functions will be compiled without having to learn the behemoth that is the modern x86 ISA. (LLVM IR is relatively simple, and its SSA format makes code relatively easy to follow.)

Anyway, I only started with Julia recently, but I'm a fan. I should also mention that the members of the developer community are very, very smart. (Most are associated with MIT.) BTW I am starting a Julia meetup in Chicago for folks in the Midwest who want to learn more: http://www.meetup.com/JuliaChicago/

pjmlp4• 13 years ago

There are structures and there are functions, that's it. But then multiple dispatch in Julia gives you the main benefits of OO without all the ontological crap that comes with it.

Multi-dispatch is a form of OO.

StefanKarpinski2• 13 years ago

I would argue that it's the other way around: single-dispatch o.o. is a special case of multiple dispatch.

mrcactu5• 13 years ago

I have installed iJulia -- which involved compiling Julia from source.

What little bit I've tried with Julia so far is impressive.

What is the story anyway? Did a bunch of MIT people decide they needed something faster than R & numpy ?

Julia inventor Alan Edelman is an fellow alumnus of Hampshire College Summer Studies in Mathematics (HCSSiM). Scores points in my book. https://en.wikipedia.org/wiki/Alan_Edelman

He wrote a terrific paper back in '95 http://arxiv.org/abs/math/9501224 "How many zeros of a random polynomial are real?"

StefanKarpinski2• 13 years ago

Alan is a rare and brilliant fellow. Just to clarify, he has been Julia's patron from the very early days, but the actual design and development of the language was done by Jeff Bezanson, Viral B. Shah, and myself. These days it's a collaborative community effort.

army• 13 years ago

Very cool. I think the point about transcribing equations directly from papers glosses over some of the hard issues with building numeric libraries around floating point precision and numeric stability.

It's not that uncommon to see code that would have worked perfectly if double precision floating point numbers were actually real numbers, but which blows up badly once rounding errors happen.

Unfortunately there's not really any way to abstract those issues away. In general, I'd hope that any the authors of widely used numeric libraries have carefully thought through these issues.

Open Big Data Computing with Julia

Related Stories