Revolution R Open: The Enhanced Distribution of Open Source R

http://mran.revolutionanalytics.com/

by orin_hanner • 12 years ago

80 36 12 years ago

mran.revolutionanalytics.com

plg• 12 years ago

iPython / SciPy

MATLAB

Julia

It's nice to have so many options today, it means when you bang up against the shortcomings of one language/system you can often jump to another one to get what you want (e.g. speed, or specific libraries, or graphics niceties, etc).

I only wish MATLAB wasn't so expensive and so proprietary.

wolfgke• 12 years ago

I only wish MATLAB wasn't so expensive and so proprietary.

Try Scilab or GNU Octave.

GFK_of_xmaspast• 12 years ago

I have, and I wish matlab wasn't so expensive.

chollida1• 12 years ago

includes The Intel Math Kernel Libraries, which bring multi-threaded computations to R.

Say no more, I'm sold!

Now that I can use multiple threads, please let me distribute my calculations over multiple machines and I can drop python.

I used to be the compiler developer for a proprietary language that was vectorized like Matlab and R. If R was where it is today in 2000, we probably wouldn't have created a new language(Quicscript) for quants to use.

I can't recommend R enough to anyone doing modelling. It's almost replaced excel for me and, in finance, that's pretty high praise :)

snoman3• 12 years ago

I was looking into them yesterday, and the best source of info on what exactly REvolution R seems to be is from Wikipedia (Their sites is fraught with buzzwords and little info). That is, it's an R distro that has multi-threaded computation as you noticed, but also has the facility to deal with datasets larger than can be stored in memory and supposedly also has some integration with Hadoop.

mbq• 12 years ago

To be fair, free R+CRAN is also capable of all that stuff, not to mention that it can be easily compiled with ICC using MKL (provided that one has a license for them, but most academias have).

jorgemarsal• 12 years ago

DistributedR is multicore and distributed. It may be what you are looking for. http://www.vertica.com/distributedr/ https://github.com/vertica/DistributedR

etrain2• 12 years ago

Check out SparkR - http://amplab-extras.github.io/SparkR-pkg/

nkurz2• 12 years ago

This doesn't negate what you are saying, but I happened to be helping a friend yesterday optimize her already optimized R routine by writing a C++ extension. In this case, a straightforward translation gave us about a 10x speedup, due much less memory shuffling. Switching to intersecting 2KB bitmaps with AVX2 intrinsics instead of non-vectorizable IntegerArrays (average length about 20 ints) gave another ~10x speedup. Profiling with 'perf', staring at the generated assembly, and choosing very specific syntax that g++ didn't mangle gave another ~2x (although Intel's icpc is still about 20% faster, and I haven't yet been able to trick g++ into generating the comparable).

Was this worth it? In this case, I think so. It me took a day and half, and brings her per model runtime down from about 3 minutes to 1.5 seconds. The project requires running about 30,000 models, so the improvement will reduce her total CPU time down from about 2 months to about 12 hours. The AVX2 requirement means she can no longer run it on the usual cluster, but the speedup is enough that she do multiple runs per day on a standard Haswell desktop.

This is to say, while there are great things about prototyping with R, and multithreading can't hurt, raw speed and efficiency may still be issues for some users.

random_number• 12 years ago

Cool result. But:

"Was this worth it?" - looking at the last step of optimisation, it looks like that would be the hardest to do correctly without bugs, hardest to maintain, document and so on, but it produced a gain of only 12 hours of compute time (so cheap it's almost free).

On an 8-5 schedule, 12 hours is 24. Was it really worth, or just fun?

pjmlp2• 12 years ago

R was totally unknown to me until around three years ago.

Nowadays I see it everywhere. Oracle is using R for big data analysis. And there is a version being ported to the JVM.

And Julia is looking quite good.

I see Python getting lots of heat if they don't sort out the issues with CPython.

grayclhn3• 12 years ago

Can someone please provide some context to the submission? Is this something new?

edit: politeness.

_deh• 12 years ago

It looks like they released it yesterday

grayclhn3• 12 years ago

Thanks!

twelfthnight• 12 years ago

Not sure I feel about this. It reminds me of Enthought or Anaconda python or RStudio or RedHat for linux. Should companies be able to 'own' an open source software by funneling a majority of users through their distributions and by funding developers to build the language in a way advantageous to the company? In some way, sure, since it can be mutually beneficial to the company and the users. But in another way, it goes against the spirit of a community, rather than oligarchy, contributing to the software.

EDIT: On second thought, it seems naive to think that any large scale open source software doesn't have some corporate backing. For instance, Google and python / Go. I guess that's not so bad.

snoman3• 12 years ago

RStudio has the Open Source AGPL v3 version that, ostensibly, is only different from their commercial offering in that it there is support for the commercial version, I'm perfectly fine with it.

This MRAN looks to be a similar situation (that is, it's available on their OSS offering Revolution R Open)

pjmlp2• 12 years ago

I find it very positive. Those developers need to pay their bills and not all open source projects can survive from university students building up their CV portfolio.

Once upon a time I was a strong FOSS proponent, during university days. Then I worked in environments that were FOSS friendly and seen up close that making a living out FOSS doesn't really cut it.

The majority of FOSS consumers aren't like HNERS They just want everything free without giving back a cent or any other form of contribution.

voisin• 12 years ago

How does this compare in terms of speed to SAS?

minimaxir• 12 years ago

What functions benefit from the multi threading? The examples they give are matrix PCA and SVD, which is cool, but would this help normal library functions? (E.g lm() and glm())

etrain2• 12 years ago

Various linear solvers (either via normal equations, QR, etc.) all have really fast multi-threaded implementations in, e.g. OpenBLAS. These could directly benefit lm() and glm(). That said - there's no reason why you couldn't already call out to these (multithreaded) libraries with (single threaded) R.

mahmoudimus• 12 years ago

Can someone explain why R is preferred more than Python for modeling?

Is it just because of ggplot2?

msherry• 12 years ago

My 2c is that R has a wealth of libraries and utilities (for statistics, machine learning, finance, etc.) that are not as easily available for other languages. Things like scikit-learn for Python close some of the gap, but looking at packages on http://cran.r-project.org/, it seems that Python might have some catching up to do.

I make no representation as to the quality of the libs available for R, as I don't have a whole lot of experience with it yet.

epistasis• 12 years ago

Quality is all over the map in R, but given the choice between an R library and a Python library for the same recently developed stats method, I would think that the R library has been more widely used, had more attention from experts, and is probably more robust.

strontium_90• 12 years ago

I prefer R because of the wide variety of statistical methods available in base R and CRAN. ggplot2 is nice, but having it in the same system as generalised additive models, robust regression, ARIMA and state-space time series modelling, GARCH packages, and a huge range of other statistical methods is very convenient.

My experience is that if there's a new statistical method developed, it will be often be implemented in R first. (This may be more true for statistical methods than machine learning or other fields).

I don't think there's quite such a wide range in the Python ecosystem yet (although it is rapidly growing, and is definitely a better choice for certain analytics situations).

None of this is to say that you necessarily lose anything other than convenience by using other languages eg Python for modelling.

phloxicon• 12 years ago

It's more about the libraries. It's hard to find a statistical method that doesn't have an R implementation. That said, the quality of these libraries can be very low.

JasonCEC• 12 years ago

It's interesting to me that Open Revolution R supports R Shiny (developed by RStudio). A natively multi-core version of R could be really useful in dealing with the larger applications that are built on shiny, as computations, input, graphing, and DB i/o are all (often) thread blocking.

hamiltont• 12 years ago

Is this anything more than a repackaging of R? If you can put multithreaded libs behind core R functions, why not do it on the main codebase instead of repacking the project?

Stubb• 12 years ago

R has high-quality libraries for just about everything. You're good to go if your need mostly involves stringing together calls to them.

Main problem for me was expecting to find things like namespaces, utility classes, and other things you'd take for granted in more modern languages like Ruby or Java. I loved R for whipping out quick analysis utilities, but writing larger ones became insanely painful. I switched to Ruby for this kind of stuff. The library support isn't as good, but it's reached critical mass and the actual coding side is so much easier.

grayclhn3• 12 years ago

There are namespaces, but you have to put your code in a package to use them.

bipin-nag2• 12 years ago

I checked their site. Revolution R has 4 products: Open, Plus, Enterprise, Cloud

1. Open: "This one’s not a difference at all: Revolution R Open 8.0 beta is based on R 3.1.1. No modifications are made to core R".

Simply put it is a repack, comes with extra packages like Reproducible R Toolkit, and has a mirror for CRAN.

2. Their Revolution R Plus is what is RHEL to linux. They provide technical support on top of the Open distribution.

3. This is where it smells fishy. "Revolution R Enterprise Workstation is licensed for a single named user, and available in two editions:". But is it a modified R version. They mention no change to core for open, but not for this. If they use R which is licensed under GPL how can they sell it ? Else if it is proprietary why call it "R"?

4. They provide assistance in running Revolution R Enterprise on a Server.

rjcz• 12 years ago

3. This is where it smells fishy. "Revolution R Enterprise Workstation is licensed for a single named user, and available in two editions:". But is it a modified R version. They mention no change to core for open, but not for this. If they use R which is licensed under GPL how can they sell it ? Else if it is proprietary why call it "R"?

https://www.gnu.org/philosophy/selling.html

No wonder F(L)OSS has such a bad PR in some business circles.

How are they, or regular folk for that matter, to ever understanding it, if even some of us don't!?

Bootvis• 12 years ago

I believe chollida comment, placed 10 hours before yours answers the question how they can sell it:

It's linked with the Intels commercial BLAS, see here:

http://mran.revolutionanalytics.com/documents/rro/open/#inte...

bipin-nag2• 12 years ago

2 issues here:

1. You can't package GPL stuff into another and then sell the new product.

2. If it is required for Intel's commercial BLAS and they are giving it away for free, it would be a great loss to Intel. So whatever they are giving away must be available for free. Otherwise it makes no sense.

Edit: Open version makes use of non-commercial license MKL, which you can get anyway, see https://registrationcenter.intel.com/RegCenter/NComForm.aspx.... And most likely they are using commercial version for enterprise. But again can you compile R like that and charge for it.

wodzu• 12 years ago

I wish there would be a maven for R. Hope the "Reliable R code" (RRT) can achieve a similar effect. Currently CRAN is heavily broken. Installing exactly the same version of the package will work today but might not work tomorrow.

Revolution R Open: The Enhanced Distribution of Open Source R

Related Stories