Clojure vs Scala - Anecdote

http://groups.google.com/group/clojure/browse_thread/thread/b18f9006c068f0a0

by plinkplonk • 15 years ago

163 49 15 years ago

groups.google.com

It's interesting that rewriting in Clojure was "less work and more timely" than migrating to Akka or waiting for Scala 2.10. Any insight as to why? Having written no Scala, I'm not sure what the Akka changes would entail.

jzoidberg• 15 years ago

Akka actors are the accepted production solution in Scala.

I wonder how the author translated actor heavy code to clojure - I guess the closest available concept would be agents.

seancorfield12• 15 years ago

The original Scala code was written back in November 2009. Akka wasn't really the "accepted production solution" back then. There's no question that it is _today_.

We used actors to create a pool of workers in order to run some slow processes in parallel. The simplest equivalent in Clojure was to run the processes via pmap. Cheap, quick and very effective. We didn't have as much control over the parallelism but it didn't really matter for our use case.

We had about 1,000 lines of Scala that we'd developed as we learned Scala. Today we have more experience in Clojure than we did in Scala back then. Writing 260 lines of Clojure with the Scala code as a guide took a couple of days, including writing a new set of unit tests.

Learning enough about Akka to migrate to it would have taken us some time - and would still have left us with three languages in the mix. By replacing this Scala code with Clojure, we only had two languages in the mix, which was also a win.

Uchikoma10• 15 years ago

[edit: See my comment below on how this might be a culture thing]

I wonder how 15 hours in the post become a couple of days here on HN. But of course, the post was written as an anecdote and should be taken as that.

Could you elaborate what makes Akka "a language" for you? I did assume Akka actors are for all practical reasons a drop in replacement for scala actors.

More people should btw. read "Anecdotal Evidence and Other Fairy Tales" in Hacknots book:

http://www.scribd.com/doc/459372/hacknot-book-a4

sreque• 15 years ago

I'm surprised that you chose to use actors to parallelize your code. I always thought they were more valuable for concurrency. Back before Scala 2.8 came out with parallel collections, when I need parallelism I wrote my own pmap-like functions. It was really easy, probably easier than learning the whole concept of actors, let alone an implementation of them, and I got the performance speedup I was looking for.

It sounds to me like you got a large code reduction because in Scala you used a heavyweight concurrency solution to a lightweight parallelism problem.

snprbob862• 15 years ago

I'm not so sure this is a Clojure vs Scala story, as much as it is a Write One To Throw Away story.

"The form of the Clojure code mostly follows the form of the Scala code, most of the same functions"

Well evidently not. Otherwise, the code would not have seen a 75% reduction in lines of code.

What specific transformations resulted in this delta?

rjd2• 15 years ago

I was a little disappointed in the point where he said he rewrote it in 15 hours, and was 75% lighter code... and didn't explain why.

Makes me think is think a few things...

- the original code was bloated and needed a rewrite anyway, and hence was performing badly. Most importantly having written lots of code that has evolved myself over the years... I have taken my own code (written in the same language) and cleaned them unto be half the size of they where before hand.

- The closure rewrite was incomplete as compared to the scala version and totally invalidates his comparison

- would the out of memory issue and performance difference be solved by not using actors with known memory leaks and swapping to akka.

Would love a follow up on this as I'm slowly learning scala as a hobby ATM

seancorfield12• 15 years ago

It's Clojure, not "closure".

Yes, switching to Akka would almost certainly have solved the OoM problems. That's the whole point. We had to switch from the original code. The choices were: move to Akka, wait for 2.10 (when Akka replaces the default actors implementation - and whatever migration was involved), or rewrite. Given that we're getting heavily into Clojure, it seemed a worthwhile spike to create a Clojure version to see how it performed...

Was the original code bloated? I don't think so. I've written a LOT of code over 30+ years in dozens of languages. When the Scala code was written two years ago I was very impressed with its conciseness compared to Java (and other languages). My experience has been that Scala is 3-4x more concise than Java in general (and sometimes as much as 10x more concise) but Clojure is even more concise.

If you're learning Scala, don't let this put you off: Scala is an incredibly impressive language. Odersky and his team have done an amazing job.

seancorfield12• 15 years ago

I'd have to ask how much Scala and how much Clojure you know to challenge my statement about the equivalence of the code and the reduction in size? (no offence intended - a genuine question)

I like Scala. I'd hoped it would become my company's "One True Language". It has a lot less ceremony than Java. I went thru the very painful 2.7 to 2.8 transition. I attended Scala Lift Off! 2010 and Scala Days 2011. I think Scala is very impressive - a (much) better Java. I hope it will succeed and I still follow it closely.

I've always liked functional programming, and Lisp, and Clojure hits a particular sweet spot for me. It also happens to hit a sweet spot for our company - our developers like dynamic scripting languages and Clojure fits that better than Scala. Scala _allows_ you to adopt a functional style but doesn't enforce it (and, to be honest, makes you work at it) - Clojure makes you work at deviating from the functional path.

Clojure has less ceremony than Scala, is more functional (or at least more purely functional), and - in my opinion - far fewer demands on developers in terms of what you must learn to be productive...

snprbob862• 15 years ago

I've written a substantial prototype in Scala, which included half a dozen Neo4j-backed implementations of scala.collection traits. As for Clojure, I've only really toyed with it, but I have written meaty code in several other Lisp dialects. So while I am by no means an expert in either, I'd say I'm qualified to be skeptical of your claims. Furthermore, I have no vested interest in either language. I have a future production project in mind for each of them.

I'm going to ignore the discussion about functional vs OOP, and team sweet spots, and cognitive load of the Scala type system, and all of that. I agree with and disagree with several of your points, but they are not relevant.

Scala, being a statically typed language, has some additional "ceremony" over a Clojure. That's for sure. But, in my experience, that ceremony is a relatively small fixed cost of around 5 to 10 lines per "big scary top-level thinggie" in the problem and solution domains.

In general, I'd expect the Clojure source code to be shorter by about 5%, not 75%. In some cases, either Scala or Clojure will be significantly shorter that the other, depending on whether or not the type system or macros are employed cleverly.

Regarding your specific claim -- "The form of the Clojure code mostly follows the form of the Scala code" -- I am beyond skeptical. I do not trust your account of "mostly the same functions".

Without seeing concrete examples of how the code shrank so significantly, your anecdote is not useful to any reader.

BrandonM2• 15 years ago

> I like Scala. I'd hoped it would become my company's "One True Language".

With all of these exciting languages that target the JVM, I think it makes a lot of sense to have a company's "One True Language" simply be the JVM itself.

That allows a programmer to use Scala, Clojure, JRuby, or Jython to initially write software quickly. Anything that becomes a core library can be rewritten in Java for efficiency and so that bindings can be added for all of the high-level languages. Then projects themselves can be written in whatever language best suits the individual employees.

Does anyone use this approach?

Uchikoma10• 15 years ago

Excellent, the title says anecdote to set the right context. There have been other discussions on how much the knowledge of writing one solution helps in writing another solution in a different languages and what is attributable to the new language itself.

Uchikoma10• 15 years ago

Wanted to add one link to this where I did get the idea from the first time:

http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_swit...

"#7 - PROGRAMMING LANGUAGES ARE LIKE GIRLFRIENDS: THE NEW ONE IS BETTER BECAUSE YOU ARE BETTER"

Uchikoma10• 15 years ago

It takes a little away from the story that it took them a lot of time to discover a widely published performance and memory problem with scalas actors.

"after a lot of poking around, we became fairly convinced it was due (at least in part) to the default actor implementation in Scala"

seancorfield12• 15 years ago

"widely published performance and memory problems with scala actors" - remember the code dates back to November 2009 when that was not common knowledge...

Uchikoma10• 15 years ago

The post was probably a little unclear, I did not understand you poked around to find the oOms in 2009, I just assumed it was more recent as the Clojure post was from September 2011.

DannoHung• 15 years ago

Guess you weren't really using the type system if you were able to translate your Scala code directly to Clojure.

seancorfield12• 15 years ago

Not sure what you mean - if code is statically type safe, it will also be dynamically type safe. Moving from Scala to Clojure is not really affected by the type system.

vrode• 15 years ago

People and their one-liners.

rbranson• 15 years ago

A story presenting objective truth through measured experimentation and empirical data.

schiptsov• 15 years ago

Numbers of lines of code and improved readability alone are worth switching.

As a consequence, less code means less consumption of resources, for Java it is very true.))

Just by looking at the code one could see that Clojure makes the development process for JVM less annoying and frustrating while Scala makes it even more verbose and complicated. ^_^

ambrosebs• 15 years ago

Lisps make it fairly difficult to reason about resource consumption based on code size. And in general, is not a good metric for such a measurement.

foobarbazetc• 15 years ago

Without seeing the code for both implementations, this posting means absolutely nothing.

I wish I could unread these types of submissions.

Clojure vs Scala - Anecdote

Related Stories