In memory map/reduce framework in 25 lines of Clojure

http://clojurewise.blogspot.com/2011/01/in-memory-mapreduce-framework.html

by tebeka • 16 years ago

6 1 16 years ago

clojurewise.blogspot.com

lukev• 16 years ago

To me, this falls into the (rather large) class of libraries that, while a viable library in most languages, are almost too obvious to be worth talking about in a more expressive language such as Clojure.

In fact, to write a "map/reduce framework" for a lisp is to come full circle. Google's map-reduce algorithm was an extension of the well-known map and reduce functions from functional programming to massively distributed computing. To me, it's not particularly interesting to take the map/reduce engine and shrink it back down to its single-machine, functional roots, because that's already been done - it's what inspired map/reduce in the first place. It's called two functions: map and reduce.

That said, the posted code is good for pedagogical purposes, for people who want to understand how map/reduce works. But I'd argue that Clojure really doesn't need a "map/reduce framework." As a functional language, it is a map reduce framework. Almost every Clojure program will probably end up using the map/reduce pattern at some point, almost by default.

In memory map/reduce framework in 25 lines of Clojure

Related Stories