Harlan, new Lispy language for GPU programming

http://blog.theincredibleholk.org/blog/2013/06/28/announcing-the-release-of-harlan/

by mpweiher • 13 years ago

201 39 13 years ago

blog.theincredibleholk.org

Ihmahr2• 13 years ago

This is very much needed to make gpu more mainstream.

phren0logy• 13 years ago

I'm not sure if you are joking (given that it is a lisp, which is notoriously not mainstream), but my limited experience with GPU programming with the existing tools has been pretty painful. I, for one, welcome this development.

bajsejohannes2• 13 years ago

Yes, anything to make it easier to get started. I've tried before, but got lost in figuring out which SDK to download. Too bad this toolkit doesn't help in that process.

Happily, writing this answer I discovered that OS X 10.7+ ships with OpenCL already! Can't believe I missed that last time.

For anyone else wanting to try it, the first example is pretty straight forward:

https://developer.apple.com/library/mac/#documentation/Perfo...

(You'll also want the code examples at https://developer.apple.com/library/mac/#documentation/Perfo... )

burke• 13 years ago

The lisp part isn't necessary (or necessarily helpful), but it's certainly an improvement on embedding C-as-a-string in more C.

...though I am concerned that abstracting this stuff too much makes it less obvious how to optimize for the GPU. This may be a language for people that already have a solid grasp on GPU programming.

Ihmahr2• 13 years ago

Not joking.

vault_2• 13 years ago

Things like NumbaPro[1] and OpenACC[2] are going to go much further towards making gpu programming more mainstream than a lisp dialect.

NumbaPro is a python library that let's you JIT python into CUDA or CPU vector extensions via decorators.

OpenACC is similar (or possibly developed out of) OpenMP in that it takes a directive driven approach to accelerating code. Given a loop you can add a pragma above it with instructions of how it should run a GPU and it will make it happen.

[1] http://docs.continuum.io/numbapro/ [2] http://www.openacc-standard.org/

EDIT: I forgot to mention that I believe we'll be seeing less writing of GPU code directly and more using of libraries that do the hard work for you. Some algorithms on GPUs are quite tricky to get right, and even with a language that makes them easy to express they are non-trivial to design and test. (Nvidia already provides Thrust with cuda which provides a nice STL like library that implements a lot of common routines)

[3] http://docs.nvidia.com/cuda/thrust/

iskander6• 13 years ago

Harlan already has native support for rich data structures, including trees and ragged arrays. Very soon the language will support higher order procedures.

Structured data and nonuniform computations are very hard to express right now on the GPU and typically require a long and terrible process of "creative" CUDA programming. Harlan seems to be pushing the boundaries of what sorts of GPU programs we can effectively/efficiently implement. But, because the syntax is unusual, you're saying it's irrelevant compared to NumbaPro...which is what, a thin wrapper over elementwise array operations?

swannodette2• 13 years ago

Pretty cool, region inference is handled via miniKanren, a relational / constraint logic programming embedding in Scheme (also available in Clojure as core.logic):

http://github.com/eholk/harlan/blob/master/harlan/middle/inf...

eholk6• 13 years ago

We did start out with a miniKanren type inferencer and region inferencer, but I replaced it a couple months back with a unified type and region inferencer. Region inference is definitely an interesting challenge for miniKanren, since a lot of the constraints it introduces are soft. There are many legal region assignments, but deciding the best one requires some heuristics.

swannodette2• 13 years ago

I see, thanks for the update! :)

octo_t3• 13 years ago

Have you looked at GPUVerify[1] from Imperial College to verify the correctness of code produced?

[1] http://multicore.doc.ic.ac.uk/tools/GPUVerify/

BruceIV5• 13 years ago

I read through the user guide in the git repo, but it hasn't been updated for a year, and doesn't tell me what I'm most interested in (namely, what kind of optimizations does the compiler do, and what's the win for this language over CUDA, besides being able to write kernels in Lisp instead of C++) - does anyone have any insight?

BruceIV5• 13 years ago

I found a couple relevant links - a blog post of the author's about GPU optimizations [1], and an early paper on this work [2].

[1] http://blog.theincredibleholk.org/blog/2013/06/10/some-simpl... [2] http://www.cs.indiana.edu/~eholk/papers/parco2011.pdf

eholk6• 13 years ago

I really should update the user guide, since the language has changed a lot in the last year.

I have an earlier post that discusses a couple of the optimizations that Harlan does: http://blog.theincredibleholk.org/blog/2013/06/10/some-simpl...

To me the win for Harlan over CUDA is its region system, which lets you work with more intricate pointer structures in the GPU. For example, there is an interpreter for the Lambda Calculus as one of the test cases, which would be much harder to do in straight CUDA: https://github.com/eholk/harlan/blob/master/test/lambda3.kfc

Very soon, Harlan will have support for higher order procedures, which is also not available in CUDA.

BruceIV5• 13 years ago

Thanks! What kind of performance do you get with some of those more intricate pointer structures? My own research right now is focused on how to build some common abstract data types with less pointer indirection, so as to be more easily vectorizable.

iskander6• 13 years ago

How do you support higher order procedures? Do you do control flow analysis & defunctionalization or are these "real" higher order functions?

s-phi-nl• 13 years ago

This reminds me of the Chapel programming language (http://chapel.cray.com/), which seeks to be a relatively high-level language for all parallel computation.

They differ in that Harlan is just for GPUs, while Chapel aims to be good for all parallel computation. Also, Harlan uses Lisp syntax, while Chapel's syntax is C-based.

Any comments on similarities/differences between them from those who know either one better than I?

SeanDav• 13 years ago

A pity this is not available under Windows (unless I missed something).

eholk6• 13 years ago

It's not, although I don't think it should be too hard to port. I haven't done it yet because I don't have ready access to a Windows machine with an OpenCL-capable GPU.

z3phyr• 13 years ago

Eric is also working on the Rust language for GPU. https://github.com/eholk/rust/tree/nvptx

Harlan, new Lispy language for GPU programming

Related Stories