Haskell in the Large [pdf]

http://code.haskell.org/~dons/talks/dons-google-2015-01-27.pdf

by kryptiskt • 11 years ago

187 138 11 years ago

code.haskell.org

taeric3• 11 years ago

"Your choice of programming language can have a real effect on these results over time." Probably more accurate to say "Choice of programming team can have a real effect on these results over time."

I want to like Haskell. However, I have given up on wanting to dislike Java. To the point that it is painful to read such things as "you can write Java in any language."

the_af4• 11 years ago

To be fair, that's one sentence in the second to last slide of 42 slides. The presentation, at least what I can see in the slides, is hardly Java bashing :) There are 3 mentions of Java in the whole presentation, only one of which has negative connotations: the one you quoted.

tel33• 11 years ago

And to be extra fair "you can write Java in any language" is much more a dig at some of the bad patterns which Java has become known for culturally than anything about the language itself!

taeric3• 11 years ago

Yeah, the Java bashing I could (and, admittedly, should) have ignored. It is more the crediting the language for the success that I would rather focus on. It is a fairly strong assertion at the end, that I feel needs more support. Back when fewer teams were writing Java applications, I feel similar advantages were felt for it.

Of course, I still pine for lisp, so I can not claim to have no biases. (Well, that and MMIX)

stonemetal2• 11 years ago

I read that rather differently. I see it as a poke at management that wants to make the move because of hype not a real understanding of what it will take, and not put the effort in training\hiring devs who have as he puts it "have taste and guidance".

olenhad• 11 years ago

I wonder what's the primary reason for their "Mu" compiler adopting a "strict-ish" evaluation strategy.

tel33• 11 years ago

A pretty common idea in Haskell-land is that "the next Haskell would be strict". Laziness was once a dreamy ideal execution strategy, but thanks to Haskell the practical tradeoff is better understood.

At the same time (as the slides note) the strict/lazy divide is hardly decided! As many people who would rather Haskell be strict are willing to defend laziness to the death for being the key reason Haskell is so composable. [0]

So Lennart Augustsson—the author of the first standard's compliant Haskell compiler, I believe—wrote up Mu and made it strict. There is probably a justification in SC's case for why it was an experiment worth trying.

[0] At the very least it's indisputable that laziness forced Haskell's hand with purity and that ended up being an enormous win in everyone's opinion.

the_af4• 11 years ago

This is interesting, because the well-known paper "Why functional programming matters" identifies two key aspects of FP: higher-order functions and lazy evaluation. I wonder if John Hughes has reviewed his opinion on this, or if the FP community thinks the paper is no longer an accurate insight into FP...

In particular, I'm thinking of the last paragraph of Hughes' conclusions:

"It is also relevant to the present controversy over lazy evaluation. Some believe that functional languages should be lazy, others believe they should not. Some compromise and provide only lazy lists, with a special syntax for constructing them (as, for example, in SCHEME [AS86]). This paper provides further evidence that lazy evaluation is too important to be relegated to second-class citizenship. It is perhaps the most powerful glue functional programmers possess. One should not obstruct access to such a vital tool."

Maybe it turned out that practical evidence has shown that lazy evaluation wasn't as important for modularity as Hughes thought, or at least that its drawbacks have been found unacceptable in practice?

rtpg• 11 years ago

It's so weird to me that in a world of more asynchronicity than ever we want to bring Haskell to strict-land.

On the opposite end, there is so much boilerplate optimisation out there to get around the strictness of other programming languages that would be solved with a non-strict mode

ThatGeoGuy• 11 years ago

I tend to feel this is one area where perhaps Scheme had the right idea (if you ignore set!). Now, I don't know a whole lot about Haskell, but one of (IMO) the elegant parts of Scheme is that you can choose when and where to use streams instead of lists.

The main benefits of streams vs. lists is usually composability, but also that delayed computation can save you from computing something you'll never need. In this way, I think that being able to choose when and where you apply laziness is a huge gain. Now, of course this is where people will complain that streams are either a bad abstraction, or that doing stream-cons vs cons is annoying, but I like to think "what if it was the other way around?". Namely, if you had streams used by default (a la laziness in Clojure), but could switch to lists when it was really necessary or made reasoning about your code easier.

Is there a language that does this smoothly? From what I understand, to work around laziness, you typically have to bend over backwards in Haskell, but if there was a good way to just transform lazy data structures into strict structures similar to the relationship between [stream]-map or [stream]-cons/car/cdr and the list equivalents, I think it'd be pretty exciting. Though to be honest, I'm not sure how this would interact with Monads / Type Classes / etc, especially if you have Type Classes relying on both lazy and non-lazy structures.

14113• 11 years ago

It may be that their applications have high memory usage, which lazy evaluation would make slightly worse. Alternatively, they may have found that the applications that they tend to run benefit from a slightly stricter evaluation strategy, and have thus changed the compiler to better reflect that...

jlarocco• 11 years ago

I don't know, but in my limited experience, lazy evaluation makes memory use worse (usually not much), but more importantly makes performance (time and memory) harder to reason about, because you don't easily know when something will actually evaluate.

Besides that, there's also not much practical gain from it, IMO. One commonly cited benefit is a function that doesn't use all of it's arguments, therefore saving computation time when they're not evaluated. But realistically, an unused parameter should probably be removed.

gizmo686• 11 years ago

Some level of lazyness is nessasary for Haskell to work. For example, you can do:

    a=1:a
    main = show $ head a

With lazy evaluation, this will print "1", but with strict evaluation, this program will never terminate as it attempts to fully evaluate "a=1:a", which creates an infinite list.

But realistically, an unused parameter should probably be removed. You also have cases where a parameter is only used some of the time.

tel33• 11 years ago

Generally the argument is never around saved computation but instead around composability. Lazy languages ensure that everything behaves like a value and in that domain operations compose much more effortlessly. You can't reason about operation as easily, so you don't, and the language can cope with making that work more or less correctly.

Which is definitely suboptimal in some cases!

I think honestly the goal should be reasoning about evaluation order statically instead of trying to find some clever argument such that laziness or strictness is clearly "correct".

the_af4• 11 years ago

But realistically, an unused parameter should probably be removed.

Think about the function if-then-else, which you may be familiar with from your favorite language :)

    if-then-else true branch1 branch2 = branch1
    if-then-else false branch1 branch2 = branch2

Obviously you don't want both branches evaluated in any given invocation, and obviously you cannot remove the unused parameter. Note that the purpose is not to "save computation", since for example branch1 may be undefined if cond is false!

When using a language with support for lazy evaluation, you encounter this kind of functions all the time.

dragonwriter• 11 years ago

One commonly cited benefit is a function that doesn't use all of it's arguments, therefore saving computation time when they're not evaluated. But realistically, an unused parameter should probably be removed.

How do you do that when the values of the other arguments determine whether or not that argument will be used?

stonemetal2• 11 years ago

an unused parameter should probably be removed.

It doesn't have to be completely unused to be avoided. In a case like:

if a then b + c else c + d

a and c are always evaluated, only b or d is evaluated not both. Removal isn't an option because b and d maybe used.

michaelochurch3• 11 years ago

Not sure, but having encountered laziness in Clojure and Haskell, it can be non-intuitive and it can be a bitch to debug. It allows for some conceptual beauty, though, and there are certainly some use cases in which laziness is the right behavior. The question is what should be the default; both ought to be allowed. In Haskell, they are, but you start using bangs a lot (e.g. Point !Int !Int and the ($!) operator instead of ($)) and there are also shallow vs. deep considerations, because forcing a thunk only evaluates it one constructor-level deep-- to "weak head normal form".

That said, I much prefer Haskell's laziness or Clojure's laziness in seqs over the broken laziness in other languages. There's a lot to like about R's libraries but... fuck this:

    > Map(function(x) (function(y) x + y), 1:5)[[3]](0)
    [1] 5

Python can be tricked into the same evil if you build closures in a loop. Haskell doesn't have that, thankfully.

tel33• 11 years ago

Laziness in data structures has the biggest benefit in the spine. Leaf laziness is just more surface area to hide unexpected thunks. If you really want that, do something like

    data Box a = Box a

    type Lazier a = Tree (Box a)

Generally, I find that a little habit around leaf strictness ends up eliminating laziness concerns entirely until you get to explicit concurrent programming and need to think carefully about what thread is forcing what execution.

tome5• 11 years ago

https://news.ycombinator.com/item?id=8964949

melling• 11 years ago

Has anyone used both Haskell and OCaml (or F#?)? How do they compare in practice? I've been wanting to put some time into a functional language and I've been debating between Haskell and OCaml. Because of F#, OCaml seems like it might be the more practical language (i.e. direct job opportunities). However, excluding F#, Haskell does seem to much more popular than OCaml.

codygman5• 11 years ago

How do they compare in practice might be too broad a question, I'm willing to bet you'd get more responses if you asked for comparisons in certain use cases.

tel33• 11 years ago

Either one is great. Short of it is that OCaml and Haskell will both teach you a lot of the same things. OCaml has better modularity features (namely, genuine modules which are replicated almost nowhere else) and Haskell has better purity features (namely... purity).

It turns out that sophisticated functional programming essentially stresses that good modularity and good purity are both killer features and techniques and they should be in heavy use all of the time. In OCaml you'll have an easy time expressing the exact modularity concerns you find important, but will struggle a bit to manage purity explicitly. In Haskell you'll find it beautiful to express pure and typed-effectful computation as much as needed, but will struggle a bit with expressing modularity as you'd like.

Haskell also has the whole typeclass system which is really, really interesting and valuable for many kinds of expressiveness. It's sort of an oddball feature, but interesting to see the impact of.

Personally, I find strict purity more important than expressive modularity. But if anyone could really put them together in a way that worked it'd be great. I'm not so sure such a language exists today, though.

thinkpad204• 11 years ago

Personally, I find strict purity more important than expressive modularity. But if anyone could really put them together in a way that worked it'd be great. I'm not so sure such a language exists today, though.

There is Ur, which has both ML-style modules and Haskell-style type classes. It's also pure and strict. And it has advanced type-level programming features and record types. There's a lot to like.

There are a few problems with it, though. It's mostly a one-man project. The compiler is immense, complex and almost entirely devoid of documentation (it's pretty amazing actually, like 1 comment per 1000 lines or so), so contributing is hard. It's very difficult, if not impossible, to use the language for anything but web development: there's no way that I know to execute a program directly, rather than as a sub-program of the ur/web framework. The standard library is tiny and doesn't even include `print`. But even with those warts it has enough good things about it that it's worth checking out.

michaelochurch3• 11 years ago

I tend to think of OCaml as a functional C. It's strict and not purely functional. You can write for-loops and while-loops if you want (but you shouldn't) and use refs to have mutable state (but you shouldn't). It has a better module system than Haskell, but doesn't have type classes. It has functors, which are equivalent-- OCaml's functor is an operation over modules and only very loosely connected (through type theory, which isn't essential to being proficient in either language) to Haskell's Functor type class-- and better in some ways and worse in others.

Haskell is more expressive and has a much more powerful type system, but it's probably harder to reason about performance.

OCaml's biggest issue (note: I may be out of date on this, since I haven't heavily used it since the late 2000s) is the GIL. This probably limits your ability to use it for multithreaded programming, but it can compile down to extremely fast single-threaded executables.

acjohnson55• 11 years ago

Can you compare that to Scala? Scala also has non-pure constructs built in, but most people tend to opt in to its purity. And I don't know much about OCaml, but it seems like Scala may have better type system parity with Haskell's type level features.

sea6ear• 11 years ago

This book actually teaches C in relationship to previous assumed knowledge of Standard ML (not quite OCaml, but close).

http://eprints.eemcs.utwente.nl/1077/02/book.pdf

I'm not sure I agree with all of the conventions, but it's interesting to see a deliberately functional approach applied to C code.

tel33• 11 years ago

I'd really love to be able to think of ATS as "functional C"!

(Not that I actually do. Maybe someday).

eru9• 11 years ago

I used both Haskell and OCaml professionally.

OCaml's strictness makes some things easier to reason about. Haskell is a nicer language and has more momentum.

mercurial5• 11 years ago

OCaml has, IMHO, strong syntactic advantages over Haskell which have a major impact on readability:

- the way you access records (which also means it's not going to clutter your namespace)

- named arguments

- default values

- not an OCaml-the-language property, but there is much less of a race in the OCaml ecosystem to write the shortest variable name possible and accumulate the most ASCII operators

Also, the great module system, with easy abstract types or "public" types with private constructors. By contrast, having to export accessors (if using a lens library) is embarrassing.

stream_fusion• 11 years ago

My experience is that, Haskell syntax is a dream compared to Ocaml, with 'where' clauses, beautifully simple lambda syntax, do notation support, . and $ composition, typeclass operator overloading.

Ocaml is more practical in a hard to explain way. Much more predictable in terms of it's memory and cpu use with non-lazy default evaluation, and I found much better performing on general code.

mercurial5• 11 years ago

Funny, I have the opposite experience. Where clauses force you to look down and then up again (bonus if you managed to use both let and where in the same function!). The lambda syntax is a matter of debate, writing "fun" instead of "\" doesn't bother me and is, IMHO, clearer. OCaml has an equivalent of $ with @@, but I haven't seen it used very much, since chaining functions with |> is so convenient.

OCaml does not have generic ways of doing monadic stuff (do notation or generic >>=), which means in practice that monadic code looks like "fun1 x >>= (fun r -> )" which is kind of awkward. It also doesn't have typeclasses (yet).

However, one thing you're unlikely to find in OCaml and which is unfortunately not uncommon in Haskell is functions with a long list of positional parameters (since you have named arguments) or different versions of the same function with a "_" suffix, depending on various defaults (since you have named arguments). It makes a tremendous difference of readability when dealing with complex code.

Also, interface files (.mli), while a bit of a pain to maintain at times, give a very clear idea of what interface a given module exposes.

All in all, I find OCaml code much easier to read, not to mention less "clever" than equivalent Haskell code.

evincarofautumn2• 11 years ago

I have used all three languages professionally—Haskell & some OCaml at Facebook, and some F# at Xamarin—and would recommend learning Haskell.

There’s a strong community around Haskell, with a decent library ecosystem and high-quality implementations, and learning it will make it easy to pick up F# or OCaml—the reverse is not necessarily true.

F# is a decent choice if you want to write functional code in .NET land, but Visual Studio and MonoDevelop support for F# aren’t as good as for C#.

OCaml is a good language, but I would recommend against it. Like F#, it has some syntactic and semantic conveniences that Haskell lacks. However, the community is small, there are relatively few libraries, and the runtime falls down when you need to do anything with large arrays or floating-point math.

codygman5• 11 years ago

OCaml is a good language, but I would recommend against it. Like F#, it has some syntactic and semantic conveniences that Haskell lacks.

I haven't had a chance to use OCaml or F# yet, what syntactic and semantic conveniences are you talking about?

spopejoy• 11 years ago

“Make illegal states unrepresentable”

Types pay off the most on large systems. Architectural requirements captured formally.

This more than ANYTHING else is why I want to move enterprisey app code to Haskell. Having worked on numerous ginormous enterprisey systems -- which are usually doing pretty straightforward things, just at scale, and needing to be maintained by non-brilliant developers -- I can say pretty securely that north of 95% of the invariants could be lifted into the type system, making run-time errors a thing of the past.

Also, in most frameworky big systems, you WANT to stop devs from "just doing IO" or pretty much doing anything without a strong contract around it. Monad stacks do wonders for circumscribing your computational context in an app.

I wonder why they rewrote Aeson though ... perhaps before it was mature? Aeson's pretty awesome.

carterschonwald• 11 years ago

It might have to do with the way the default generics for aesons fromjson and tojson instances for record types are defined? Or maybe the lack of incremental streaming read write support in the aeson data model? Or maybe they wanted better date or integer support? It'd be pretty easy to ask don or Lennart I'd imagine.

I have to say that I definitely enjoy using haskell when faced with dirty complicated data, which I'm told happens a lot in larger organizations.

eru9• 11 years ago

Monad stacks do wonders for circumscribing your computational context in an app.

Wait until you discover the power of Applicative Functors. They are more restricted in their operations than Monads, so they compose better and allow for analysis.

davdar• 11 years ago

I'm curious, what do you mean they compose better and allow for analysis [better than monads]?

dkarapetyan5• 11 years ago

Mathematically speaking applicatives are less restrictive since more things can be applicatives than monads. A monad is a "very special" kind of functor and that is why a composition of monads often fails to satisfy the monad laws. The least restricted thing in this hierarchy is the functor since composition and other kinds of operations always give you back another functor.

fsloth2• 11 years ago

A sidenote: There is nothing in Java/C++ etc that would stop one from making only legal states representable. It's just easier and more obvious in ML family...

lomnakkus5• 11 years ago

That's only true in the most trivial Turing Tarpit sense, though. Ease of use and defaults matter in practice.

E.g. you can do compile-time exhaustiveness-checked pattern matching using the visitor pattern in Java/C++, but try doing that in practice and the amount of boilerplate just becomes unbearable -- and actually obscures (rather than illuminate) the essence of the data structure.

There's also the lack of higher-kinded types. (Or just plain syntactic pain of using HKTs e.g. in Scala.)

pjmlp2• 11 years ago

Another thing is contract programming, but it is really hard to sell to managers where shipping cheap outweights quality.

I don't envision FP or contract based programming in enterprise offshore code.

codygman5• 11 years ago

Imagine a super-charged version of the method where Java Architects make class hierarchies and interfaces for offshore/contract programmers.

I think FP (and contract based programming) would do a much better job than Java for varying qualities of programmers on very large projects.

marmaduke5• 11 years ago

invariants could be lifted into the type system

Is there a reason this is only possible in Haskell, or does Haskell just make it super convenient / idiomatic?

pdpi• 11 years ago

The way I see it, immutability and purity are hard requirements for actually enforcing the contract, and a very robust type system is then a requirement to go from code that makes you want to shoot yourself in the head, to code that's incredibly expressive and easy to reason about. Haskell just happens to sit in that exact niche.

yummyfajitas2• 11 years ago

Haskell/Scala/etc makes it easy to nudge other developers into avoiding mistakes. A concrete example:

I had a Scala system (build on Scalaz, which is a library providing Haskell for Scala), and one of our core types was DBTransaction[_] (a monad). A developer (a skeptic of the type system) was complaining to me about all the excess work he needed to go through, how he couldn't get properly construct a LazyStream[Foo] as a result of this.

He wanted to construct a DBTransaction[LazyStream[Foo]], call runTransaction on it, and get the LazyStream[Foo] out. Then he was going to call f(lazyStream). The compiler just wouldn't allow this, so his "workaround" was to instead call lazyStream.map(f).

Turns out this workaround prevented a runtime error. If he did get his LazyStream[Foo] out, generating the next element in the stream would have called resultSet.next() after closing the connection.

This sort of thing happened quite often. People would complain that the type system made it harder for them to do what they wanted. They'd ask the FP "guru" types how to fix it and the "guru" would point out that what they wanted to do was fundamentally unsafe.

rprospero• 11 years ago

A toy example would be the following C code

    #include <math.h>
    
    double sine(double x){
      reformat_hard_drive();
      return sin(x);
    }

which can be easily translated to Haskell

    sine x = do
      reformat_hard_drive
      return (sin x)

So we can write the same stupid function in both languages. The bit that's specific to Haskell, as opposed to just any typed language, is that sine and sin have different types. sin is (Double -> Double) while sine is (Double -> IO Double), so swapping in sine for sin causes the code to fail to compile. Comparatively, you can drop in a random sine anywhere in your C code and the compiler will happily chug right along. While I've seen ways of implementing various invariant catching techniques like the Maybe type in C, I haven't seen anything that would catch the sine function.

Now, as I said, this is a toy example. The real solution would be to fire whoever wrote the sine function. However, in larger systems, it can be translated into invariants such as declaring that a function cannot access the database or send packets on the network.

dkarapetyan5• 11 years ago

I see. So all you need are compiler/interpreter experts that can turn any problem into a interpreter/compiler problem and you're golden.

I'm not saying the approach is not worthwhile but how exactly does this generalize to other workplaces where there is no critical mass of such experts? I mean they have their own compiler for Haskell for Pete's sake. I would also like to know how many of the core team members have PhDs and MScs. We can check off Don and Lennart. Maybe Don is really measuring the effects of PhDs in language/compiler design on how to manage complexity?

Alternatively, Facebook has been experimenting with Haskell and OCaml. Seeing their case studies would be valuable as well.

tel33• 11 years ago

I think "interpreter/compiler" problems arise way more frequently than people expect. It's just they're often not called that until they're being tackled by interpreter/compiler experts.

In particular, you can think of things like message passing, interpreter/command patterns, and anything which uses reflection as being at least a little like an AST/interpreter pattern if you look at it in the right light. Then, if you have a team of compiler writers they can make a big win there.

dkarapetyan5• 11 years ago

I completely agree. When viewed from the right angle almost everything is a combination of some "VM bytecode" and a "compiler" targeting that bytecode. Greenspun's tenth rule applies to anything large enough. That's not what I'm getting at.

What I'm getting at is that when you have a bunch of problem solvers well versed in programming language design and theory the programming language at that point is no longer relevant. Standard Chartered doesn't have all that software because Haskell somehow gave them superpowers. Standard Chartered has all that software because a bunch of PhDs chose Haskell to write it in and along the way Lennart wrote another compiler because why not, the man is good at it. So instead of being a case study in how to apply a functional language to solve problems this is really a presentation about what kind of people you want to hire to build systems, i.e. PhDs, MScs, and MDs (apparently).

Imagine if the title of this post was "PHP in the large". Would anyone take it seriously? Does anyone really believe PHP is a great language to build large systems in? No, everyone would immediately jump on the fact that all of it is running on HHVM and there are bunch of smart folks optimizing the hell out of it. Same here.

The metapoint here is really a quote by Rich Hickey from a David Nolen talk/blog post - "Not everything is awesome". There are few hidden gems in the presentation when concessions are made by including Any type and re-inventing a lot of Erlang style process management. I wonder why more trading companies don't leverage Erlang and Dialyzer?

The relevant post for the quote: http://swannodette.github.io/2015/01/09/life-with-dynamic-ty....

eru9• 11 years ago

I would also like to know how many of the core team members have PhDs and MScs.

Don't forget the MDs! (Not joking, one of the people Don's team has an MD.)

I used to work for Don at Standard Chartered. Getting good Haskellers seemed way easier than the hiring efforts of my current employer (Google) focussing on more traditional languages.

But I guess, that's mostly a function of pent up demand for Haskell jobs.

dkarapetyan5• 11 years ago

Where is all this pent up demand? I mean I get what you're saying. Most academics know Haskell and Standard Chartered hires academics but there is a bit of circular thing going on there.

boothead3• 11 years ago

I second this. It's embarrasingly easy to hire Haskellers (especially if you can hire remote or are in a tech centre like London/Chicago/NY etc)!

pjmlp2• 11 years ago

So all you need are compiler/interpreter experts that can turn any problem into a interpreter/compiler problem and you're golden.

Any text processing, symbolic data structure manipulation can be reduced to a compiler phase, it is just apparently many CS degrees don't teach it properly.

Alternatively, Facebook has been experimenting with Haskell and OCaml. Seeing their case studies would be valuable as well.

Microsoft Research also does lot of FP related work.

dkarapetyan5• 11 years ago

Microsoft as well. I don't know what kind of production systems are built with F# but maybe there are a few.

thu2• 11 years ago

"You're golden."

Indeed you are: you can use Haskell and GHC and reap the benefits of programming language and compiler experts. You can read the slides again and see that many things explained there are possible with GHC and don't require Mu.

I guess you can make a similar comment about PHP and Facebook. Facebook do have experts in compiler/interpreters to change the language/implementation to make it good at their scale.

michaelochurch3• 11 years ago

I'm really happy to hear about Standard Chartered's success with this, and I want to know more. This is really promising stuff.

My current company is looking into our "next generation" platform for when our datasets exceed what we can do in R. R may not be the best language, but it's great for exploratory data science, has the best or the only library out there for some ML purposes, and we've done a lot of things to make it production-worthy (on small- and medium-data). We'll probably need to involve something else, down the road, when our data sets get larger than what fits on one box. The leading candidates are Haskell, Clojure, and Scala (Scala because of Spark). I'll have to evaluate the languages fairly and relative to our needs, but I hope Haskell wins for a number of reasons, including the fact that Chicago + Haskell is an unfilled niche and we'd attract a ton of talent.

For those who've taken Haskell this far into production: have you encountered any negatives? Are there any times when you think it might be better not to have a strong type system?

To me, the biggest drawback of Haskell isn't anything intrinsic to the language, but the amount of stuff it forces a person to learn. For me, that's a fun challenge... but trying to convince 110 programmers to use a language that forces I/O into a set of types (loosely) called a monad seems like an epic task. Clojure has the advantage of being simple and beautiful once you get past the parentheses. Haskell is demanding and frustrating for the first 6 months (and pays off handsomely later on, but this can make it a hard sell).

Also, how does the Any type in Mu (if anyone familiar with it is here) differ from Data.Dynamic?

ajtulloch• 11 years ago

I'm sorry - I love Haskell, and it's a great choice for a lot of domains, but it's a poor choice for numeric/scientific computation. The ecosystem just isn't there yet.

Here's a recent r/haskell thread discussing this: http://www.reddit.com/r/haskell/comments/2rsxrb/is_haskell_a...

tel33• 11 years ago

Exploratory data analysis isn't killer in Haskell. I think types can help a lot here (god knows the amount of pain that R/Numpy spaghetti has inflicted) but the jury is still out for how to do it well. Row types seem like an important technology which gets a little bolted on to Haskell---both exciting and unclear at the moment.

https://acowley.github.io/Frames/

On the other hand, if you've got a solid data pipeline you're scaling then there are grand tools for streaming data in Haskell. You're in the complete sweet spot for strong typing. You can also drop down into C easily when speed is an issue. Matrix libraries exist for the intrepid user. HMatrix is BSD now so you can at least touch LAPACK.

dasmoth• 11 years ago

Not something I've used myself yet, but is there any particular reason Julia isn't on that list?

yrlson• 11 years ago

I haven't checked on Julia for like a year, but back then Julia didn't offer anything above and beyond R when it comes to "big" (really just not fitting into memory) data. As it is Spark with Scala is both faster and more convenient than R or Julia.

boothead3• 11 years ago

I've put a large Haskell app in production, and previously to that have seen the reasearch process at a couple of hedge funds. I'd have a couple of suggestions:

* You don't need everyone to be a ninja Haskeller. Get a couple of early ninjas to flesh out the architecture and you'll find that you'll be able to "fill in the gaps" around them with less experienced people (a bit like yummyfajitas experience above). FWIW I very quickly became one of the fill in the gaps people ;-)

* If you can move to stream based abstractions, you'll be onto a winner. Event streams are inherantly immutable, and force you into a much purer data model. See the "unified log" linkedin blog post or read [1] for more

* I don't think that you'll be able to retrain 110 programmers in Haskell. While I think that anyone with the right mindset can learn it, there will be a significant portion of any team who lack that mindset.

Hope that helps. You know where to find me if you'd like anyone to help you sell adopting Haskell to your management. :-)

[1] http://manning.com/dean/

tome5• 11 years ago

I haven't encountered any negatives, by which I mean I never wished I was using another language. In fact if I'd been using any other language I'm confident I would have wished to use Haskell. (This is for reporting software using Opaleye to generate multi-hundred line Postgres queries in a composable and type safe manner.)

saosebastiao• 11 years ago

I've toyed with Haskell, ultimately moving on to the Ocaml/F# camp. There are only two things that I miss from Haskell without an appropriate equivalent or easy workaround: Type Classes, and Higher Kinded Types.

This big roadblock that everyone claims with Haskell, Monads, didn't give me any problems at all...even if it didn't make any sense to resort to them to do something as trivial as IO. What really turned me off more than anything was the combination of the academic focus of the community combined with this weird culture that I could only describe as a cleverness competition.

I'm a pragmatist, and it is obvious that real software can be created with Haskell, but it doesn't really come through in the tutorials or books. There is a lot of "Look, this is really cool!", but rarely a follow up with "and this is why it matters!". Everything seemed to revolve around cool tricks with no practical concern behind them, and then in the comments you inevitably find comments from other Haskellers claiming to do it just a little bit more cleverly using <<lenses, GADTs, zippers, continuations, or some other obscure abstraction>>.

Ultimately it was Ocaml and F# that taught me all of the really important lessons of the ML family, despite the relative lack of learning resources out there. It was there that I found the benefits of making illegal states unrepresentable, expressive pattern matching, type inference, composition over inheritance, etc.

It is a shame, because as languages and runtimes, they probably are inferior to Haskell. Ocaml does fine with concurrency, but parallelism is a disaster. Its class system, its one major distinction from SML, is mostly considered a code smell. Its stance on operator overloading requires me to keep a table of operators handy, instead of the usual intuitive ones. And F# has the sane operators and parallelism story but doesn't have SML/Ocaml Functors, and is still a Windows-first ecosystem (Mono has gotten a lot better, but its still a kludge).

Ironballs• 11 years ago

The Windows-first status of F# is changing. There have been major improvements in this recently, most notably, the whole compiler being open sourced and its development moved to github[1], and .NET is going to be available on Linux[2]. This is not reality yet, but it will be during this year.

[1] https://github.com/microsoft/visualfsharp [2] http://www.hanselman.com/blog/AnnouncingNET2015NETasOpenSour...

lucian1900• 11 years ago

It is indeed not bad on unix. The only major flaw left is the lack of higher-kinded types.

joeyh• 11 years ago

The "and this is why it matters" is often left to be inferred by the reader. ;) General kinds of reasons include "preventing (whole classes of) mistakes" or "writing even more reusable code" or "making things fast".

I know the kind of thread you mean -- it happened to me last weekend https://www.reddit.com/r/haskell/comments/2tl8v1/making_prop... . But I really appreciated those suggestions about even more clever ways to make my code better, even though I have so far not incorporated any of them.

It helped me learn more about the shape of a space I'm only starting to explore, and since I'm exploring it for real-world reasons, the very abstract and hifaluting suggestions were easy to relate back to real-world concerns. Which is a great way to learn.

And I suspect several of the commenters there were commenting for exactly that reason -- if there's one thing the haskell community excells at, it's noticing when someone is in a receptive state (or coaxing them into one), and helping them learn.

mercurial5• 11 years ago

This big roadblock that everyone claims with Haskell, Monads, didn't give me any problems at all...even if it didn't make any sense to resort to them to do something as trivial as IO.

That's a bit uncharitable. You're likely going to end up working with Lwt (monadic concurrency/IO library) as soon as you need to do something interesting with IO in OCaml.

Everything seemed to revolve around cool tricks with no practical concern behind them, and then in the comments you inevitably find comments from other Haskellers claiming to do it just a little bit more cleverly using <<lenses, GADTs, zippers, continuations, or some other obscure abstraction>>.

While I agree that showering code in mathematical abstractions doesn't make for readable code, a zipper is not "cleverness for cleverness' sake". It lets you manipulate easily manipulate an immutable tree (and it doesn't decrease readability).

Ocaml does fine with concurrency, but parallelism is a disaster.

Hopefully the multicore runtime will turn into something concrete eventually :)

It's also lacking real type classes, at least until modular implicits are ready.

jongraehl• 11 years ago

I wonder why they don't just use GHC with some customization to make the default eval strategy less lazy. I understand why someone might prefer more predictable (strict) evals by default.

tel33• 11 years ago

Lennart Augustsson is a skilled compiler writer, especially for Haskell, to say the least. Mu does more than just strictify Haskell—the code portability stuff is pretty unique as well, for instance.

eru9• 11 years ago

Starting from scratch, building on GHC might have made sense. There were historical reasons for going with their own compiler. (Not sure, how much I can say.)

And Lennart pulled it off.

tome5• 11 years ago

They were targeting an already existing runtime for on older in-house functional language, and the runtime only supported strict evaluation.

thu2• 11 years ago

Lazy vs strict evaluation: Lennart, the author of the Mu compiler, has written about it: http://augustss.blogspot.co.uk/2011/05/more-points-for-lazy-...

boothead3• 11 years ago

This

    An awful lot of data mining and analysis is best done with relational algebra

Is an interesting statement. Would anyone care to elaborate?

tome5• 11 years ago

I think it means that a plethora operations you perform on sets, lists, maps, hash maps and indeed collections of all kinds are just specific implementations of the general "relational algebra" operations.

Haskell in the Large [pdf]

Related Stories