Haskell in Production: 12 Months Building ApiEngine

http://mth.io/talks/haskell-in-production/

by lelf • 13 years ago

136 61 13 years ago

brotchie• 13 years ago

Awesome. Is there a video recording of your presentation anywhere online?

What strategies did you use to combat the lack of stack traces, especially in production?

I was happy to see that you are fellow Brisbanites. I'm a bit out of the "local loop", is there much of a local Brisbane start-up scene?

alberich• 13 years ago

I think i'm missing something. On slide 4 it says (the code) it has 0 exceptions and is robust, but then on slide 7 it says that (the system) it crashes without stack traces? Is that right?

What did the OP meant by being robust, when the code crashes unexpectedly?

Guvante• 13 years ago

Likely that is referring to the fact that when it crashes, it crashes in a way that can be interesting to debug. I took it as a comment about development vs one about production.

alipang• 13 years ago

In Haskell one typically eschews exceptions in favor of for instance the Error monad, as you don't get a typical stack trace in a lazily evaluated language.

Probably they have experienced some exception during development, but have had no exeptions in productin.

bermanoid• 13 years ago

The cynic in me says that unless Haskell has a magical "do the right thing in every situation, with automatic full knowledge of your application's business requirements" operator, if there's no specific logic in the code that responds to errors, they're just ignoring them (or else they're building a very bare bones app - any nontrivial piece of software will have potential error conditions out of its control that it has to react to in very specific ways that depend on the app itself).

In which case I could produce an equally "robust" Java program by wrapping everything in a try/catch and not logging anything. Except that in Java, or just about any other language, with a single method call I could obtain that elusive stack trace that OP is sorely missing. Which, in real applications (which, to be fair, I've never been masochistic enough to write in Haskell), I always do, and it significantly improves our ability to track down the remaining unanticipated conditions that our error handling logic hasn't already handled in well defined ways. And that sort of work should consume most of anyone's time spent developing, because the happy path is the easiest piece of the puzzle - engineers handle all code paths, hackers handle the most important ones, and hacks only handle the best case scenarios.

I'm not really getting what Haskell has helped here, apart from making the article more upvote-able.

That said, the presentation slides don't give me even a vague sense what the real thrust of the presentation was, so I'm probably missing the substance. I take issue with this being posted without any additional context, but I'll grant the benefit of the doubt and assume that there may actually have been something worth listening to if we didn't just have the slides to look at.

gtani3• 13 years ago

I think s7 was emphasizing issues getting what strict/eagerly evaluated languages refer to as stacktraces

http://hackage.haskell.org/trac/ghc/ticket/3693

http://www.haskell.org/wikiupload/6/6c/Hiw2012-simon-marlow....

tome2• 13 years ago

Reddit discussion here:

http://www.reddit.com/r/haskell/comments/1etssw/haskell_in_p...

laumars4• 13 years ago

I appreciate the need some people feel to make modern-looking web pages, but the format of this really is just a triumph of style of substance:

* I have no idea how far through the content I am

* The lack of verbosity means I'm constantly having to apply guesswork to their statements (ie an explained point will tell the readers why Haskell gave them better communication. A slide show like that with just headings mean I have to guess the reason Haskell had those benefits)

* It's scales terribly (at least on Opera, some of the content fell off screen - so I had to maximize the browser. But this might just have been an Opera bug. In either case, this wouldn't have been a problem with a more traditional blog layout).

* The navigation requires guesswork

* And most annoying of all, it doesn't actually save the reader any time by displaying the content like that than it would have done with blog post as the jarring navigation presented with slides require the reader to pause before each page to understand how to read the slide, and then pause after each page to apply the aforementioned guesswork. With a blog post, the information is presented in a more natural communicative way which makes it quicker to digest the information.

It's a great pity I'm having to pick post a negative comment about the presentation of the submission rather than comment about the point being raised - particularly as I've been very interested in Haskell lately. But I found even my interest in the subject wasn't enough for me to put up with this slide show layout and thus I ended up closing the browser tab before the end. :(

aidos• 13 years ago

In terms of how far through the content - there seems to be a little marker (in yellow) that runs along the bottom of the screen. I suspect, depending on the browser, YMMV.

laumars4• 13 years ago

That may have been chopped of by Opera then. Thanks for the correction though :)

tome2• 13 years ago

It didn't work at all on my (admittedly old) version of Firefox.

I suspect the author made the slideshow to display on his/her own system, and generously shared it publically as an afterthought.

laumars4• 13 years ago

Interesting point. You might be right as someone else did hint that this was part of a talk.

Sarien• 13 years ago

Doesn't work with my version of chrome (Version 26.0.1410.64 m) either.

Edit: Ooooh, found a button that actually does something. Apparently you have to use the scrollwheel, space or the arrow keys. Just clicking on things doesn't seem to work. Hooray for modern UX design.

ahoge2• 13 years ago

> modern-looking web pages

Well, herein lies the problem: It's not a web page, it's a slide deck.

But I do agree in general. The slides alone are often not very interesting or useful without the talk. Hopefully, it was recorded.

ams6110• 13 years ago

I assumed it was a deck but couldn't find a way to advance from the first slide.

davidw• 13 years ago

Good presentations do not put every single thing in the slides. They are there to support the speaker. The better the presentation, the more worthless it is to click through on a web page, and therefore I generally avoid them in favor of something I can actually read.

thiagoharry• 13 years ago

You are having a better experience than me. To me, what I see is a blank page... A pity, I was really interested in the content. :-(

rwmj• 13 years ago

* It came up completely blank when I opened it (presumably because of NoScript disabling Javascript, but I didn't check -- just closed it and left).

akama• 13 years ago

It also does not appear to be working on my nexus 7 as well. I find cases like this frustrating because I was very interested in the content, because I might start learning Haskell, but it is unlikely I will remember to check it later on my desktop.

lifeisstillgood2• 13 years ago

Dear pg

I am beginning to see what you mean by the easy-negative-up-vote effect.

I hit that presentation and saw the first two pages - Mark Hibberd, and REST API documentation with Haskell. You could slideshow the phone directory at half speed and I would still be doing exactly what I did - go look up more info.

On the other hand, the responsiveness of that deck did feel like a phone book at half speed. So no downvoting from me.

laumars4• 13 years ago

> I am beginning to see what you mean by the easy-negative-up-vote effect.

I'm someone who normally hates such posts as well. However the reason I did leave such a comment this time round was out of disappointment rather than wanting to be negative for the sake of "internet points". The topic was one that I had a particular personal interest in and the content looked like it should have been really juicy. But sadly the item failed to offer any substance and that was entirely down to the choice of layout.

However it now seems like this presentation was designed to accompany a talk and likely wasn't expected to be viewed on it's own and without explanation. In which case my post was unfair and I genuinely wouldn't have left those points had I known. In fact the only reason I haven't deleted my post is because I think that would now do more harm than good due orphaning the replies.

So I do actually agree with your disappointment about seeing such a post - even in spite of being the author of it.

sejje• 13 years ago

Why wouldn't the landing page say "use arrows to navigate?" I spent a good little while trying to figure out what I was missing.

kbenson• 13 years ago

Took me a second until I noticed that the scroll wheel seemed to take me to a different page. As soon as I determined it was a slideshow, I tried the arrow keys, as they work on every HTML slideshow I've seen in the last 5 years.

verroq• 13 years ago

Scroll wheel works.

stonemetal• 13 years ago

Does anyone have some insight to what is going on in the Z monad bit? It looks like an Either monad with a bunch of error states instead of just the one.

boothead• 13 years ago

Z isn't a monad from what I can see... ZZ is the monad. It's an IO action that returns a ZResult (which is either a value or an error state as you said) and has access to an environment for logging, a database connection and a configuration.

I'm not a Haskell expert though - so don't take that as truth until confirmed by someone else :-)

tel4• 13 years ago

There are two independent parts. The Z monad is nothing more than a ReaderT monad giving access to some config and environment information. Since it's a transformer they can execute actions in higher monads (like IO) pretty easily from Z, but it can also be used in pure code if you just need the configuration information.

Parallel to that is the ZResultT monad. It's built on the ZResult monad (it should be easy enough to write a Monad instance for it anyway) which looks like an Identity monad with some "Z-specific" error handling layered in. ZResultT just lets you have a transformer stack interleaved with the ZResults.

Then you combine those two stacks in ZZ which gives you both "Z-related" configuration and "Z-related" error states.

It looks like the whole thing is specialized to make it easy to build functions pure functions which only need a handful of the effects and then weave them together into the ZZ monad which is likely to be run at the top level, right below Yesod.

hcarvalhoalves• 13 years ago

The fact I read this and have absolutely no idea about what's going on makes me compelled to learn more about Haskell.

Is it really that alien (I know Scheme)?

chc• 13 years ago

It's pretty weird. It's a very strongly statically typed lazily evaluated pure functional language with a type system that is actually Turing-complete.

njs123452• 13 years ago

The type system isn't Turing-complete by default, you need to enable a fair number of extensions: http://www.haskell.org/haskellwiki/Type_SK

njs123452• 13 years ago

Haskell is definitely pretty far away from almost any other language (I can't think of any others which are lazy by default, for example).

Learn You a Haskell is probably the best resource if you want to take a look, although it seems to be down right now: http://learnyouahaskell.com/

gtani3• 13 years ago

Not alien, but not like ordering a sausage mcmuffin for breakfast. You'll recognize ideas taken from haskell (list comprehensions, especially) in other languages but the function definitions (multiple pattern matching clauses/"equations"), may be alien, unless you've looked at erlang, F#, ocaml, scala or some such.

You could look at these to get why

http://www.haskell.org/pipermail/haskell-cafe/2012-June/1015...

https://www.fpcomplete.com/page/case-studies

http://elaforge.blogspot.com/2011/07/what-haskell-doesnt-hav...

http://haskell.cs.yale.edu/?post_type=publication&p=112

http://www.jerf.org/iri/post/2908

(or a few dozen tutorials: http://haskell.org/haskellwiki/Learning_Haskell

-----------

here's my attempt at organizing the learning process (having lots of places to ask questions (SO, reddit, beginners list, G+) is great

http://www.reddit.com/r/haskell/comments/1b7axp/haskell_home...

GrandTheftR• 13 years ago

I have been programming Haskell for the last few month for a sideline project, and really enjoying the experience so far. This has been the second or third time I revisit Haskell as a programming language, and this time I do feel that I am "getting" it in Haskell and could be productive ;) and I do feel very safe to write in Haskell, usually it works if it compiles.

Here is one great post about starting in Haskell (via jerf@Reddit) http://www.reddit.com/r/haskell/comments/1e3cl0/haskell_intu...

it would be the exact way if I would summarize my experience over last few months..

but I do feel the lacking of good communicators in Haskell community who know how to talk to non-Haskeller.

lobster_johnson5• 13 years ago

That Reddit post is great. For me, in learning Haskell, the biggest challenge has been libraries. Many of them just seem badly designed, or unfinished.

My first project I wrote a simple scraping tool that scrapes a certain web site and organizes the information in a database behind a front end and API. Immediately I encountered several problems that would have been a cakewalk in, say, Ruby:

* The HTTP library (Network.HTTP) is not encoding-aware. It ignores Content-Type and returns a String with some undefined encoding. So if you grab a resource which is, say, ISO-8859-1, anything that works with UTF-8 will potentially blow up.

* (And if you do this yourself, there is no built-in way to parse MIME charsets into Haskell encoding names except by writing it yourself, I think.)

* The string libraries are confusing. There is String, but also Text. Both are supposed to be Unicode-aware, but many of the operations you want are in Text, requiring conversions back and forth.

* I could not, and still haven't, figured out how to convert a ByteString containing ISO-8859-1 to an UTF-8 string. Data.Encoding apparently exists for that, but I could not get it to work like I wanted. I ended up with this lovely mantra (which you wouldn't think had anything to do with ISO-8859-1, but it does) and left it like that just because it works:

    decodeLatin1 bytes = Data.Text.unpack $ Data.Text.pack $
      Data.ByteString.Char8.unpack bytes

* There are multiple libs for HTML/XML parsing, and no clear winner. HaXml was buggy and did not parse my HTML correctly. TagSoup and HandsomeSoup had weird APIs I did not like. I ended up with HXT, which uses Haskell arrow syntax extensively and is therefore incomprehensible, even when I understand what and how it does it. It feels a lot like people who write ambitious libs in Ruby with too much "magic". Haskell is a functional language and should be wonderful for parsing HTML/XML using XPath or CSS selectors, but it's a nightmare. HXT's parser is impure by default (!) and you have to google some tutorials to find out how to use its pure version, which the tutorial writers (in the same breath) dissuade you from using.

* For the frontend I ended up using Happstack Lite because it's small and about as easy to get started with as Sinatra. Unfortunately, the first templating system it recommends, Blaze, is crap. With Blaze you write the template inline, building an XML tree, similar to Ruby's "Builder" gem, but the syntax is ugly and unnecessarily complicated. The other recommended templating systems, HSP and Hamlet, are like ERB or PHP, so a serious step backwards from HAML. In the end, I could not find anything like HAML.

* I haven't delved very deeply into Happstack yet, but like many Haskell libs it seems to make simple things more complicated than they ought to be (eg., the routing system). One annoyance I remember is the fact that HTTP verbs aren't IO, which makes code less intuitive, since obviously in a modern web app, HTTP verbs are going to be doing IO (things like talking to databases), and so you have to use liftIO a lot.

* Perhaps the biggest issue for me was the general lack of high-quality documentation (Happstack itself has almost none). Sure, there are tons of searchable machine-generated docs, which is great, but that documentation lacks the bits that tie everything together. Since Haskell is mainly about applying functions to values of types, it is very decentralized; knowing what function to use is only part of the game. It's just as important to know how its return value plays with the rest of the function space, because so much of Haskell revolves around composing functions in idiomatic ways. This is very different from, say, Ruby, where you know a method returns an object, and an object only has the methods its class provides; this centralizes, clusters and constrains the information into easily digestible pieces.

A veteran haskeller would probably not struggle too much with these things. But as a novice, I expected the path to be slightly easier. The language was rarely a problem, the libraries definitely were.

Tyr42• 13 years ago

It's not that different from Scheme, really. If you are only using lists of one element type, then most scheme code translates fairly naturally to Haskell. Sure you use a `case _ of ...` instead of a `cond`, but Scheme is more similar to Haskell than it is to Python for example.

lobster_johnson5• 13 years ago

The slides are admittedly fairly obtuse, with very little context. Presumably there's an accompanying talk we are missing.

Haskell is really not alien. The stuff that usually comes up, such as discussions about monads, make the language seem weirder and more esoteric than it really is.

The base language, if you disregard the more advanced, complicated topics (for example, the mathematical basis for monads), is very straightforward, although some of the concepts (typeclasses, lazy evaluation, partial application/currying) can seem very novel and strange and daunting if you are not used to ML-based languages. I was not, and I had no problem grasping the basics, although I am still not fluent in the more complicated monad-based stuff.

The best intro, by far, is Learn You A Haskell For Great Good! [1]. It starts very gently, and is tailored to people who are more familiar with current imperative languages.

[1] http://learnyouahaskell.com/

salimmadjd• 13 years ago

Not only I have no clue about the slides, but I went to their site (apiengine) and I have no idea what their site does. Interestingly, both the slides and their site (apiengine) has clean and beautiful designs, however neither one of them are a bit informative.

DevMonkey• 13 years ago

He He... Their site is hard to read and understand and so is Haskell at first glance.

Haskell in Production: 12 Months Building ApiEngine

Related Stories