Ask HN: What have you built with Erlang?

https://news.ycombinator.com/item?id=1683528

by SnootyMonkey • 16 years ago

72 43 16 years ago

news.ycombinator.com

A recent Erlang article in Communications of the ACM portrays Erlang as a specialized language for messaging/chat (Facebook chat, XMPP, etc.)and distributed storage (CouchDB, Riak, etc.)

I think this is probably right, but I'm wondering what else you've built in Erlang. Was it a good fit? Why or why not?

KevinMS2• 16 years ago

I built a very specialized mail server (MTA).

It was a tedious, frustrating experience, even though you'd think it would be the perfect language.

It ended up being too slow (probably because of all the string processing)

In my entire programming career, I've never before had to join a mailing list and ask questions to get things done, that should tell you something.

tptacek• 16 years ago

I'd be interested in hearing more about the frustrations.

keefe• 16 years ago

yes indeed I'm increasingly using couchdb, planning on picking up erlang and it would be useful to avoid the mailing list if you can pass info along...

ovidiu• 16 years ago

I have built the backend (game logic and comet) for http://www.iluminatus.ro, a trivia-like multiplayer game for the Romanian market. We're going to release an english version soon.

It was a great fit. It lacks Unicode processing capabilities by default, but there are a few open source libraries for that. The only thing I dislike about Erlang is the poor string manipulation and lack of clean structures (Erlang's records are an ugly hack IMO).

ak13942• 16 years ago

I built a Twitter client, with j2me on handsets and Erlang server at the back to handle all communications with twitter and few other services for image hosting, etc. It was my second project in Erlang and was very rewarding experience. Part I especially liked, is that I could deploy updates to the server while it was running, with no desruption to clients at all. http://pavo.me/

francescoc• 16 years ago

Banking systems, both retail and commercial. Logistics. Embedded. Clusterware. Telecoms. Robotics, AI. When it comes to messaging, don't forget RabbitMQ and all the SMS systems.

Francesco

sgt• 16 years ago

For those of you who don't now, Francesco Cesarini is the founder and CTO of Erlang Training and Consulting, and has been a major proponent of Erlang for years, also having co-written the book "Erlang Programming" (http://oreilly.com/catalog/9780596518189 - it's a great book, I highly recommend it myself)

BrandonSmith• 16 years ago

Phonebooth, a hosted telephone system http://www.phonebooth.com

Great fit. OTP process restart characteristics. Per-tenant process separation. Process abstraction, in general, rather than threads. Finite state machine behaviors. Live code upgrades. Mnesia database.

Only real downside was it took the rest of the team a bit of time to become effective in Erlang... about 4 to 6 months.

iamaleksey• 16 years ago

SMS/MMS gateway (aggregator) with SMPP, CIMD and MM7 support and AMQP interface (RabbitMQ as a broker). Also building a simple SMSC right now.

It's a perfect fit.

bnicholso• 16 years ago

We built a system to connect teams via texting and conferencing. http://www.connectteams.com. Erlang provides the web front end, the backend telecommuncations, and the reporting interfaces. All perfect fits.

jayair• 16 years ago

We have an erlang cluster that runs all our services @ http://thecadmus.com

High number of processes distributed across multiple nodes with supervisor trees taking care of them all has worked well for us.

hassy• 16 years ago

I worked on a spreadsheet compiler at hypernumbers.com and some backend stuff for vidiowiki.com (video transcoding, file uploading/streaming and some other smaller stuff).

Erlang excels at distribution. People tend to forget its raison d'être is fault-tolerance (not concurrency), so when you set off working on an Erlang application you presume that it will run on 2+ machines. That's why it's great for things like Facebook Chat and Riak. In addition to those, it's also good for writing control and supervision layers, where workers are written in Python, Java, C etc.

While I'm at it, I'd like to plug http://veldstra.org/whyerlang/ It's in a real need of an update, and I'd love to hear any feedback.

ams61102• 16 years ago

Not mine, but there is a CMS built in Erlang: http://zotonic.com

samuelth• 16 years ago

Not me but deserves attention: http://www.wings3d.com/

"Wings 3D is a subdivision modeller with an user interface that is easy to use for both beginners and advanced users"

gordonguthrie• 16 years ago

http://hypernumbers.com - an online spreadsheet which is a native web app - lurching towards a proper launch after a lot of development.

Erlang is a great fit - works well, allows you to build a native stack without any cross-language impedance. Your application is cluster ready from the get-go and allows you to focus on the functionality rather than any of the utilities required to make the damn thing stay up.

ngerakines• 16 years ago

A Facebook application that had nearly 2 million users.

A game data aggregation platform and feed system for EA.

A lot of open source Erlang stuff (http://github.com/ngerakines)

KirinDave• 16 years ago

I built a fault-tolerant system for binding lots of disparate and network-unaware services via a monitored event-driven communication protocol to a network-fronted cloud architecture. It's called Katamari, and it's the version of Fuzed (this same concept just for Rails instances) that I modified to run as the majordomo of Powerset's realtime query analysis layer. It let us take un-threaded and network-unaware software that talks over local file descriptors and turn them into a cloud service.

Katamari also has some fairly good fault tolerance characteristics. It aggressively tries to maintain a cloud and it has no real single points of failure. It recognizes hung or crashed instances of individual workers and restarts them. It also supports arbitrary versioned resources existing in parallel, which is great when you want to do A/B testing.

It was also designed so that it was usable in both a development and production environment. While currently little slow for most production uses (~2ms overhead for most queries, with a special case of ~12ms), it's plenty fast and runs in less than 10mb of memory on most linux systems, and can transparently go from a 1-box to a N-box solution.

Microsoft open-sourced it for me (after 2 years of bureaucratic nonsense), so I am just pulling out the useless and crufty parts before I put it up on GitHub. I've given 2 talks about the technology, one for Fuzed at Railsconf and one on Katamari at the last Erlang Factory. I hope to have a modernized version maintained this year, and start moving the tech forward.

Katamari is about 10k lines of code; with about 2/3 of that being Erlang and the remaining 1/3 being Ruby code for the glue (Ruby was Powerset's primary glue and binding language, so it is the only supported binding). Katamari aggressively leverages the characteristics of Erlang to accomplish this feature set; its safe to say all the fault tolerance wouldn't have been doable without Erlang's implementation providing the basics for it.

zaphar• 16 years ago

Started etap: http://github.com/ngerakines/etap which ngerakines took over.

Iterate: http://github.com/zaphar/iterate an agile user story tracking app. Which is mostly useable but still in development. Used nitrogen which I found to be a really good web framework for dynamic applications.

martinjlogan• 16 years ago

I built one of the fist comprehensive call detail record collection systems for a SIP voip network. I also built a least cost routing system for same.

A options feed aggregation and charting backend.

A package management system.

many many other small apps.

Cheers, Martin Logan erlangcamp.com

yariv• 16 years ago

ErlyWeb, an open source web framework: http://github.com/yariv/erlyweb/.

Twoorl, an open source Twitter clone: http://github.com/yariv/twoorl/.

Vimagi, a graffiti-like facebook app: http://apps.facebook.com/vimagi-paint.

Also, parts of Facebook chat (I work at Facebook).

ak13942• 16 years ago

Also, thanks for your work on mysql driver, which I happen to use for PavoMe.

adelevie• 16 years ago

HN is such a nice community :)

bosky101• 16 years ago

- a key-value stored modeled caching system that can run functions during set/get, have expiry, counters, counters that wait for N as a buffer before running transactions, and machine learning - all as primitives. the tail-recursive processes acting as mini-servers holding state and the best thing is that you have a mini-automatic scaling if you combine this well with expiring of processes. increase load when there's a need. they die when there's no more need/lesser traffic.

- a distributer crawler that works with python. from a typical rdbms approach that brought down creating inverted index's from more than an hour to few seconds. the euroka moment was when we stopped porting/writing for another language, sequentially, for-loop style approach to a more - processes and message passing. big 'why didnt i do this moment' a few years ago!

- a website , dashboard for the ad/content network using yaws on the LYME stack

- and btw - all that rumour about unicode is just bs. actually it was a blessing for us that content aka strings are just integers which when converted to binaries not only takes less space, pattern matches like a dream but all of a sudden your sytem can handle any language because to it - it's all just integers, english or swahili. that was a huge bonus even when getting investors, new untapped market opp's.

- an ad network, ( content network i guess without saying)

- a fire-and-froget thumbnail/screenshot system with queues & talking to imagemagick . i love writing and fire-and-forget tools, and with erlang, its so darn easy to write proceses's that have a life of their own. sure at some point you need to compromise on erlang's unique ability to handle RAM vs giving that up to research on how to handle large mnesia tables ( mnesia = inbuild db that has a limit on size of tables) . but it's worth the tradeoff.

case in point - our idea of scaling was...'reducing nodes' ie increasing efficiency

-2008 a million hovers in the year, 4 nodes

-2009 a million hovers every month, 3 nodes

-2010 a million hovers every week, 2 nodes i could bring it down to 1 node for doing the tasks as well, on just one 1 8gb RAM node.

the idea of reducing your costs, means that much lesser to profitability. ( we're now profitable, pretty much on auto-pilot on the backend, the 3-4 member team concentrating on increasing toplines,sales now )

in addition... - a wrapper to iui for nitrogen framework for building iphone webapps easily - wrappers to tokyocabinet ive contributed to called medici - remember writing wrappers to other things like rrd, aws utils, etc

all of the above have been used at hover.in, more at http://slideshare.net/bosky101

there's nothing i can think of that i wont/cant do in erlang anymore.

~B @hoverin

lzw• 16 years ago

Let me guess, all this over engineering nonsense set your startup back 3 years.

Groxx• 16 years ago

Not mine, and though I can't find a page on their site that states it, Vendetta Online[1] uses Erlang heavily for their server code. I went to a conference they spoke at a while back, they've said it works wonderfully for them and lets them handle large amounts of players easily.

They are also working to release a game engine (NAOS[2]) built from their server code, which also uses Erlang pretty heavily

[1] http://www.vendetta-online.com/ [2]: http://www.guildsoftware.com/products.html

EvanMiller• 16 years ago

ErlyDTL, a Django Template Language compiler: http://github.com/evanmiller/erlydtl

Chicago Boss, an MVC web framework in progress: http://github.com/evanmiller/ChicagoBoss

This doesn't get mentioned enough, but Erlang has excellent compiler tools, so if you're willing to roll up your sleeves, you can usually hack the language to fit your problem. For instance, Chicago Boss ships with a nice but completely non-standard database querying syntax, described here:

http://www.chicagoboss.org/api-db.html

You will weep for joy when you rewrite a gnarly SQL query with a few well-chosen Unicode set operators.

The compiler tools also make it easy to generate Erlang AST and compile it to BEAM (Erlang bytecode). That's how ErlyDTL works. I haven't done performance benchmarks myself, but the folks at Zotonic report that their Django templates and controller code come out 10X faster than comparable PHP code.

I think a couple of things prevent Erlang from being a do-all language:

1. Lack of libraries. The situation has been steadily improving, but Erlang's VM and processing model makes it difficult to draw upon the "C tradition" in the same way that Perl, Ruby, and the others can take a popular C library and write a thin wrapper around it.

2. Inferior OS integration, at least with UNIX. For instance, custom signal-handling facilities are non-existent, and you have to jump through strange hoops to do simple stuff like close only the write end of a read-write pipe, or get the exit code of an exec'd process. Weak sauce for anyone who has spent a lot of time with Perl, Ruby, or C.

fmstephe• 16 years ago

I wrote a distributed sat-solver in Erlang (good old fashioned search problem). It was my intro project to get to know the language. Has been great fun and now Erlang is in charge of computers talking to each other while Java takes care of the heavy lifting. Also a similar problem solved with Erlang (nothing to do with me), and featured here on hacker news...

http://blog.ezyang.com/2010/08/tour-of-preach-distributed-er...

Great stuff. Erlang may make you smile (again).

rubyrescue• 16 years ago

- we wrote the backend of http://bidit.eu in Erlang (front end in Rails) - we're writing the backend for a TV co-viewing chat application for the iPad in Erlang right now - we're rolling out a new Nitrogen-powered website for Inaka Networks this weekend

metabrew• 16 years ago

I'm currently building a web-based IRC client with bouncer-like functionality - ie, stays online when you close the browser, remembers backlog etc.

Built github.com/RJ/playdar-core in erlang, after initially writing it in C++/boost

sofuture• 16 years ago

Simple stuff, but:

- An IRC bot -- great fit for the language

- A URL shortener service (three times) -- first on Yaws, then on Nitrogen, then on ChicagoBoss (Hi EvanMiller!)

Erlang is the most fun I've ever had writing code. While a lot of that is the functional language aspect, I find it an extremely intuitive language at least at a low level (once you get past the 'there are 4 ways to end a line?!?!' stage). I'm still getting used to thinking through bigger problems in an Erlang way, but thats more a function of my brain than the language.

I think Erlang gets a lot of unfair knocks given it's pigeonholed reputation for 'concurrency and fault tolerance'. Certainly it has it's limitations (libraries), but generally, I think its a pretty well-rounded general purpose language.

vito• 16 years ago

Super late (a misjudged apache-bench got me banned for a day or something), but: http://darcsden.com/alex/aliter

It's a reverse engineering of a popular MMORPG called Ragnarok Online (or RO). Last time I was working on it, you could log in with the official client, walk around, teleport, chat, etc. (The walking had to be done in C, though; one pathfind per walk request is not something I'd want to leave to Erlang.)

mononcqc• 16 years ago

Mostly toy projects, many of them still in development.

A little homemade web framework. A blog compiler to generate a static site. A one-on-one chat client to be integrated with other already existing websites, a browser-based version of the Apples to Apples game, a brainfuck interpreter, maze solver, etc.

nivertech• 16 years ago

I did several small Erlang projects and now working on two relatively large.

Erlang is a good fit for:

* Irregular concurrency: Task-level, Fine-grained parallelism

* Network servers

* Distributed systems

* Middleware: Parallel databases, Message Queue servers

* Soft Realtime / Embedded applications

* Monitoring, control and testing tools

Not so good fit for:

* Concurrency more appropriate to synchronized parallel execution: Data Parallelism

* Floating-point intensive code (HPC)

* Text Processing / Unicode (Unicode support now much better)

* Traditional GUI (some improvement with wx now)

* Hard Realtime applications

* Extensive interop with other languages/VMs (some improvement here with NIFs and Erjang - Erlang on JVM)

* No Windows 64-bit VM

I using Erlang for the things listed under "not so fit", because it's advantages out-weight some incomvinience.

brianjesse• 16 years ago

I built a real-time news widget that streams (hyper)local news and public records (marriage licenses, liquor permits, etc). Media companies partner with us http://nozzlmedia.com we've doubled the time visitors spend on their Web pages.

Erlang was a good fit for us, mostly because of the tools available around it, such as Nitrogen http://nitrogenproject.com a powerful JavaScript/Erlang framework. Erlang apps seem robust and lean.

joelreymont• 16 years ago

OpenPoker

gleber• 16 years ago

Erlang is perfect for any messaging systems. And on other hand Erlang works well for GIS:

Vector Map Server - a BSc thesis project. Vector geospatial data streaming server, very similar thing to server's streaming vector data to Google Earth or Map24. Also works as WFS server. Has to handle a lot of data, but since data is mostly immutable it works well.

Backend service for not yet released liveblogging site. Done in collaboration with few other developers. Massive pubsub system. Done tests with 1M concurrent users on 200 small-node EC2 cluster with end-to-end latency mostly < 1.5s.

Texspace - spatial search server based on Microsoft Research paper. Fuzzy spatial search capable of working with completely unstructured queries. Had pretty big issues with processing, indexing and analyzing big amounts of data due to slowness of all DB solutions available at that time. Writing geometry algorithms in Erlang is not trivial due to immutability, since all of them are described in imperative languages.

IM subsystem for microblogging site (which is pretty big in Asia) - implemented 5 proprietary protocols. Erlang is perfect for such tasks! The only issue was lack of documentation for proprietary protocols ;)

Map routing server - implemented optimized A* search with route graph generalization. Writing fast A* is tricky in immutable world. Books like Okasaki's "Pure Functional Data Structures" FTW!

WMS Tiling / TMS service with caching. Simple and easy task for Erlang. Mnesia + file-level cache.

Overseer - Nitrogen + CouchDB app for tracking employees time by doing screenshots via VNC and marking them as "work" / "not work". A breeze :)

Erlang FTW!

Ask HN: What have you built with Erlang?

Related Stories