rabbitmq

:-) The founding team, yes.

RabbitMQ supports HTTP and has a clean easy API. Scales in lots of ways. Supports very high throughput on a single cheap AMI. Has awesome GUI and UI. Etc. YMMV.

"RabbitMQ could not enqueue/dequeue fast enough"? I think the STOMP plugin was still pre-production a year ago. You may want to try using RabbitMQ 3.x..

Sidebar --- anyone interested in running tests with RabbitMQ should check out these posts as a starting point.

http://www.rabbitmq.com/blog/2012/04/17/rabbitmq-performance...

http://www.rabbitmq.com/blog/2012/04/25/rabbitmq-performance...

Please note that:

1. the results are from last year, and so not 100% representative of RabbitMQ 3.x which was released after these blogs were posted

2. what actually matters with message queue performance is stability and scalability over time .. testing this is extremely hard without making over-specific assumptions

Thanks for this lengthy reply. I'll try to keep mine short and that means skipping over a bunch of stuff.

First, re "whilst exchanges and bindings survive the loss of individual nodes, queues and their messages do not" -- this needs to be clarified. When queues are replicated, their messages do survive, in order, in the replica. The queue itself may die but the client can get back into context by finding another node in the group. This is observationally similar to DHTs.

Second, "a single logical broker" and "sounds an awful lot like a single element in the system that is responsible for message traffic".... NOOOOOO the point of 'logical' is the opposite of what you say!

Finally the BIG difference between DHTs and something like Rabbit is that in a DHT each datum is replicated around the ring N times, non-uniformly. Whereas in a system like Rabbit each datum is replicated N times in a uniform manner, such that ordered pairs of messages A and B will exist in every queue which has either A or B.

Paddy, thanks for asking me if I am "familiar with the basics" of a DHT. Just to be clear, I don't think DHTs are the same as messaging systems. But, they touch on a set of related issues.

My assertion was that "Just because software runs on multiple machines it does not mean that there is 'no SPoF'". You seem to have read this as saying "in all cases where there are multiple machines, there is always a SPoF". In any case, I was trying to make a more general point, which is that (A) lots of cases which seem to have no SPoF, in fact do have SPoFs, (B) some cases get accused of having a SPoF, but doing so is a mistake, and Rabbit is in this category because you can set up a number of multi-machine scenarios with the required redundancy, and (C) in many cases having a SPoF is not a problem anyway, and may even be a good thing.

You said "From what I can see in the docs, RabbitMQ is a client/server relationship. Meaning there is a server. Meaning a single point of failure and a bottleneck. I hate those..". With all due respect this represents a misunderstanding of Rabbit. Moreover you assert that 'there is a server' means that there is a 'single point of failure'. Because the conclusion does not follow from the premise, I assumed you must have meant something else.

Perhaps you could explain why you think your system has fewer 'bottlenecks' than RabbitMQ or other messaging systems. I don't think that it does but would love to be enlightened.

From your more recent comment, you seem to be saying that using a DHT is a good thing. Yes, I agree, sometimes this is the case. Are you familiar with how RabbitMQ does any of the following: clustering, HA, federation? You say that "Am I saying my solution is closer to what I want than RabbitMQ is? Yes. Yes I am.". I would love to understand this.

This does not make any sense. All software runs on machines. Each machine is potentially a 'single point of failure'. This sometimes is a problem, and sometimes is not.

Just because software runs on multiple machines it does not mean that there is 'no SPoF', for example in a distributed messaging system you need to replicate state, unless you don't mind losing it when you have a crash.

If you don't mind losing state, then it is very easy to run RabbitMQ on more than one server.

If you do mind losing state, then RabbitMQ provides several ways to use multiple servers in a decentralised and redundant manner. See eg http://www.rabbitmq.com/distributed.html

It is very easy to write a system with multiple components. The difficulty is preserving all the additional capability (eg uptime) while maintaining a coherent system. This role can be played by a "broker" which does NOT imply a single machine or "SPoF" - it can be a distributed service. See eg http://www.rabbitmq.com/blog/2010/09/22/broker-vs-brokerless...

Nice post. One way to delegate the Java client redirect is to use Spring AMQP for automatic connection failover. This has been available since 1.1.0 GA in May 2012.

For other languages, and in general, we are looking into ways to make the entire reconnect more transparent for cases both with and without intermediaries. Solving this at the protocol level and in the general case is non-trivial - you have to make some assumptions. Do please email the rabbitmq-discuss list if you want to talk about any of this.

shameless rabbitmq mqtt plug here :-) http://www.rabbitmq.com/blog/2012/09/12/mqtt-adapter/

So, why not just deliver that topology using off the shelf components? Rabbit, Kafka, ZeroMQ, ... the toolbox is deep. This is not meant as a criticism, I am simply trying to understand what I am missing when I look at your design :-)

Master-slave etc are HA techniques for replicating state eg queues (or "channels" as nsq calls them?). "Decentralized and distributed" has nothing to do with state, master/slave or any of that; it's a topological property of your system. You can certainly build a decentralized and distributed system using a set of brokers, regardless of whether they support HA.

yay! congratulations Tim

Our thoughts on sockets -- SockJS

http://www.rabbitmq.com/blog/2011/09/13/sockjs-websocket-emu...

I don't know what WebLogic can do for flow control exactly. Please email rabbitmq-discuss to get a definitive answer and discussion on what you can do with Rabbit, which supports various mechanisms.

If you want to know what motivated the creation of RabbitMQ please read chapter 1 of Jason and Alvaro's book, "RabbitMQ in Action" - http://www.manning.com/videla/Videla_MEAPCH01.pdf

RabbitMQ's main motivation has been to make it easier to join systems together, scale your applications and manage complex environments. That is what messaging is for. Back in 2006, we felt there was a need for a good, stable and scalable open source licensed product that could compete with the incumbents.

Notice that I did not mention performance. RabbitMQ has good performance and it is used quite a lot in finance, but the majority of users are what you might categorise as "anyone using MySQL or Postgres".

Re activemq vs zeromq, I recommend reading "broker vs brokerless" on our blog.

Hope this helps.

If you have any questions or problems involving RabbitMQ please email us (info@rabbitmq.com) or post to the mailing list.

"one interesting concept about queues is contention and this is where RabbitMQ and others are behind"

Can you explain what this means? What kind of contention are you talking about. You say "RabbitMQ and others" - which others? Who implements this feature and what does it look like?

"you can't block a produced in RabbitMQ based on the number of messages on the queue"

Yes you can.

Well - it depends on your use case. RabbitMQ enables you to determine queue length, and supports flow control.

But, perhaps you had something else in mind?

"a feature of the open source Erlang VM that's improved in the pay-for-support version"

Erlang was created by a telco in the 1990s and since then has been battered in production by serious users. It's real software. The version that Ericsson provide as open source is not crippleware as you seem to imply.

"RabbitMQ pays the performance cost of bouncing everything off the disk"

No it doesn't.

RabbitMQ only uses the disk if tell it to do so, eg if you require messages to be persisted when they cannot be delivered immediately.

"memory usage can grow and grow"

If you stuff data into a messaging server without draining it on the consumer side, then memory usage will grow. This will also happen if you write your messaging system in C++.

There are two solutions to this problem: - flow control, where you tell producers to back off - paging to disk, where you flush data from memory when it is on the disk

Neither of these is trivial to implement which is why there is a big gap between toy messaging systems and serious products.

As others point out on this page, RabbitMQ has support for both these features. In particular a lot of memory management capability has been added since 2.0.

I hope this helps.

Thank-you.

The docs for HA are here: http://www.rabbitmq.com/pacemaker.html

This also works with Veritas if you use that instead of DRDB.

Please note that we are currently QAing a new HA model which is active/active. Watch this space!

So For The Record.

Reddit are atill using RabbitMQ and as far as I can tell they are happy with it.

If anyone has any doubts or questions about this, please email us. info@rabbitmq.com

We have always accepted server patches via the mailing list: https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq...

But it's true that the community has been much more involved in clients like Pika, for example.

And yes we are planning to open up the bug tracker.

A piece of advice for anyone doing an open source project - start with an open tracker, because opening up a previously closed tracker is a royal pain in the butt.

Tim, what do you think is cool about BQL that might be needed in the future?

Rabbit.js implements the ZeroMQ patterns in node.js.

Some other things that might help understanding here:

http://www.rabbitmq.com/blog/2010/09/22/broker-vs-brokerless... http://www.rabbitmq.com/blog/2010/10/18/rabbitmq0mq-bridge/

The second article is written by one of the main authors of ZeroMQ, and includes info that may indicate that, as he says: "RabbitMQ and 0MQ are focusing on different aspects of messaging".

On being "faster", do note that almost all modern messaging software is more fast enough for web apps which rarely require more than a few 1,000 messages per second.

I'm sorry you don't like things that are written in Erlang. But there are some quite good reasons for using it, e.g. http://jlouisramblings.blogspot.com/2010/11/on-erlang-state-...

More details here: http://lists.rabbitmq.com/pipermail/rabbitmq-announce/2010-A...

The new persister is going to help a LOT of people. Check it out!

OK, here's my reply:

> Well, I'll try to elaborate on my rant. Most of it from memory (been a year ago since I last touched AMQP). Back then when I looked at AMQP (and used it for one project):

AMQP is quite new - think about the early days of HTTP and XMPP. Some of the things you list have improved in the last year - but we have more work to do.

> * It was very hard to find reasonable documentation. The most accessible I found was the (excellent) http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/ blog post. This was what eventually got us off the ground.

It's a very good post. I think the documentation at least as far as RabbitMQ goes has improved, see the link store here: http://www.rabbitmq.com/how.html and the del.icio.us repo that it links to, which is organised by tag.

We'd very much appreciate assistance in improving all this!

> Imho a testament to the shortcomings of the "official" documentation (where is it?).

The working group has not done a good job at putting out 'official' documents about AMQP, other than the specs, which do provide documentation. For some reason people choose to not read the specs, but I do recommend the 0-9-1 spec which is about 40 pages long (not too bad considering).

There will be more of a push to do 'official' AMQP docs this year.

> * RabbitMQ only implemented AMQP 0.8. I cannot remember the exact differences between 0.9 and 0.8 (and heck, good luck trying to find a document explaining them even today).

We do actually have such a doc but have not released it since almost everyone uses 0-8. The main point is that 0-8 and 0-9 are almost identical.

> For us it boiled down to some clients simply not working and it took a while to figure out what the problem even was.

This has all got better, there are hundreds of projects now that work with RabbitMQ: http://delicious.com/alexisrichardson/rabbitmq+usecase+clien...

> * Client interoperability was generally spotty, even in clients supposed to be compatible. It seems the idea of that whole spec-file stuff was to make the protocol pluggable and flexible. However the only thing it seems to have achieved is to make client development hell. The client we eventually went with (py-amqplib) doesn't even use the spec-file because "parsing takes too long".

Most people have a good experience with most clients. I am sorry you did not - did you try asking for help on the mailing list?

> * Moreover the sheer existence of http://www.rabbitmq.com/interoperability.html is a testament to a flawed protocol.

Er.. no. It's testament to our desire to be clear about semantics.

You mention STOMP - I assume you are aware that STOMP's semantics are underspecified so that interoperability is not possible without vendor agreement?

> If I can't rely on a 0.8 client to talk to a 0.8 server properly, then something is fundamentally wrong.

How long did it take for HTTP browsers to work with all web servers? Years and years - and most people only use a subset of HTTP.

> * There's a huge pile of semantic baggage (Exchanges, Bindings, Queues, Channels) that left us with a giant "WTF" the more we used it. Yes, it can be made work. But hell, talk about overengineering. Much simpler constructions could have been used to achieve the same goals.

There are a lot of messaging protocols, and none of them have managed to be complete, interoperable, open and simple. Please do help us to make RabbitMQ better, whether for AMQP or one of the other protocols it supports.

> * As I just learned during my googling for this post, apparently some of the problems have even been acknowledged by the officials and are supposed to be simplified in AMQP 1.0 (http://www.amqp.org/confluence/display/AMQP/Background+to+AM...).

Those comments refer to 0-10 which is a lot more complex.

> But I'm not expecting much, to be honest.

That's encouraging :-(

Why not? You can help with this stuff you know.

> So, that's just a few points that popped into my head immediately, there's more if I'd dig deeper. It's just a fundamentally bad protocol if I have ever seen one,

Thanks for taking the time to provide feedback. I'm not sure what to conclude from your specific comments. The interop stuff has been cleaned up in 0-9-1. The fact that people get confused by Exchanges vs Queues, and the programmable nature of AMQP, is being addressed in 1-0.

In any case, as far as RabbitMQ goes, it's a multiprotocol broker and so far we have learnt two things about using AMQP - (1) that it's a good core protocol for messaging, around which other gateways are easily provided; (2) yes the API could be simpler - we are working on that.

> and I'm honestly surprised to see people defend it even after the creators themselves (iMatix) acknowledge it to be a dead-end.

I think you are reading a lot into iMatix's public statements that may not be borne out by the facts. You may find this odd, for example: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2010-Ma...

> To put it from an different angle: When you think about what AMQP really does then how long would it take you to whip up a near-trivial but feature-equivalent protocol from scratch? I could think of many ways to skin that cat in a better way, with a simple, hackable, plain-text protocol.

I agree that plain text would be good, but it does not cut it for some cases. I'm a fan of plain text nonetheless. But if you think this is so easy, please get involved and help!

> You know, one of those protocols where I, as a developer, can even sniff and see what is going on at any time. Those are possible, as demonstrated by redis, memcached, STOMP, beanstalkd and many others.

I agree that plain text has some advantages, and am a huge fan of Redis, but don't see any of those as delivering a messaging protocol. This stuff is non-trivial unfortunately; as I say we'd welcome your input.

> I acknowledge that some complexity is unavoidable for advanced use-cases. Still, a good protocol layers the complexity in a way so that simple stuff remains simple and complex stuff becomes manageable. To me AMQP is a complete failure in both regards; you are confronted with the full complexity even for the most trivial use-case. And it seems for complex use-cases it adds a hefty layer of complexity of its own, rather than helping to wrap up.

I definitely agree that most users want a simple entry point.

Cheers,

alexis

Hey thanks for that. Good response. I'm travelling so a full response is on its way soon..ish.

It turns out that RabbitMQ's throughput of tens of thousands of transient messages per second per CPU is enough for a lot of cases. Similar story with latency.

ZeroMQ does a lot of neat stuff to make it go faster still, e.g taking out the server, and removing all routing info from the messages, but ends up being much more like an alternative to sockets as a result, which makes it less useful for general work.

In summary - RabbitMQ and ZeroMQ are complementary, rather than really being alternatives to one another.

alexis

RabbitMQ

Since ZeroMQ has no broker but is peer to peer, that would be somewhat unnatural.

Cheers

alexis

Hi, Alexis here from RabbitMQ.

Wtf? You may not like what we do with AMQP but I can assure that lots of people are very happy using it in its RabbitMQ incarnation. Which version of AMQP have you "had the misfortune to work with" and how have you found it "hopelessly overengineered"? Is there something else you would recommend - and if so why?

We do actually like making our product better and appreciate constructive criticism.

Please do explain.

Also please note that RabbitMQ supports multiple protocols and one reason we like AMQP 0-8 and 0-9-1 is that they are flexible enough to make this easy (see e.g. RabbitHub on Github).

Cheers

alexis

Thanks for your comment.

I could not find the 139 page document to which you refer.

There are two 0-91 docs, one is the spec definition for users, which as I said short and is mostly edge cases you can ignore. There is a second doc for implementers which defines the classes and methods in more detail. This is 63 pages long. Note also that for the purposes of client codegen, the BSD licensed XML file in 0-91 is only a few pages long - because the surface of the spec is surprisingly small.

As a comparison, the definition of core XMPP (a Jabber server) in http://www.ietf.org/rfc/rfc3920.txt is 90 pages. BTW the core spec is just for IM not pubsub.

In the case of both AMQP and XMPP the length comes from the requirement to interoperate between implementations.

You make a good point about STOMP above. We love STOMP too. There is as you say a place for it - for lots of protocols. We have however found with STOMP that because many behaviours are completely unspecified, that it costs us a lot more to support (find and fix bugs, maintain stable behaviour under different conditions, etc). It is less likely that the same application talking to two STOMP brokers will behave the same way with both brokers - deterministically and predictably. Maybe this is a good thing - there is more scope for competing implementations? I don't think it's ideal. And let's not talk about JMS in this regard.

I would not discourage people from writing their own brokers. You are among many who have done this and people will go on doing it. But although you may understand every line - what happens when someone else has to take over managing your code? What if the requirements change - or the scope of use grows? This is where products add value.

A lot of our customers have extremely simple requirements like "don't ever lose my messages" or "broadcast to twenty different types of subscriber". So, I don't think it's fair to make generalisations about "complex requirements".

I completely agree with you about RPC.

Cheers,

alexis

Hi, alexis here from RabbitMQ.

Yes, we implement AMQP. We also provide support for other useful things like STOMP and HTTP Pubsubhubbub. We implement these other protocols as well as AMQP because some times people don't need to use the full and awesome power of the AMQP model.

AMQP is initially hard to grok. I think the main reason for this is that AMQP combines three things: Queues, Pubsub, and Messaging. These are Not The Same. Queues manage data in flight as state, Pubsub routes data to consumers, and Messaging frames it.

So yes, as someone pointed out above, it would be nice to use some but not all of this from time to time. We are working on ways to make that super easy - please get in touch if you can help.

Another thing that people find hard is figuring out when to use message hub technology, and when to use a database as a hub. Using a database to queue and manage subscriptions to data streams is generally Not A Good Idea. Here’s a presentation I did which attempts to articulate some of the issues with this: http://www.rabbitmq.com/resources/RabbitMQ_Oxford_Geek_Night...

So, for someone using AMQP or any other Pubsub tech for the first time, there can be a 'huh, where do I start' element. But as some commenters point out, if you look at the client libraries it may be easier to get started. We've actually lost count of how many clients there are, so take your pick.

List of clients: http://delicious.com/alexisrichardson/rabbitmq+client

Getting started: http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/ (Python centric) and http://www.infoq.com/articles/AMQP-RabbitMQ (Ruby centric)

To the commenter who said the AMQP spec is 300 pages long. You may have a better time if you look at AMQP 0-91 which is much shorter than that at 40 pages mostly covering edge cases that you can ignore. The nub of AMQP can be communicated in under a page.

BERT and BERT-RPC look cool. But - re the comments above - I would not see BERT-RPC as an ‘alternative’ to AMQP though. The GitHub blog post talks about PB and Thrift and JSON-RPC, all of which have been integrated with RabbitMQ. If you want to do RPC, there is no ‘one true system’ yet. Typically we have found that different people get value from different RPC metaphors in different languages. Maybe BERT-RPC will get more traction than the others - it definitely looks interesting.

I hope this is all useful or at least of passing interest. Here are some more links that may be worth a glance:

General background: http://www.rabbitmq.com/how.html

AMQP and XMPP: http://www.igvita.com/2009/10/08/advanced-messaging-routing-...

Feel free to contact us directly at info at rabbitmq dot com.

Cheers,

alexis