RSMQ: A lightweight Redis based message queue for Node.js

http://smrchy.github.io/rsmq/

by Exinferis • 11 years ago

36 18 11 years ago

smrchy.github.io

vkjv• 11 years ago

First off, this looks great! It has a simple interface and could compliment some simple projects very well.

However, this quote, "If you run a Redis server and currently use Amazon SQS or a similar message queue you might as well use this fast little replacement" is greatly over simplifying things.

Amazon SQS gives you acknowledgement guarantees that are not possible with Redis. Don't get me wrong, I love Redis, but you can't reliably trust that writes happen.

https://aphyr.com/posts/283-call-me-maybe-redis

rubiquity• 11 years ago

Guaranteed delivery of a message to exactly one recipient within a messages visibility timeout.

I'll just leave this here for you: http://bravenewgeek.com/you-cannot-have-exactly-once-deliver...

Welcome to message queues. Where the network is both your best friend and enemy.

Smrchy• 11 years ago

It's not exactly "once". It's exactly "one" within a defined period of time. So in that articles context it would be the "at-least-once" variant.

If the recipient fails to process the message for some reason it will reappear after the visibility timeout.

zwily• 11 years ago

I don't think you can provide exactly once even with that constraint. What if the defined period of time was 100 years?

kitd2• 11 years ago

MQTT uses a 2PC-like mechanism for exactly-once delivery [1]. The sender sends and the receiver acks as usual. But the message is only delivered once the sender then sends a release message, which the receiver acks. End result: sender can assume receiver distributed the message only once.

[1] http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-o...

kasey_junk3• 11 years ago

That's not exactly once. There are several failure modes that can cause duplication. You can detect duplication but not prevent it.

hoggle2• 11 years ago

Did anybody here use the corresponding worker module too? How does it compare with kue (in production)?

Exinferis2• 11 years ago

We're have been using this in production for a few months now. We never tried kue but used the rsmq-worker module extensively.

Currently we're processing about 30 million messages per day on a single redis server with two worker instances.

hoggle2• 11 years ago

Thanks, I will give it a try - even though I have been very happy with kue.

juliangregorian• 11 years ago

What does this do that I can't get from plain Redis Pub Sub?

Exinferis2• 11 years ago

This is a lot more than pub/sub.

A message queue stores messages so that nothing gets lost, when there are no subscribers listening.

A new message will be delivered to only one recipient at a time while in a pub/sub system all subscribers would receive a message and would either all work on the same message/job or would have to decide who does what.

Whoever receives a message will "work" on that message and after success will delete the message. Usually within seconds. So no one will ever receive that message again. If a receiving process crashes or some error happens the message will just pop up again and will be received again after a set invisibility time (default is 30s).

So dropping messages in this message queue keeps them there until some receiver(s) delete them. It does not matter how many message producers / consumers there are. The queue handles the problem of not delivering a message to two consumers within a set time.

vlucas• 11 years ago

Nice library.

This is unrelated, but what are you using for the docs? I love how it looks and scrolls with the topics and code examples right beside the methods.

skinofstars• 11 years ago

Shame they're unviewable with the HTTPS Everywhere plugin enabled.

o_____________o• 11 years ago

https://github.com/rstacruz/flatdoc

vasquque• 11 years ago

Simply queues in Lua https://github.com/tarantool/queue i don't know what's reason to use redis+node.js, if exist tarantool queues

RSMQ: A lightweight Redis based message queue for Node.js

Related Stories