Bhavin's Blog

4 Apr, 2010

Selecting a Message Queue – AMQP or ZeroMQ

Posted by Bhavin Turakhia

I have spent a better part of my Sunday researching some of the messaging queue options available out there. My research was divided primarily between AMQP queues and ZeroMQ. For those who came in late - iMatix the company behind ZeroMQ was also involved in defining the AMQP specs. iMatix recently announced that they would no longer continue supporting the AMQP spec and instead focus on ZeroMQ – their brokerless queue product

ZeroMQ is a fundamentally different approach to message queues from AMQP. Pieter and folks from iMatix have made several posts on the flaws in the design and philosophy of AMQP (check the resources section below). Infact during the course of my study today these posts were amongst the most interesting material I have read.

Here are some thoughts around considerations on our part with respect to selecting a queue for our application needs -

Ordered Messages

Some queues provide guarantees of ordered delivery, while others do not. Order of delivery may not be required in all applications, and adds complexity. However as an eg – If your messages consist of updates to state, then two messages updating the same entity, may result in inconsistency if their order is switched. These complexities would then have to be resolved in your application if the underlying messaging system does not offer ordering guarantees.

Persistence

If the queue itself does not manage persistence, it will have to be handled by your application. Persistence is required for reliability. A queue that manages all messages and state in memory will lose any undelivered messages incase of a node restart/downtime/crash.

Clustering

Broker based queues may or may not offer clustering. Some message queues offer fail-overs but not clustering. Clustering allows multiple nodes to be used as a single active-active instance, increasing availability.

I also have a short summary on the 2 options I am currently considering -

ZeroMQ

• Crazy fast
• Brokerless architecture
• In-process library
• Lower latencies
• Very simple to use
• No persistence – requiring higher layers to manage persistence

RabbitMQ

• AMQP compliant
• Written in erlang
• Small footprint and seemingly fewer lines of code in comparison to other AMQP compliant queue managers

Resources

Here is a list of the more interesting resources I read through today:

• Whats wrong with AMQP – http://www.imatix.com/articles:whats-wrong-with-amqp - The most interesting article I read in my research. This is a fairly long article but I would strongly recommend anyone who is planning on using queues to read it end to end. Irrespective of whether we choose ZeroMQ or any AMQP compliant queue, this article provides some good insights.
• http://www.imatix.com/articles:introduction-to-restms – A nice document explaining restms protocol by pieter. RestMS can act as a bridge between Messaging systems and HTTP/REST based systems
• http://www.ipocracy.com/blog:10-principles-for-amqp – Ten ways that AMQP can be made simpler, more backwards compatible, more interesting, and overall more enjoyable and successful for all who work on it and use it
• http://www.amqp.org/
• http://www.zeromq.org/
• http://www.openamq.org/
• http://www.restms.org/
• http://www.zeromq.org/area:docs-v20
• http://api.zeromq.org/zmq.htmlhttp://storage.synchost.com/eanderson/2010/2010-02-18%2010.02%20Low%20Latency_%20High%20Throughput_%20Durable_%20RESTful_%20Open_%20Standards_%20___.wmv
• http://www.zeromq.org/local–files/area:whitepapers/messaging-2010-02-17.pdf
• http://www.zeromq.org/area:whitepapers
• http://www.zeromq.org/faq
• http://www.zeromq.org/local–files/area:studies/solvians.pdf
• http://www.zeromq.org/whitepapers:message-matching – highly optimized message maching algorithm
• http://www.zeromq.org/results:ib-tests-v206 – 4.7 million messages per second for a 64 byte message
• http://www.imatix.com/articles:how-to-build-utterly-reliable-systems
• http://www.slideshare.net/mattmatt/rabbitmq-and-nanite
• http://lists.zeromq.org/pipermail/zeromq-dev/2008-December/000246.html
• http://www.zeromq.org/docs:zsock

Comments

Leave a comment

Name (required)

E-mail (required)

URI

Your Comments

Spam protection by WP Captcha-Free