[rabbitmq-users] ANN rabbitmq-clusterer, a declarative clustering plugin

[Michael Klishin]

On behalf of the RabbitMQ team, I'm happy to announce a new
plugin that makes automatic provisioning of RabbitMQ clusters easier:
https://github.com/rabbitmq/rabbitmq-clusterer.

Binary builds are available from our community plugins page:
https://www.rabbitmq.com/community-plugins.html

If you've tried automating provisioning of a cluster, you've found out
that there are things that make it needlessly painful:

 * Traditional clustering has been developed with a human being
   using command line. This is not 
 * When performing a rolling restart, there's a dependency on
   node startup order.

In this cloud age, both points above are not exactly adequate.
What RabbitMQ needs a completely declarative clustering, and this
is our first step towards that direction.

The plugin can
work with RabbitMQ versions starting with 3.3.x but would be much easier
to use with 3.5.4 and later. Note that it is considered to be experimental
at the moment, even though it has been used internally at Pivotal for
close to 2 years.

We plan on gathering feedback from the community, porting the plugin to use
well known and verified distribute system algorithms, and eventually
roll it into the core.

Please give the clusterer a try and let us know how it goes!
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Gopi Kandhadi]

Thanks MK. I am trying to use this plugin in an environment running 3.5.4
on Windows and get this error on checking the cluster status with this
command. Appreciate any help on this matter.

**************************************************************************
config info:

setx -m RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS "-pa C:\Program Files
(x86)\RabbitMQ
Server\rabbitmq_server-3.5.4\plugins\rabbitmq_clusterer-3.5.x-189b3a81.ez\rabbitmq_clusterer-3.5.x-189b3a81\ebin"

**************************************************************************************************************
rabbitmqctl.bat eval rabbit_clusterer:status().

Error: {badarg,[{io,format,[<0.23.0>,"Clusterer is booting.~n",[[]]],[]},
{erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,669}]},
{rpc,'-handle_call_call/6-fun-0-',5,
[{file,"rpc.erl"},{line,206}]}]}

***************************************************************************************************************
rabbitmqctl.bat status

Status of node ***@P70VMXXXXX05 ...
[{pid,2384},
{running_applications,
[{rabbitmq_clusterer,"Declarative RabbitMQ
clustering","3.5.x-189b3a81"},
{rabbitmq_management,"RabbitMQ Management Console","3.5.4"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.5.4"},
{webmachine,"webmachine","1.10.3-rmq3.5.4-gite9359c7"},
{mochiweb,"MochiMedia Web Server","2.7.0-rmq3.5.4-git680dba8"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.5.4"},
{rabbit,"RabbitMQ","3.5.4"},
{amqp_client,"RabbitMQ AMQP Client","3.5.4"},
{xmerl,"XML parser","1.3.8"},
{os_mon,"CPO CXC 138 46","2.4"},
{inets,"INETS CXC 138 49","6.0"},
{mnesia,"MNESIA CXC 138 12","4.13"},
{sasl,"SASL CXC 138 11","2.5"},
{stdlib,"ERTS CXC 138 10","2.5"},
{kernel,"ERTS CXC 138 10","4.0"}]},
{os,{win32,nt}},
{erlang_version,
"Erlang/OTP 18 [erts-7.0] [64-bit] [smp:4:4] [async-threads:30]\n"},
{memory,
[{total,41550144},
{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,5424},
{queue_procs,2712},
{queue_slave_procs,0},
{plugins,246552},
{other_proc,13530360},
{mnesia,63288},
{mgmt_db,136752},
{msg_index,47136},
{other_ets,1199680},
{binary,43056},
{code,20193288},
{atom,711569},
{other_system,5370327}]},
{alarms,[]},
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{amqp,5672,"0.0.0.0"}]},
{vm_memory_high_watermark,0.75},
{vm_memory_limit,12884502528},
{disk_free_limit,50000000},
{disk_free,321870512128},
{file_descriptors,
[{total_limit,8092},
{total_used,4},
{sockets_limit,7280},
{sockets_used,2}]},
{processes,[{limit,1048576},{used,192}]},
{run_queue,0},
{uptime,1271}]

********************************************************************************************************************************************************

Thanks
GK

[Michael Klishin]

Post by Gopi Kandhadi
setx -m RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS "-pa C:\Program
Files (x86)\RabbitMQ Server\rabbitmq_server-3.5.4\plugins\rabbitmq_clusterer-3.5.x-189b3a81.ez\rabbitmq_clusterer-3.5.x-189b3a81\ebin"

You need to override RABBITMQ_BOOT_MODULE as well (only available in 3.5.4+).
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Gopi Kandhadi]

Post by Gopi Kandhadi

Post by Gopi Kandhadi
setx -m RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS "-pa C:\Program
Files (x86)\RabbitMQ

Server\rabbitmq_server-3.5.4\plugins\rabbitmq_clusterer-3.5.x-189b3a81.ez\rabbitmq_clusterer-3.5.x-189b3a81\ebin"
You need to override RABBITMQ_BOOT_MODULE as well (only available in 3.5.4+).
--
MK
Staff Software Engineer, Pivotal/RabbitMQ

Sorry missed mentioning it in my previous email. I do have this set up in
my "rabbitmq-defaults.bat" file. The BOOT_MODULE entry has been changed
from "rabbit" to "rabbit_clusterer"

BOOT_MODULE=rabbit_clusterer

GK

[Michael Klishin]

Post by Gopi Kandhadi
Sorry missed mentioning it in my previous email. I do have this
set up in my "rabbitmq-defaults.bat" file. The BOOT_MODULE
entry has been changed from "rabbit" to "rabbit_clusterer"
BOOT_MODULE=rabbit_clusterer

Are all nodes listed in the clusterer config online? (if so, you will
see a note about that printed to stdout) 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Gopi Kandhadi]

Post by Michael Klishin

Post by Gopi Kandhadi
Sorry missed mentioning it in my previous email. I do have this
set up in my "rabbitmq-defaults.bat" file. The BOOT_MODULE
entry has been changed from "rabbit" to "rabbit_clusterer"
BOOT_MODULE=rabbit_clusterer

Are all nodes listed in the clusterer config online? (if so, you will
see a note about that printed to stdout)
--

I am using a file to configure the clustering and have the nodes listed
there. This is just after starting the node and before executing the
apply_config(cluster.config) command. I tried the cluster status command
(copied from my above email) and I get this response and so did not proceed
any further.

*rabbitmqctl.bat eval rabbit_clusterer:status().*

* Error: {badarg,[{io,format,[<0.23.0>,"Clusterer is booting.~n",[[]]],[]},*
* {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,669}]},*
* {rpc,'-handle_call_call/6-fun-0-',5,*
* [{file,"rpc.erl"},{line,206}]}]}*

[Michael Klishin]

Post by Gopi Kandhadi
I am using a file to configure the clustering and have the nodes
listed there. This is just after starting the node and before
executing the apply_config(cluster.config) command. I tried
the cluster status command (copied from my above email) and I
get this response and so did not proceed any further.

Yes, this can be an issue with running the status function
when the clusterer isn't initialised.

I highly recommend using an external config file or at least inline
configuration. We'll remove the eval section from the docs, as setting
things up that way are not necessary and go against the goal of making
automation easier. 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ