[rabbitmq-discuss] 1 Fanout Exchange, 2 queues, 1 slow, 1 fast

[Kenneth Loafman]

Hi,

I've got a fanout exchange running two queues. One queue is responding
very quickly like normal, the other is intermittent. What would cause
this? The fast queue has 3 consumers, the slow queue has 64 consumers. Is
there a limit to how many consumers can be attached to a queue?

I'm on the current version, 2.8.2, Ubuntu Lucid, up-to-date. What would
you need to diagnose the problem.

...Thanks,
...Ken
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120507/3566bc8a/attachment.htm>

[Emile Joubert]

Hi Kenneth,

Post by Kenneth Loafman
I've got a fanout exchange running two queues. One queue is responding
very quickly like normal, the other is intermittent. What would cause
this? The fast queue has 3 consumers, the slow queue has 64 consumers.
Is there a limit to how many consumers can be attached to a queue?

If you say the queue is responding intermittently, what do you mean
exactly? Are you measuring latency between publish and deliver, or
something else?

The broker does not impose a fixed limit on the number of subscribers to
a queue, but the server capacity may be a limit.

It is possible that the problem is caused by some behaviour of the
clients, or a difference in pre-fetch count. Do you get more comparable
performance if you set the pre-fetch count to 3 on the "intermittent" queue?

Depending on what you are measuring you may wish to enable the firehose
tracer for debugging:

http://www.rabbitmq.com/firehose.html



-Emile

[Kenneth Loafman]

Thanks for the reply.

By intermittent, I mean burst, pause, burst, pause, ...

The problem has been 'solved' for now. We were using basic_get and 96
consumers was too much even with a 1 second delay on no-data-received
(we're swapping over to callbacks ASAP). I reduced the number of consumers
to 32 and got much better throughput, so the queue is keeping up now.

...Ken

Post by Emile Joubert
Hi Kenneth,

Post by Kenneth Loafman
I've got a fanout exchange running two queues. One queue is responding
very quickly like normal, the other is intermittent. What would cause
this? The fast queue has 3 consumers, the slow queue has 64 consumers.
Is there a limit to how many consumers can be attached to a queue?

If you say the queue is responding intermittently, what do you mean
exactly? Are you measuring latency between publish and deliver, or
something else?
The broker does not impose a fixed limit on the number of subscribers to
a queue, but the server capacity may be a limit.
It is possible that the problem is caused by some behaviour of the
clients, or a difference in pre-fetch count. Do you get more comparable
performance if you set the pre-fetch count to 3 on the "intermittent" queue?
Depending on what you are measuring you may wish to enable the firehose
http://www.rabbitmq.com/firehose.html
-Emile

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120508/af071491/attachment.htm>