[rabbitmq-discuss] AMQP authentication with RabbitMQ

[Laurens Van Houtven]

Hi :-)


I'm wondering about encryption and authentication within AMQP (using RabbitMQ).

I'm porting existing code that did not have a messaging component.
This code used per-client (in AMQP these would be producers)
pre-shared certificates to do authentication. The problem in porting
it (as I see it at least), is that with RabbitMQ+stunnel, the servers
(= AMQP consumers) never get to see the SSL'ed data coming from the
clients, so they cannot see the client certificate, so I can't use it
to do authentication.

I'm trying to find a solution to this problem. I'd like to keep using
SSL, but just using SSL with pre-shared certificates only guarantees
my consumers that the producer is *a* known user -- my consumers don't
know *who* the producer is. For example, given two registered users
(with pre shared SSL certs) Alice and Mallory (the latter being up to
no good), I want to prevent that Mallory logs in with his SSL cert and
his username, but then pretends to be Alice in the actual message
contents.

I'm not yet intricately familiar with the way RabbitMQ ACL's work, but
I think that it could be solved by using SSL (with stunnel), together
with a username and a password, and then have one vhost per user and
per consumer role. I'm not entirely sure how well RabbitMQ is designed
to scale as the number of vhosts increase, since this would result in
N*M queues (with M the number of services and N the number of uesrs).
Also, I think this means I also need one queue per user and per
service, since otherwise my consumers still don't know anything about
the user identity. I think this (unfortunately) means that my
consumers will need to be reconfigured every time a new user is added,
which obviously sucks pretty bad.

Am I better off putting authentication in my message contents?

Thanks in advance
Laurens

[Darien Kindlund]

Hi Laurens,

I'm not an OpenSSL nor RabbitMQ expert, but I got the impression that
stunnel (which uses OpenSSL) somehow populates certificate information
after the connection is established via environment variables. See
this URL for more information about the types of environment variables
available:
http://httpd.apache.org/docs/2.2/mod/mod_ssl.html

Again, I'm not sure if 'mod_ssl' had to do anything with OpenSSL in
order to expose these variables to apache, but I'm guessing it _can_
be done with stunnel.

Hope this helps,
-- Darien

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi :-)
I'm wondering about encryption and authentication within AMQP (using RabbitMQ).
I'm porting existing code that did not have a messaging component.
This code used per-client (in AMQP these would be producers)
pre-shared certificates to do authentication. The problem in porting
it (as I see it at least), is that with RabbitMQ+stunnel, the servers
(= AMQP consumers) never get to see the SSL'ed data coming from the
clients, so they cannot see the client certificate, so I can't use it
to do authentication.
I'm trying to find a solution to this problem. I'd like to keep using
SSL, but just using SSL with pre-shared certificates only guarantees
my consumers that the producer is *a* known user -- my consumers don't
know *who* the producer is. For example, given two registered users
(with pre shared SSL certs) Alice and Mallory (the latter being up to
no good), I want to prevent that Mallory logs in with his SSL cert and
his username, but then pretends to be Alice in the actual message
contents.
I'm not yet intricately familiar with the way RabbitMQ ACL's work, but
I think that it could be solved by using SSL (with stunnel), together
with a username and a password, and then have one vhost per user and
per consumer role. I'm not entirely sure how well RabbitMQ is designed
to scale as the number of vhosts increase, since this would result in
N*M queues (with M the number of services and N the number of uesrs).
Also, I think this means I also need one queue per user and per
service, since otherwise my consumers still don't know anything about
the user identity. I think this (unfortunately) means that my
consumers will need to be reconfigured every time a new user is added,
which obviously sucks pretty bad.
Am I better off putting authentication in my message contents?
Thanks in advance
Laurens
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Use GnuPG with Firefox : http://getfiregpg.org (Version: 0.7.6)
iEYEARECAAYFAkpUz+4ACgkQT5v5zGkvKT7VjQCfRxz/PD00FieuaEhQ3aAsjOFB
zyYAoLwdWb8pd34zaHCldzGUFnrD70yh
=wttN
-----END PGP SIGNATURE-----
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[eric]

Post by Darien Kindlund
Hi Laurens,
I'm not an OpenSSL nor RabbitMQ expert, but I got the impression that
stunnel (which uses OpenSSL) somehow populates certificate information
after the connection is established via environment variables. See
this URL for more information about the types of environment variables

This isn't how stunnel works, it creates simple SSL server and client
wrappers/proxies. Stunnel doesn't need to know anything about the
application-layer protocols it encapsulates, doesn't need environmental
variables, LD_PRELOAD, or any other funny magic. For these reasons,
stunnel is a pretty good fit for Rabbit, where encryption is needed.

How it works is quite simple, really. It sends and receives clear-text
on one side (i.e. a local RabbitMQ), and encrypted communications on the
other (i.e. the internet). This is what allows applications that work
purely with clear-text such as Rabbit, to communicate securely via
stunnel.

The server side is relatively simple to set up, because aside from the
extra process running, it requires no changes to the server daemon
process. You can simply set up a proxy for the AMQP port and walk away.

The client side, however, will need either SSL support in the client's
AMQP library, or will need to connect to a local stunnel instance to
provide a reverse SSL proxy. That is, your client application can speak
clear-text to the local stunnel, this stunnel will speak SSL to the
remote stunnel (which, in turn, speaks clear-text to RabbitMQ). Its
very clear that SSL support in the client's AMQP library is preferable.

[Darien Kindlund]

Hi Eric,
Thanks for the clarification. So what you're saying is: "If you need
to access these environment variables, don't use stunnel and just have
your client application talk to OpenSSL directly."
-- Darien

Post by eric

Post by Darien Kindlund
Hi Laurens,
I'm not an OpenSSL nor RabbitMQ expert, but I got the impression that
stunnel (which uses OpenSSL) somehow populates certificate information
after the connection is established via environment variables. ?See
this URL for more information about the types of environment variables

This isn't how stunnel works, it creates simple SSL server and client
wrappers/proxies. ?Stunnel doesn't need to know anything about the
application-layer protocols it encapsulates, doesn't need environmental
variables, LD_PRELOAD, or any other funny magic. ?For these reasons,
stunnel is a pretty good fit for Rabbit, where encryption is needed.
How it works is quite simple, really. It sends and receives clear-text
on one side (i.e. a local RabbitMQ), and encrypted communications on the
other (i.e. the internet). ?This is what allows applications that work
purely with clear-text such as Rabbit, to communicate securely via
stunnel.
The server side is relatively simple to set up, because aside from the
extra process running, it requires no changes to the server daemon
process. ?You can simply set up a proxy for the AMQP port and walk away.
The client side, however, will need either SSL support in the client's
AMQP library, or will need to connect to a local stunnel instance to
provide a reverse SSL proxy. ?That is, your client application can speak
clear-text to the local stunnel, this stunnel will speak SSL to the
remote stunnel (which, in turn, speaks clear-text to RabbitMQ). ?Its
very clear that SSL support in the client's AMQP library is preferable.
--
Regards,
Eric Windisch

[eric]

One, hopefully final, thought is that in all of these scenarios, we are
not discussing the security of clustered Rabbit configurations and
encryption between brokers. This is also something that the RabbitMQ
FAQ fails to address in its coverage "Channel Encryption", which it
otherwise addresses very well.

For now, I'd say to put all of the RabbitMQ brokers on a secure network
for intercommunication. If not, in *addition* to using SSL, you should
encrypt the messages as well. Presumably, you could also wrap the OTP
communication in stunnel, but that would be a bit more complex to
implement.

[eric]

Post by Darien Kindlund
Thanks for the clarification. So what you're saying is: "If you need
to access these environment variables, don't use stunnel and just have
your client application talk to OpenSSL directly."

There are a few use-cases to consider here.

First, is the use-case where you simply want encryption and don't care
about knowing how it is done. That is easy, you can just use stunnel
directly on both ends. This is encryption pure and simple, without any
AAA.

Second, you have a case where you simply want to your consumer or
publisher to know basic details such as the server's DN, the cipher
used, etc. In that case, the server can use stunnel and then the
client's AMQP library must understand SSL. This provides, at a minimum,
verification of the server's identity to the clients (otherwise, someone
could install a rogue stunnel proxy and you'd never know!)

Third, there are certainly use-cases where a publisher speaks SSL with a
client-certificate, and the consuming client wants to know where the
message came from, which client published the message. This is
impossible without modifications to RabbitMQ itself and, more
importantly, as I understand it, the AMQP protocol.

For use-case #3, your best bet is to implement the solution to the
second use-case and then having the contents of your messages signed.
This is how e-mail is done currently, and it works reasonably well.

Before anyone publicly suggests mentions DTLS or SCTP, they only solve
the transport issues and do not solve any AAA issues for Rabbit.

[Darien Kindlund]

Okay, I think Laurens original issue was that she wanted RabbitMQ to
verify SSL connections -- not individual messages... with the idea of
different client certificates being allowed access to different
vhosts.

So, based on this, there's no need to have consumers verify
signed/encrypted messages ... since the logic of comparing credentials
per vhost is more suited on the RabbitMQ server than pushing that
verification to each consumer. I agree that if you're looking for
end-to-end security, signing/encrypting/verifying messages certainly
prevents any consumers from handling untrusted data. However, having
only this signing/encrypting/verifying capability at the end points
(producer/consumer) and doing nothing on RabbitMQ does not address the
issue of dealing with an _untrusted_ producer from shoving malformed
messages onto RabbitMQ -- although these messages would eventually be
discarded by the consumer, it could still end up being a
denial-of-service (DoS) condition.

Post by eric
Second, you have a case where you simply want to your consumer or
publisher to know basic details such as the server's DN, the cipher
used, etc. In that case, the server can use stunnel and then the
client's AMQP library must understand SSL. This provides, at a minimum,
verification of the server's identity to the clients (otherwise, someone
could install a rogue stunnel proxy and you'd never know!)

I think this is closer (but not exactly) what Laurens was looking for.
Specifically, the RabbitMQ server needs to verify the CLIENT's
certificate credentials (not any server certs) -- specifically (I'm
guessing) verifying the client's certificate is valid and that its
path is valid (which stunnel can already do for you, given a valid set
of CA certificates to use). But also, look at the client
certificate's DN and depending on what the DN matches... allow or deny
access to the corresponding vhost channel.

After thinking about it further, yes, this capability would definitely
require modifying the RabbitMQ server's source code... the trick is
whether or not stunnel exposes the client certificate's DN as an
environment variable which can then be used inside of the Erlang code.
If that's not possible, then Lauren would have to not use stunnel and
simply develop a new wrapper which talks directly to the OpenSSL
libraries.

-- Darien

Post by eric

Thanks for the clarification. ?So what you're saying is: "If you need
to access these environment variables, don't use stunnel and just have
your client application talk to OpenSSL directly."

There are a few use-cases to consider here.
First, is the use-case where you simply want encryption and don't care
about knowing how it is done. ?That is easy, you can just use stunnel
directly on both ends. ?This is encryption pure and simple, without any
AAA.
Second, you have a case where you simply want to your consumer or
publisher to know basic details such as the server's DN, the cipher
used, etc. ?In that case, the server can use stunnel and then the
client's AMQP library must understand SSL. ?This provides, at a minimum,
verification of the server's identity to the clients (otherwise, someone
could install a rogue stunnel proxy and you'd never know!)
Third, there are certainly use-cases where a publisher speaks SSL with a
client-certificate, and the consuming client wants to know where the
message came from, which client published the message. ?This is
impossible without modifications to RabbitMQ itself and, more
importantly, as I understand it, the AMQP protocol.
For use-case #3, your best bet is to implement the solution to the
second use-case and then having the contents of your messages signed.
This is how e-mail is done currently, and it works reasonably well.
Before anyone publicly suggests mentions DTLS or SCTP, they only solve
the transport issues and do not solve any AAA issues for Rabbit.
--
Regards,
Eric Windisch

[eric]

Post by Darien Kindlund
After thinking about it further, yes, this capability would definitely
require modifying the RabbitMQ server's source code... the trick is
whether or not stunnel exposes the client certificate's DN as an
environment variable which can then be used inside of the Erlang code.
If that's not possible, then Lauren would have to not use stunnel and
simply develop a new wrapper which talks directly to the OpenSSL
libraries.

Then, to do what Lauren wants requires SASL/EXTERNAL. While related, it
is a much different question. This same mechanism would be how one
would support LDAP and other authentication mechanisms.

I don't know how complete the SASL implementation used by Rabbit is, or
what that implementation is.

Pardon me for, uh, not reading the original post ;-)

[Matthias Radestock]

Darien, Eric, Lauren,

Post by Darien Kindlund
Okay, I think Laurens original issue was that she wanted RabbitMQ to
verify SSL connections -- not individual messages... with the idea of
different client certificates being allowed access to different
vhosts.
So, based on this, there's no need to have consumers verify
signed/encrypted messages ... since the logic of comparing credentials
per vhost is more suited on the RabbitMQ server than pushing that
verification to each consumer. I agree that if you're looking for
end-to-end security, signing/encrypting/verifying messages certainly
prevents any consumers from handling untrusted data. However, having
only this signing/encrypting/verifying capability at the end points
(producer/consumer) and doing nothing on RabbitMQ does not address the
issue of dealing with an _untrusted_ producer from shoving malformed
messages onto RabbitMQ -- although these messages would eventually be
discarded by the consumer, it could still end up being a
denial-of-service (DoS) condition.

We are in the final stages of qa'ing "native" SSL support for the
RabbitMQ server, Java, .net and Erlang clients that addressed the above.
A description of the new functionality is available at
https://dev.rabbitmq.com/wiki/SslSupport, and you can find the code in
the 'bug19356' branches of the respective hg repositories.

Post by Darien Kindlund
Specifically, the RabbitMQ server needs to verify the CLIENT's
certificate credentials (not any server certs) -- specifically (I'm
guessing) verifying the client's certificate is valid and that its
path is valid (which stunnel can already do for you, given a valid set
of CA certificates to use). But also, look at the client
certificate's DN and depending on what the DN matches... allow or deny
access to the corresponding vhost channel.

The new code can verify the client certs. Access to particular vhosts,
and specific resources can be controlled by the existing RabbitMQ
permissions system based on the supplied user name.

Note that there is no association of subjectDNs and users, so in
principle any user can use any trusted cert. That should be ok though as
long as the username/password are treated as sensitive information in
the same way as the private keys for the client certs.


Regards,

Matthias

[Darien Kindlund]

Hi Matthias,

Great start!

Couple of basic questions:
1) So, for reference, would we call this AMQPS, AMQP-SSL, or SAMPQ ?
2) Are you planning on supporting CRLs and/or OCSP for certificate revocation?
3) Can we specify the cipher strength?

Note that there is no association of subjectDNs and users, so in principle
any user can use any trusted cert. That should be ok though as long as the
username/password are treated as sensitive information in the same way as
the private keys for the client certs.

4) Okay once SSL is supported natively, do you think a future version
of RabbitMQ would be able to map particular subjectDNs to existing
username/password credentials? It would be really nice if clients
could authenticate with only client certs and nothing else. An
example lightweight implementation could simply be:
- Create username/password pairs using rabbitmqctl, where each pair
really represents a "client profile". We assume the password portion
will never be used, so we set it to something ridiculously long.
- Use rabbitmqctl to declare a many-to-one relationship between sets
of subjectDNs and usernames, where each subjectDNs match maps to a
valid username. Conflicting subjectDN entries could be resolved
either by: (a) first match wins (so order matters) or (b) most
specific match wins (and actively disallow duplicate
subjectDN/username bindings).

I'm guessing #4 may actually break the existing AMQP spec, since we're
talking about bypassing username/password authentication. If that's
the case, I'm not sure if you typically wait for the spec to get
ratified before implementing any experimental features, such as this.

Regards,
-- Darien

[Matthias Radestock]

Darien,

Post by Darien Kindlund
1) So, for reference, would we call this AMQPS, AMQP-SSL, or SAMPQ ?

Does it have to have a name?

Post by Darien Kindlund
2) Are you planning on supporting CRLs and/or OCSP for certificate revocation?
3) Can we specify the cipher strength?

We support whatever the Erlang SSL implementation supports. See
http://www.erlang.org/doc/man/new_ssl.html for details. That's a moving
target, and ATM the answers to the above are 'no' and 'yes'.

Post by Darien Kindlund
4) Okay once SSL is supported natively, do you think a future version
of RabbitMQ would be able to map particular subjectDNs to existing
username/password credentials? It would be really nice if clients
could authenticate with only client certs and nothing else.
I'm guessing #4 may actually break the existing AMQP spec, since we're
talking about bypassing username/password authentication. If that's
the case, I'm not sure if you typically wait for the spec to get
ratified before implementing any experimental features, such as this.

AMQP has some built-in support for negotiating different security
mechanisms, so your latter concern isn't an issue. Making the necessary
changes at the server and client end would take some time, but it
shouldn't be a big job. Perhaps this is something you could have a stab
at yourself once the new SSL support has landed?


Regards,

Matthias.

[Darien Kindlund]

Post by Matthias Radestock

Post by Darien Kindlund
1) So, for reference, would we call this AMQPS, AMQP-SSL, or SAMPQ ?

Does it have to have a name?

Well, AMQP over SSL is a little long; plus, I assume IANA will want a
protocol name if you're going to make 5672 (amqp) and 5673 (amqps) in
/etc/services as standards.

Post by Matthias Radestock

Post by Darien Kindlund
2) Are you planning on supporting CRLs and/or OCSP for certificate revocation?
3) Can we specify the cipher strength?

We support whatever the Erlang SSL implementation supports. See
http://www.erlang.org/doc/man/new_ssl.html for details. That's a moving
target, and ATM the answers to the above are 'no' and 'yes'.

Okay, gotcha. #2 wasn't a hard requirement, I was just more
interested to see if it was on anyone's radar.

Post by Matthias Radestock

Post by Darien Kindlund
4) Okay once SSL is supported natively, do you think a future version
of RabbitMQ would be able to map particular subjectDNs to existing
username/password credentials? ?It would be really nice if clients
could authenticate with only client certs and nothing else.
I'm guessing #4 may actually break the existing AMQP spec, since we're
talking about bypassing username/password authentication. ?If that's
the case, I'm not sure if you typically wait for the spec to get
ratified before implementing any experimental features, such as this.

AMQP has some built-in support for negotiating different security
mechanisms, so your latter concern isn't an issue. Making the necessary
changes at the server and client end would take some time, but it shouldn't
be a big job. Perhaps this is something you could have a stab at yourself
once the new SSL support has landed?

Depends. I'm not an erlang expert by any means; I'm currently working
on integrating some erlang supervisory code. Assuming I can grok the
syntax and language, I could take a stab at this later on. A change
like this would probably also mean digging through the nmesia DB
schema, as the mappings would probably have to reside there (wherever
the username/passwords are stored).

-- Darien

[Matthias Radestock]

Darien,

Post by Darien Kindlund
Well, AMQP over SSL is a little long; plus, I assume IANA will want a
protocol name if you're going to make 5672 (amqp) and 5673 (amqps) in
/etc/services as standards.

amqp and amqps have already been registered with IANA for ports 5672 and
5671 (NB: not ...3) respectively. So AMQPS it is.

Post by Darien Kindlund
I'm not an erlang expert by any means; I'm currently working on
integrating some erlang supervisory code. Assuming I can grok the
syntax and language, I could take a stab at this later on. A change
like this would probably also mean digging through the nmesia DB
schema, as the mappings would probably have to reside there (wherever
the username/passwords are stored).

Yes, some tweaks to the mnesia schema will be necessary. Should all be
fairly straightforward though and not lead you anywhere near the really
complex pieces of the rabbitmq code base.


Matthias.