Gavin M. Roy
2011-03-12 23:12:29 UTC
I've been listening to a lot of the conversations at PyCon about asynchronous development. The basic sentiment I've picked up on is that Callback Passing Style (CPS) is not currently in favor in the Python community. This in addition to the popular use of the BlockingConnection in Pika has lead me to think about how to plan Pika's future enhancements. After some conversation with Tony, I believe I have an outline that should appeal to Python developers while keeping Pika asynchronous at its core and retaining CPS. I think that CPS is very powerful and believe it's still very important to Pika's future.
After I release 0.9.5, I will start development on Pika 2.0 which will be an effort to create a more pythonic approach to using Pika while retaining the ability to use CSP and keeping it asynchronous at its core.
The roadmap for changes to Pika 2.0:
- Backwards incompatible change that drops Python 2.4, 2.5 support
- Adding Python 3 support
- Remove existing connection adapter system
- Implement new pattern for use, behavior based use focused on both Asynchronous callback passing style and "Pythonic" development.
- Both behaviors available from the same API calling same classes and methods
- Async:
- Merge existing connections into one connection system with IOLoop override
- Supporting internal IOLoop, tornado, twisted
- Pythonic:
- high-level blocking on synchronous AMQP commands
- Generator for receiving messages from Basic.Publish
- API notation closer to AMQP spec for implementation.
- *.*Ok frames will only be passed back in CPS use.
- Calling methods like queue.declare will return a success indicator and attributes returned in the Ok frame will be assigned to attributes of the class.
- basic.consume and basic.get will return a single object with a materialized view of the Method, Header and Body frames.
- Build in support for specific broker types and pure AMQP 0-9-1.
Here's an example of what I expect Pika 2.0 to look like for non-CSP use. Note this is more of an idea of how it will work for someone using Pika than a spec or actual code.
from pika.rabbitmq import Connection
from pika import Basic
from pika import Channel
from pika import Exchange
from pika import Queue
from sys import exit
# All the attributes can be passed in via constructor or assigned
connection = Connection()
connection.host = 'localhost'
connection.port = 5762
connection.user = 'guest'
connection.pass = 'guest'
connection.vhost = '/'
# Not much new here
try:
connection.open()
except pika.ConnectException as e:
print "Could not connect: %s" % e
sys.exit(0)
# Channel construction outside of connection context, instead pass
# the Connection in
channel = Channel()
try:
channel.open(connection)
except pika.TimeoutException as e:
print "Could not open a channel: %s" % e
except pika.ConnectionClosedException as e:
print "Could not open a channel, the connection is closed"
# All the attributes can be passed in via constructor or assigned
exchange = Exchange(channel)
exchange.name = 'test'
exchange.type = 'fanout'
exchange.durable = True
exchange.declare()
# All the attributes can be passed in via constructor or assigned
queue = Queue(channel)
queue.name = 'my_queue'
queue.auto_delete = False
queue.durable = True
queue.passive = False
# Declare the queue and expect a bool
if not queue.declare():
raise Exception("Could not declare my queue")
# Print info about the queue that was mapped automatically when
# Queue.DeclareOk was received
print 'Queue "%s"' % queue.name
print ' Depth : ' % queue.message_count
print ' Consumers : %i' % queue.consumer_count
# Bind the queue
queue.bind(exchange=exchange, routing_key='test.my_queue')
# A generator returns one object for a message
for message in Basic.consume(my_channel, routing_key="test.my_queue"):
print 'Delivery Tag : %s' % message.delivery_tag
print 'Channel : %i' % message.channel
print 'Body Size : %i' % len(message.body)
print 'Properties'
print ' Content-Type : %s' % message.properties.content_type
print ' Timestamp : %s' % message.properties.timestamp
print ' User Id : %s' % message.properties.user_id
print ' App Id : %s' % message.properties.app_id
print 'Body : %s' % message.body
I am looking for feedback on this direction. Do these changes and the example make sense to existing Pika and RabbitMQ uses? Would you change anything about this direction? What would you improve?
Regards,
Gavin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110312/9762e6be/attachment.htm>
After I release 0.9.5, I will start development on Pika 2.0 which will be an effort to create a more pythonic approach to using Pika while retaining the ability to use CSP and keeping it asynchronous at its core.
The roadmap for changes to Pika 2.0:
- Backwards incompatible change that drops Python 2.4, 2.5 support
- Adding Python 3 support
- Remove existing connection adapter system
- Implement new pattern for use, behavior based use focused on both Asynchronous callback passing style and "Pythonic" development.
- Both behaviors available from the same API calling same classes and methods
- Async:
- Merge existing connections into one connection system with IOLoop override
- Supporting internal IOLoop, tornado, twisted
- Pythonic:
- high-level blocking on synchronous AMQP commands
- Generator for receiving messages from Basic.Publish
- API notation closer to AMQP spec for implementation.
- *.*Ok frames will only be passed back in CPS use.
- Calling methods like queue.declare will return a success indicator and attributes returned in the Ok frame will be assigned to attributes of the class.
- basic.consume and basic.get will return a single object with a materialized view of the Method, Header and Body frames.
- Build in support for specific broker types and pure AMQP 0-9-1.
Here's an example of what I expect Pika 2.0 to look like for non-CSP use. Note this is more of an idea of how it will work for someone using Pika than a spec or actual code.
from pika.rabbitmq import Connection
from pika import Basic
from pika import Channel
from pika import Exchange
from pika import Queue
from sys import exit
# All the attributes can be passed in via constructor or assigned
connection = Connection()
connection.host = 'localhost'
connection.port = 5762
connection.user = 'guest'
connection.pass = 'guest'
connection.vhost = '/'
# Not much new here
try:
connection.open()
except pika.ConnectException as e:
print "Could not connect: %s" % e
sys.exit(0)
# Channel construction outside of connection context, instead pass
# the Connection in
channel = Channel()
try:
channel.open(connection)
except pika.TimeoutException as e:
print "Could not open a channel: %s" % e
except pika.ConnectionClosedException as e:
print "Could not open a channel, the connection is closed"
# All the attributes can be passed in via constructor or assigned
exchange = Exchange(channel)
exchange.name = 'test'
exchange.type = 'fanout'
exchange.durable = True
exchange.declare()
# All the attributes can be passed in via constructor or assigned
queue = Queue(channel)
queue.name = 'my_queue'
queue.auto_delete = False
queue.durable = True
queue.passive = False
# Declare the queue and expect a bool
if not queue.declare():
raise Exception("Could not declare my queue")
# Print info about the queue that was mapped automatically when
# Queue.DeclareOk was received
print 'Queue "%s"' % queue.name
print ' Depth : ' % queue.message_count
print ' Consumers : %i' % queue.consumer_count
# Bind the queue
queue.bind(exchange=exchange, routing_key='test.my_queue')
# A generator returns one object for a message
for message in Basic.consume(my_channel, routing_key="test.my_queue"):
print 'Delivery Tag : %s' % message.delivery_tag
print 'Channel : %i' % message.channel
print 'Body Size : %i' % len(message.body)
print 'Properties'
print ' Content-Type : %s' % message.properties.content_type
print ' Timestamp : %s' % message.properties.timestamp
print ' User Id : %s' % message.properties.user_id
print ' App Id : %s' % message.properties.app_id
print 'Body : %s' % message.body
I am looking for feedback on this direction. Do these changes and the example make sense to existing Pika and RabbitMQ uses? Would you change anything about this direction? What would you improve?
Regards,
Gavin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110312/9762e6be/attachment.htm>