Re: [amqp] [management] Proposal for event notifications (initial outline)

[Alan Conway <aconway@redhat.com>]

On Thu, 2015-02-12 at 09:01 -0800, Matthew Arrott wrote:
> +1

Wow, you just ... agree? I don't think that's ever happened to me before
on a standards committee :)

A note on implementation: the naive way to implement this proposal is to
keep a list of filters and iterate over it with each event to decide
whether to send to that link. That is linear in the number of links and
won't scale with many subscribers. A better way to implement it is to
compile all your filters into a decision tree. That gives constant-time
look-up (worst-case depth of a branch is limited by the number of event
attributes defined) The qpid c++ broker's 0-10 topic matching went
through this linear-oops-no-tree cycle so I'm not just making this up.
Probably that's not suitable discussion for the spec, but wanted to
reassure anyone who was worrying that this might require linear look-up.

> -----Original Message-----
> From: Alan Conway <aconway@redhat.com>
> Date: Wednesday, February 11, 2015 at 1:54 PM
> To: <amqp@lists.oasis-open.org>
> Subject: [amqp] [management] Proposal for event notifications (initial
> outline)
> 
> 
> This is a quick sketch of a design for management event notification. I
> wanted to get feedback before getting more formal. I will probably be
> implementing something like this in the near future which will further
> inform the discussion.
> 
> 
> Goal: Allow clients to subscribe for notification of selected management
> events.
> 
> Justification: Polling (with READ or QUERY) for changes in management
> state is inefficient. Either the clients poll too often, which overloads
> the management agent, or too seldom which causes delays in the system.
> 
> DESIGN:
> 
> Events are sets of named attributes, like entities. An event type name
> MAY begin with an entity type name to indicate the event is associated
> with that entity type or it MAY be independent.
> 
> All event types MUST have an "eventType" attribute that carries the
> event type name. An implementation defines additional attributes for
> each event type.
> 
> A notification message contains a single event. The message subject MUST
> be the event type name.  The application-properties MUST include all the
> event attributes including "eventType".
> 
> To subscribe for events, a client creates a link from the address
> "$management.events" with a filter describing the notification messages
> that it wants. Closing the link ends the subscription.
> 
> The filtering capabilities will depend on the filters provided by the
> agent. An agent SHOULD provide the APACHE.ORG:SELECTOR described in
> <http://www.amqp.org/specification/1.0/filters>. This allows clients to
> filter on arbitrary SQL-like expressions over all the attributes of the
> event including "eventType".
> 
> The event type is also in the message subject, so the legacy filters
> described in <http://www.amqp.org/specification/1.0/filters> MAY also be
> used to filter by event type.
> 
> NOTES:
> 
> This design does not allow multiple events to be batched in a single
> message. IMO an implementation should batch messages at the transport
> (into TCP packets) NOT batch events into a message. Batching in messages
> complicates the client, defeats transport level batching (by forcing
> "fat" messages on the transport) and rules out (or badly complicates)
> the use of AMQP filters to select events.
> 
> I considered an alternative design where the client sends a subscription
> request with a reply-to address and the agent then sends notification
> messages to the reply-to address.
> 
> I rejected that design for the following reasons:
> 
> 1. It is impossible in general to tell when such a subscriber goes away
> if it crashes or fails to send an explicit unsubscribe message.
> Implementations must already deal with closing links when clients
> disconnect or heartbeat out so using links solves this problem. Even in
> indirect topologies (like link-routed Qpid dispatch networks) the
> routers must proxy link closure to be AMQP-correct.
> 
> 2. We should use AMQP standard features to solve AMQP problems, not
> re-invent the wheel in a form only useful for management.
> Filters/selectors were designed for exactly this purpose we should make
> them work for us.
> 
> Feedback much appreciated!!
> 
> Alan.
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
> 
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that 
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 
>