Re: [amqp] [management] Proposal for event notifications (initial outline)

[Alan Conway <aconway@redhat.com>]

On Sat, 2015-02-14 at 15:50 +0000, John O'Hara wrote:
> Alan
> 
> 
> It's been a while since I commented.

Hey John, long time no see :) I like what you've done here, and the
rationale section at the bottom is very good also.
> 
> 
> One question: why reverse domain?  I know this shows up occasionally
> in DNS, but is there a logical reason why it has to be this way?

That is the convention already used for entity type names in the
management spec so I followed that. I guess it originates with the
influence of Java developers on the spec ;) The Qpid Management
Framework, which is a big influence on the proposal, also used that
convention (Java developers are everywhere.) I'm not averse to other
conventions but consistency with the naming of entity types is important
IMO. 

In the implementation I am working on I treat the prefix like a
namespace - it is optional in our tools, generally omitted in
documentation, and our agent will accept un-prefixed names. However we
always include the prefix on any message we send.

> Also for events, is there a core set of event types that means we
> don't need every management system to be aware of all vendors?

Not yet. I expect to start implementing this in coming months as I
already have components that are polling when they should be getting
events. My first driving use case is specific to Qpid Dispatch Router
(updating network topology maps, which is alien to traditional
client/server AMQP products.) However there are other things I need
events for that would be common to all AMQP implementations

I think we could actually lift a set of standard events right out of the
AMQP spec (sections 2-5): e.g. connection open/close, link open/close,
message sent/received, delivery settled etc. We could almost duplicate
the performative arguments as event attributes... Hmm... There might be
a way to generate a schema for a set of standard events right out of the
AMQP XML... Hmm... Would love to hear other peoples thoughts!

> Finally, I really like how you've spoken at the bottom of doing things
> the AMQP-way.  There should be a handful of standard patterns, and
> these agents are clearly just using one of them - one which is very
> common in apps which use messaging.  Has anyone thought to write up
> the core patterns so 1) its quicker to get things done and 2) more
> importantly, people "re-inventing" don't miss some important detail.

"Thought", almost certainly. "Written", not that I know of :)



> On 13 February 2015 at 19:12, Alan Conway <aconway@redhat.com> wrote:
>         After some encouraging feedback, here it is again specified
>         and
>         simplified (thanks Jakub). I'm still interested in feedback on
>         the spec
>         or potential implementation issues. I will probably be
>         implementing this
>         or something like it shortly.
>         
>         Cheers
>         Alan.
>         
>         ----
>         # AMQP Management Event Notification
>         
>         Management clients often need to know when the state of a
>         managed entity
>         has changed.  This can be for monitoring or logging purposes,
>         or to
>         react when something becomes ready for use.
>         
>         Polling the management agent for changes with READ or QUERY
>         operations
>         is inefficient. Clients either poll too often, which overloads
>         the
>         agent, or too seldom which causes needless delays in the
>         system.
>         
>         Event notification allows management clients to subscribe for
>         continuous
>         notification of selected events as soon as the agent becomes
>         aware of
>         them.
>         
>         ## Event Types
>         
>         An Event Type is a named set of attributes. The name is a
>         case-sensitive
>         string. Implementers MAY define their own Event Types which
>         MUST be
>         named using a reverse domain name prefix owned by the
>         implementer, e.g.
>         "com.example.broker.queueDeleted".
>         
>         Every Event Type MUST have an attribute "eventType" which
>         carries the
>         Event Type name as a string. Implementers may define
>         additional required
>         or optional attributes for their Event Types.
>         
>         Event Types do not have operations, annotations or extensions.
>         No
>         standard event types are defined by this specification.
>         
>         ## Notification Messages
>         
>         A notification message contains a single event. The message
>         subject MUST
>         be the Event Type name.  The application-properties MUST
>         include all
>         required attributes for that Event Type, in particular
>         "eventType" with
>         the Event Type name.  They MAY include any optional attributes
>         for the
>         Event Type.
>         
>         ## Subscribing for Event Notification
>         
>         To subscribe for notification messages, a client creates a
>         link from the
>         address "$management.events" with a *filter* (TODO xref)
>         describing the
>         events that it is interested in.  Closing the link ends the
>         subscription.
>         
>         The filtering capabilities depend on the filters provided by
>         the agent.
>         An agent SHOULD provide the APACHE.ORG:SELECTOR described in
>         <http://www.amqp.org/specification/1.0/filters>.  This allows
>         clients to
>         filter on arbitrary SQL-like expressions over all the
>         attributes of the
>         event including "eventType".
>         
>         Since the event type MUST also be included in the message
>         subject, even
>         simple filters (for example the legacy filters described in
>         <http://www.amqp.org/specification/1.0/filters>) MAY be used
>         to filter
>         by event type.
>         
>         Any filter that can operate on the subject or
>         application-properties of
>         a message will be able to work with event notification
>         messages, but for
>         flexibility and interoperability implementors SHOULD provide
>         APACHE.ORG:SELECTOR.
>         
>         ----
>         
>         # Notes, not part of the spec:
>         
>         Needs cross references to relevant sections on filters etc.
>         
>         This design does not allow multiple events to be batched in a
>         single
>         message. IMO an implementation should batch messages at the
>         transport
>         (into TCP packets) NOT batch events into a message. Batching
>         in messages
>         complicates the client, defeats transport level batching (by
>         forcing
>         "fat" messages on the transport) and rules out (or badly
>         complicates)
>         the use of AMQP filters to select events.
>         
>         I considered an alternative design where the client sends a
>         subscription
>         request with a reply-to address and the agent then sends
>         notification
>         messages to the reply-to address.
>         
>         I rejected that design for the following reasons:
>         
>         1. It is impossible in general to tell when such a subscriber
>         goes away
>         if it crashes or fails to send an explicit unsubscribe
>         message.
>         Implementations must already deal with closing links when
>         clients
>         disconnect or heartbeat out so using links solves this
>         problem. Even in
>         an indirect topology (like link-routed Qpid dispatch networks)
>         the
>         routers must proxy link closure to be AMQP-correct.
>         
>         2. We should use AMQP standard features to solve AMQP
>         problems, not
>         re-invent the wheel in a form only useful for management.
>         Filters/selectors were designed for exactly this purpose we
>         should make
>         them work for us.
>         
>         
>         
>         
>         ---------------------------------------------------------------------
>         To unsubscribe from this mail list, you must leave the OASIS
>         TC that
>         generates this mail.  Follow this link to all your TCs in
>         OASIS at:
>         https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>         
> 
>