Re: [amqp] [management] Proposal for event notifications (initial outline)

[Jakub.Scholz@deutsche-boerse.com]

Hi Alan,

If I understand it correctly, this basically introduces two distinct
implementations of the events - I don't think that's a good idea.  Also,
your compact representation allows every link to define its own list of
attributes. So in theory you can have N links, each with different
attributeNames defined and you will have to generate and publish the events
N times in N different formats.

What about having just one implementation which stands somewhere in the
middle:
1) GET-EVENTS returns the list of attributes for every event type
2) The broadcasted events will have only a list of values in the same order
as was the order of attributes in the GET-EVENTS response - so we save the
attribute names.

This will not be as effective on the wire, but it would allow to have for
every event only one message regardless the number of subscribers.

Thanks & Regards
Jakub

<amqp@lists.oasis-open.org> wrote on 16/02/2015 17:24:10:

> From: Alan Conway <aconway@redhat.com>
> To: "John O'Hara" <john@rjohara.com>,
> Cc: amqp@lists.oasis-open.org, "Arrott, Matt"
> <marrott@ucsd.edu>, "Godfrey, Robert X"
> <robert.godfrey@jpmorgan.com>
> Date: 16/02/2015 17:24
> Subject: Re: [amqp] [management] Proposal for event
> notifications (initial outline)
> Sent by: <amqp@lists.oasis-open.org>
>
>
> I have a couple of thoughts for extending the event notification
> proposal, the original is below for reference. I'll put them all
> together if after any feedback.
>
> *Event meta-data*: Add a GET-EVENTS operation to the
> "self" entity (like
> existing GET-ATTRIBUTES) to get a map of event-type name
> to attribute
> name list.
>
>
> *Compact event representation*: The current proposal is
> very fat on the
> wire - attribute names are repeated in every notification
> and there's no
> way to select a subset of attributes. I suggest the
> following addition
> inspired by the existing QUERY response:
>
> When subscribing for events the link attach properties MAY
> contain the
> key `attributeNames` with a list of attribute names. If so, event
> messages are tested against the filter as before, but are sent in a
> compact form. Attribute names/values are not sent in the application
> properties, instead the body contains a list of attribute
> values in the
> same order as `attributeNames`. Missing attributes are
> represented by
> NULL values.
>
> Without `attributeNames` you get the names and values. This is less
> efficient but may be easier to work with for dynamic or exploratory
> tools that can interact with many different implementations. With
> `attributeNames` you get just what you ask for with no
> overhead. This is
> better for tools that know what they want, and in heavy
> traffic systems
> where the overhead of attribute names is excessive.
>
>
> *Standard event types*: We could probably generate a
> reasonable set of
> standard event types directly from the performatives in
> the AMQP spec.
> Every performative (link-open, connection-open,
> disposition etc.) does
> signify and event that might be of interest to a
> management console. Is
> it too early to think about standardizing this?  Are there
> any pitfalls?
>
>
>
> Original proposal
> -----------------
>
> # AMQP Management Event Notification
>
> Management clients often need to know when the state of a
> managed entity
> has changed.  This can be for monitoring or logging purposes, or to
> react when something becomes ready for use.
>
> Polling the management agent for changes with READ or
> QUERY operations
> is inefficient. Clients either poll too often, which overloads the
> agent, or too seldom which causes needless delays in the system.
>
> Event notification allows management clients to subscribe
> for continuous
> notification of selected events as soon as the agent
> becomes aware of
> them.
>
> ## Event Types
>
> An Event Type is a named set of attributes. The name is a
> case-sensitive
> string. Implementers MAY define their own Event Types which MUST be
> named using a reverse domain name prefix owned by the
> implementer, e.g.
> "com.example.broker.queueDeleted".
>
> Every Event Type MUST have an attribute "eventType" which
> carries the
> Event Type name as a string. Implementers may define
> additional required
> or optional attributes for their Event Types.
>
> Event Types do not have operations, annotations or extensions. No
> standard event types are defined by this specification.
>
> ## Notification Messages
>
> A notification message contains a single event. The
> message subject MUST
> be the Event Type name.  The application-properties MUST include all
> required attributes for that Event Type, in particular
> "eventType" with
> the Event Type name.  They MAY include any optional
> attributes for the
> Event Type.
>
> ## Subscribing for Event Notification
>
> To subscribe for notification messages, a client creates a
> link from the
> address "$management.events" with a *filter* (TODO xref)
> describing the
> events that it is interested in.  Closing the link ends the
> subscription.
>
> The filtering capabilities depend on the filters provided
> by the agent.
> An agent SHOULD provide the APACHE.ORG:SELECTOR described in
> <http://www.amqp.org/specification/1.0/filters>.  This
> allows clients to
> filter on arbitrary SQL-like expressions over all the
> attributes of the
> event including "eventType".
>
> Since the event type MUST also be included in the message
> subject, even
> simple filters (for example the legacy filters described in
> <http://www.amqp.org/specification/1.0/filters>) MAY be
> used to filter
> by event type.
>
> Any filter that can operate on the subject or application-
> properties of
> a message will be able to work with event notification
> messages, but for
> flexibility and interoperability implementers SHOULD provide
> APACHE.ORG:SELECTOR.
>
> ----
>
> # Notes, not part of the spec:
>
> Needs cross references to relevant sections on filters etc.
>
> This design does not allow multiple events to be batched in a single
> message. IMO an implementation should batch messages at
> the transport
> (into TCP packets) NOT batch events into a message.
> Batching in messages
> complicates the client, defeats transport level batching (by forcing
> "fat" messages on the transport) and rules out (or badly
> complicates)
> the use of AMQP filters to select events.
>
> I considered an alternative design where the client sends
> a subscription
> request with a reply-to address and the agent then sends
> notification
> messages to the reply-to address.
>
> I rejected that design for the following reasons:
>
> 1. It is impossible in general to tell when such a
> subscriber goes away
> if it crashes or fails to send an explicit unsubscribe message.
> Implementations must already deal with closing links when clients
> disconnect or heartbeat out so using links solves this
> problem. Even in
> an indirect topology (like link-routed Qpid dispatch networks) the
> routers must proxy link closure to be AMQP-correct.
>
> 2. We should use AMQP standard features to solve AMQP problems, not
> re-invent the wheel in a form only useful for management.
> Filters/selectors were designed for exactly this purpose
> we should make
> them work for us.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the
> OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/
> my_workgroups.php
>


----------------------------------------------------------------------------

Deutsche Börse Services s.r.o.
Managing Directors/Geschäftsführung:
Michael Gassmann, Mats Andersson.
Limited liability company with registered office at
Sokolovská 662/136B, CZ-186 00 Prague 8
recorded in the Commercial Register IC: 275 77 015.
Maintained by the city court in Prague,
Sec. C, File No. 116874.

-----------------------------------------
Diese E-Mail enthaelt vertrauliche oder rechtlich geschuetzte Informationen.
Wenn Sie nicht der beabsichtigte Empfaenger sind, informieren Sie bitte
sofort den Absender und loeschen Sie diese E-Mail. Das unbefugte Kopieren
dieser E-Mail oder die unbefugte Weitergabe der enthaltenen Informationen
ist nicht gestattet.

The information contained in this message is confidential or protected by
law. If you are not the intended recipient, please contact the sender and
delete this message. Any unauthorised copying of this message or
unauthorised distribution of the information contained herein is prohibited.

Legally required information for business correspondence/
Gesetzliche Pflichtangaben fuer Geschaeftskorrespondenz:
http://deutsche-boerse.com/letterhead