Re: Comments on the AMQP management spec

OK - so Fraser beat me to it in terms of responding, and said many of the things I was going to say :-)

On 11 April 2014 10:35, Fraser Adams <fraser.adams@blueyonder.co.uk> wrote:

Hi Justin & Rob
I've replied to this on the TC comment list, but for completeness and transparency here's what I said:

RE:

# Application properties and studlyCaps

The spec calls for studlyCaps for standard property names, whereas an earlier draft had hyphenated names. FWIW, I prefer the latter, since it’s in line with the core AMQP spec.

hyphens are an interesting case as I've found to my cost . You might have seen my post to the qpid user list back in February entitled "A write up of some AMQP 1.0 Experiments" during those experiments I discovered some interesting behaviours with hyphenated names and message selectors:

In precis whilst hyphens are indeed legal AMQP 1.0 the *are not* legal wrt. JMS message selectors, indeed they are not *actually* legal JMS application property names (I only realised that myself when I was trying to figure out the selector thing). It turns out that JMS properties have to have names that are valid Java identifiers.

Now technically that might be a moot point from the perspective of an AMQP Specification - after all if it's legal AMQP.... but there's no point making things hard for Java clients and it also doesn't seem great to have one AMQP Specification specifying a form of properties that would be illegal somewhere else:
https://svn.apache.org/repos/asf/qpid/trunk/qpid/specs/apache-filters.xml#section-legacy-amqp
Explaining the reason for the choice might be worthwhile though.

Yep - that's one of the primary reasons for using camel case rather than hyphens (hyphens all make bad for identifiers in code as well).

On a related note previous working drafts of AMQP 1.0 Management had specified using lists as message bodies in quite a few places, which is perfectly legal AMQP (though as you have noted lists and maps as property values is not) but because lists are a pain for JMS I argued that was *probably* a bad idea. I think that you do more python things than Java don't you? For python, C++ and pretty much everything else lists are fine, but for Java aaaarrrgggg!!

Re: # Ping
That's an interesting one, something like that would be useful, but I personally don't think a simple ping to a $management node scales, so that would be a very restrictive solution in a large topology of containers or where there are lots of clients. I included the following in an email to Rob but it was quite a long exchange and it may have been a bit TL;DR :-) but here goes.....

One other thing that I keep forgetting to mention; in QMF2 Agents broadcast heartbeat messages periodically to a topic so clients can identify if Agents have gone away. I guess that this is perhaps more useful in QMF2 where Agents aren't *necessarily* co-located with the Broker, but it has got me thinking. In AMQP 1.0 Management I guess that the $management node is likely to be pretty coupled to the Container's lifecycle such that a client could probably infer that the $management node is gone based on interception of a connection failure (e.g. via something like a Connection Exception Listener) but a couple of use cases fall out of that sort of scope:
1) What about Management nodes other than $management? It's possible that the Container and $management might be available, but some other Management node a user is interested in might possibly die asynchronously for whatever reason (a simple use case might be one client DEREGISTERing a Management Node that another client happens to be using).
2) What about Message-oriented (as opposed to Connection-oriented) APIs such as Messenger, with those the fact of Connection closing wouldn't necessarily register until the next time a request was made.

Clearly a Management client could figure this out by some sort of polling - and that might be better than needing a topic to broadcast a heartbeat (though it doesn't scale as well if there are lots of clients) but if polling is the expected way to accomplish this then I'd say that the Management Specification needs to specify a standard Management node operation to achieve this (perhaps STATUS ?). For someone designing a decoupled client (say a WebApp) being able to determine the status of Management nodes and provide feedback to the user is pretty useful.

It's something to consider - I'm not sure that status is anything different than trying to do a READ of "self" though. We could be more explicit in saying that the READ operation must always be permitted on "self"

Some more comments on Justin's original e-mail below...

Regards,
Frase

On 11/04/14 00:07, Rob Godfrey wrote:

Hi Justin,

can you send these comments to the public TC comment list (see [1]) so they
can be incorporated in the discussions there on any updates to the spec (in
your case - since you are employed by an organisation who is a member of
OASIS and have voting members on the TC - this may not be strictly
required... however it seems like good form :-) ).

I'll try to respond on the technical points (later) in the morning.

-- Rob

[1] https://www.oasis-open.org/committees/comments/index.php?wg_abbrev=amqp

On 11 April 2014 00:20, Justin Ross <jross@apache.org> wrote:

Hi, everyone. I've been using the draft management spec to develop some
command line tools[1], and I've got some questions and comments. These
comments are based on working draft 8 of the spec[2].

Thanks,
Justin

[1] https://github.com/ssorj/bailey (Watch out! It's a work in progress.)
[2]

https://www.oasis-open.org/committees/document.php?document_id=52425&wg_abbrev=amqp

---

# Use of AMQP application-properties

Given that values in AMQP 1.0 application properties cannot be lists or
maps (yikes--I didn't know that), I feel it's inadvisable to use
application properties to define requests. We'd be better off using
properties in the message body. Otherwise, we'll have to define standard
string encodings in every instance where we want to use a non-scalar type.

The application properties are used only to provide the highest level information about the operation. If complex data is needed it can be passed in the body. Burying all the data in the body makes it harder for management nodes that are actually managing a distributed network to effectively route actions based on the entity/operation they are applying to.

# Application properties and studlyCaps

The spec calls for studlyCaps for standard property names, whereas an
earlier draft had hyphenated names. FWIW, I prefer the latter, since it's
in line with the core AMQP spec.

# QUERY and pagination

There doesn't appear to be a way to query for the total count without
fetching all the values. Without that, you can't build a data efficient
page-navigation UI that offers links to each page (or just the last page).

Requests using offset and count should perhaps instead use offset and
"limit". That's more familiar to those who have used SQL, and it correctly
signals that the number returned may be less than the number requested. In
my view, the request should use "limit", and the response should use
"count".

The pattern we've generally used is to echo back the parameters sent in... I'm not personally to fussed either way, but I'm not sure it is a huge barrier for use. The total issue is interesting - I'd be OK optionally supplying a total, but it may be that for some implementations it is not possible / efficient to count all the records that might possibly be retrieved... in which case it might be better to have "Page 2 of lots" style ...

Also, if we're doing paging we need to do sorting as well. Otherwise, A
client UI cannot build a sortable paginated table without pulling down
everything.

That's just moving the burden onto the management node to first collate and then sort the data for every request. For management nodes which are returning large data sets I think there's a good probability that the node in question is actually collating data from a larger network, in which case large data transfers still have to take place... moreover we'd then need to define a mechanism for describing sorting on different fields and how to order complex types, etc.

# Request-response

There's a note in section 4 that multiple response messages may be produced
for a single request. How should a client determine whether the response
is complete?

Hmmm... that seems odd - I think multiple responses should be the preserve of subscription style patterns (not yet defined).

# GET-MGMT-NODES

The text says the response message must contain "a list of addresses of
other Management Nodes". Perhaps this should more explicitly prohibit
listing the currently-in-communication management node (or drop the word
"other").

Agreed

# [DE]REGISTER

What is the intended use of this? That is, what new behavior do you get
when something is registered?

It allows management nodes to be registered into the network and thus discovered. The mythical AMQP toaster example... the toaster joins the network and tells the intermediary it connected to "hey! there's a new management node at /amqptoaster/$management" ... Now anyone connecting to the intermediary can discover there is a toaster there ready to be managed (as can anyone who connected to another part of the network that was aware of the management node of the intermediary, etc).

# Ping

I'd like to see a standard PING or ECHO operation on $management. I can
simulate it by using an arbitrary operation, but that seems less than
straightforward for implementors.

# Clerical stuff

3.3.3.1: Awkward phrasing: "so if any of the changes cannot be applied,
the
entire operation should not be applied and to multiple values changed this
MUST result in a failure response"
3.4.1: Typo: "this operation supports pagination <though> which a request"
3.4.1: Awkward phrasing: "A result set of size N <can be considered to
containing elements>"

Thx - we'll address those.

# Odds and ends

- 2.3: Should this reference the AMQP definition of a "node"?

Yes it probably should

- 3.1: It would be nice to mention that locales is a comma-separated list.

The RFC mentions it, but you have to dig down a bit.

Agreed - we should add explanatory text / an example

- 3.3.1.2: What is a good example of a generic value made specific in an
UPDATE response?

3.3.1.2 is for CREATE where the example is you ask for an entity of type org.amqp.Queue and the object you get back is an org.apache.qpid.QpidQueue

For update you may get things that differ from the request in the sense that maybe you decided to add alerting to the queue but didn't specify a frequency limit, so you get a frequency limit defaulted... I'm not sure on whether there is a case where you get a more specific value - but I wouldn't rule it out ... maybe updating the type?

- 3.3.3: Is the lack of attribute append a problem? Right now, there is
only replacing list values, and dueling actors could mean lost state.

I don't think append per se is a problem, but the more general problem is that the spec provides no way of ensuring that the object was in the expected state when the updates were applied. One would need to be able to add some sort of pre-condition to verify that the object had not changed from the state the updater believed it to be in...

-- Rob

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

amqp-comment message