In response to a SUBSCRIBE packet, an MQTT server must (re)deliver all matching Retained messages to the client with the Retained flag set.
If this is a re-subscribe (i.e., a direct match of an existing subscribed Topic Filter), “Any existing retained messages matching the Topic Filter MUST be re-sent, but the flow of publications MUST NOT be interrupted [MQTT-3.8.4-3].” (Starting line 968.)
Moreover, “Where the Topic Filter is not identical to any existing Subscription’s filter, a new Subscription is created and all matching retained messages are sent.” (Starting line 971.)
We can’t find any normative or non-normative reference in the specification about the ordering of these (re)sent retained messages with respect to the subscribing/subscribed message stream.
Some concrete examples:
The key question: Is the retained message flow to SUBSCRIBE-rs considered “out of band” of live message flow? That is, can the server deliver the retained messages arbitrarily interleaved with live message flow, at its discretion? If not, what are the constraints?
If the client connection is Clean, and the client SUBSCRIBE-s to a Topic Filter that matches one or more retained messages (i.e., this is a “new” subscription), are there any constraints on the ordering of the retained message vs. the “live” message flow? For example, is it required or is there an expectation that the retained messages be delivered first?
If the client connection is not Clean, and the client had previously subscribed to the same Topic Filter, the stream of missed or in-flight/unacked messages, including messages that were sent as Retained, will be a part of the session state. We would like to verify that if the client re-SUBSCRIBE-s to the same Filter Topic, thus triggering retained message delivery, it is possible for the client to see the same “application” message multiple times: with Retained NOT SET as per the normal flow, and with Retained SET for the retained portion; all messages will go through their own separate quality of service flows, with unique Packet Identifiers if the Quality of Service is non-zero.
In an extreme edge case where the network connection is highly unreliable, and a client always resubscribes after connecting, is it possible for the same non-zero Quality of Service retained message to be in-flight many many multiple times with unique Packet Identifiers if the client can’t fully ACK them before the network connection drops?
A Retained message with an empty payload is supposed to be delivered “normally” but serves to remove any stored Retained message. If the server has a retained message A for a Topic and it receives a SUBSCRIBE to that Topic while the server receives a retained message B with an empty payload, can the server send to the client message A with the Retained flag set, and message B with no payload (and the Retained flag clear)? What about in the opposite order?
We realize that this may be an opportunity for server implementations to differentiate themselves. However we want to make sure that we are not missing any normative constraints, and that we have the right “mental model” about what our implementation choices can be.
Regards,
/djk David Kemper TIBCO Software Inc. |