Your interpretation of the effect of a PUBLISH which includes a Message Expiry Interval (which again I think is a reasonable interpretation) has two behaviours:
- How long a message enqueued towards a selected subscriber will live without progress in delivery towards that subscriber.
- How long a retained message will be retained.
I think that the existing text in 3.3.2.3.3 is clear enough as it stands to cover off behaviour #1, although there is nothing wrong with what you have proposed as an alternative. What concerns me
is that behaviour #2 is not well defined, indeed not discussed at all in the description of this PUBLISH property. If this property causes both behaviours, then when the property is being described both behaviours should be mentioned at least. Instead what
we have is an earlier statement discussing Retain which suggests behaviour #2, saying âIf the current retained message for a Topic expires, it is discardedââ. But based on the statement in 3.3.2.3.3 messages donât expire âfor a topicâ, a copy of them
expire âfor that subscriberâ.
Really I am most concerned that behaviour #2, and the statement in 3.3.1.3 relating to Retain, may have been made in error. The fact that behaviour #2 is not included in 3.3.2.3.3, and that behaviour #2 is not included as a mandatory normative
statement as behaviour #1 is, suggests to me that this was an oversight, or added in haste, or some concept which was introduced into the spec and then only half removed. I sat in on many of the working group reviews but I honestly cannot recall what the specific
intent and history was.
That being said, Iâm sure the real use-case for this feature (which could be included as a non-normative statement) is that the publisher knows that its message has no value to consumers after a certain period of time since it is published.
With that use case in mind, it makes sense that both behaviours #1 and #2 would exist. Even then, for this use-case a retained message which is delivered to a newly subscribing client should have the expiry time for its copy of the message decremented by the
time spent since it was retainedâ and that is certainly not defined in the spec. Which again to my mind brings into question the value of behaviour #2.
Anyways, I guess Iâm just suggesting that some future spin of the spec considers adding the following to 3.3.2.3.3:
- a clear statement about behaviour #1 (already there) and #2 (missing)
- a non-normative statement about the utility of message expiry, whatever that is
- the inclusion of #2 as a mandatory normative statement in Appendix B
From: Roger Light <roger@atchoo.org>
Sent: Thursday, June 4, 2020 5:47 PM
To: David Horton <David.Horton@solace.com>
Subject: Re: [mqtt-comment] Contradictory statements on Message Expiry and Retained messages
Hi David,
3.3.2.3.3 mentions "the Server has not managed to start onward
delivery to a matching subscriber", and "it MUST delete the copy of
the message for that subscriber" (emphasis on "copy"), which is why I
believe it applies to a particular client with a queue of outstanding
messages. It has the messages ready to go, but can't deliver them yet
for some reason.
How about:
If a Server has a message queued to deliver to a client, and if the
Message Expiry Interval for that message has passed and the Server has
not managed to start onward delivery the subscriber, then it MUST
delete the copy of the message for that subscriber. If the Server is
already part way through the publish protocol flow then it must
complete the flow for the give QoS level even if the Message Expiry
Interval has passed.
Regards,
Roger
On Thu, 4 Jun 2020 at 22:25, David Horton <David.Horton@solace.com> wrote:
>
> That is certainly a reasonable behaviour, but 3.3.2.3.3 isnât specifically about the case of messages being queued to being delivered to a particular client. It is defining what the meaning of the Message Expiry Interval is in a PUBLISH message, which may
be queued for delivery to a client immediately, or retained for future delivery to a client, or both. If what you describe is the intended Server behaviour then the description of the Message Expiry Interval property of a PUBLISH packet should be clearer as
to what behaviour is expected, both when that message is immediately matched to a current client subscription, and when it is retained for future delivery.
>
>
>
> I note that the statement in 3.3.2.3.3 is captured as a mandatory normative statement (MQTT-3.3.2-5), whereas the statement in
3.3.1.3 relating to Retain is not.
>
>
>
> Again, I do no dispute that the behaviours you suggest are not reasonable and useful, and probably what was intended, but I do feel that the spec is weak in clearly defining those behaviours for a Server implementation.
>
>
>
> From: Roger Light <roger@atchoo.org>
> Sent: Thursday, June 4, 2020 4:51 PM
> To: David Horton <David.Horton@solace.com>
> Subject: Re: [mqtt-comment] Contradictory statements on Message Expiry and Retained messages
>
>
>
> Hi David,
>
> My understanding is that those statements cover entirely different situations.
>
> 3.3.2.3.3 is talking about messages that are queued to be delivered to
> a client. So you may have a client that connects with a large session
> expiry interval, makes a QoS>0 subscription, then disconnects. If
> messages are published that match the client subscription they can be
> held in a queue for that client for when it reconnects. If the message
> has expiry set on it, then the message may expire before the client
> reconnects. Whether that message was set to be retained is of no
> bearing here.
>
>
3.3.1.3 is talking about a message that was set as retained, and will
> hence be sent to any clients that subscribe to the topic in question.
> Once the message expires, it would be deleted. If a new retained
> message comes along for the same topic that also has a message expiry,
> then the old message will be removed as normal, and the new message
> stored as that retained message with a new expiry time.
>
> Message expiry should apply to retained messages exactly so it is easy
> for a publisher to control when a message is too old without having to
> publish empty messages to the topic.
>
> Regards,
>
> Roger
>
> On Thu, 4 Jun 2020 at 21:30, David Horton <David.Horton@solace.com> wrote:
> >
> > The MQTT 5.0 specs has the following 2 statements, which I feel are contradictory or at least misleading:
> >
> >
> >
> > In
3.3.1.3 Retain:
> >
> > If the current retained message for a Topic expires, it is discarded and there will be no retained message for that topic.
> >
> > In 3.3.2.3.3 Message Expiry Interval:
> >
> > If the Message Expiry Interval has passed and the Server has not managed to start onward delivery to a matching subscriber, then it MUST delete the copy of the message for that subscriber.
> >
> > The implication from
3.3.1.3 is that Message Expiry applies to retained messages. Yet in 3.3.2.3.3 suggests messages expire only when they have been selected for delivery to a certain individual subscriber. How then can a retained message, which is held outside of the state
of any particular session, expire?
> >
> > I can see the possible value in retained messages expiring when they become too old, but why would this be tied to their having been delivered to any particular Client? Does this mean that a retained messageâs expiry timer gets restarted every time it is
delivered to a new Client? That doesnât make much sense. Surely it is up to the publisher of the retained message to refresh it periodically if that is what is desired, or remove it when it is too old by sending an empty message to the same topic.
> >
> > I feel the statement in
3.3.1.3 is simply in error, and retained messages should not be exposed to Message Expiry.
> >
> >