OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

mqtt message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: [OASIS Issue Tracker] Commented: (MQTT-58) How many messages are received for overlapping subscriptions?

    [ http://tools.oasis-open.org/issues/browse/MQTT-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=34228#action_34228 ] 

Raphael Cohen commented on MQTT-58:


Trying to think about this from an implementor's perspective, as filters are quite an expensive thing to implement.* I'd also look to see what happens in other protocols (AMQP).

1) Implementor Perspective
I'm assuming a model that uses a dispatch queue. Of course, there's lots of other ways to do things, too, but, ultimately, one wants to match a queue of messages - whether that's global or per topic is sort of moot.

What I'd want to do is avoid any work that required:-

- parsing and re-parsing a list of filter expressions to find (a) where to put the new filter expression with two subscribes; (b) normalise the expressions
   - this is so one can just append data to a buffer or list, rather than having to insert into a memory structure, memmove, etc.
- filter evaluation that wasn't just 'iterate over list', match first (i.e. a logical OR)

- now if we need to have 'later expression wins', it just requires iterating in last to first order for the filter expression. I can live with that, but I'd still worry about efficiently managing normalisation so the list didn't grow for ever! Taking part of 1. / 2. above, we could allow replacement for exact matches of topic. That would give last one wins. Where multiple expressions match, last used expression matches.

- multiple copies. Logically, yes, I see use cases for it. In reality, please, G-d, no. Just run multiple connections. Creating apparently more frequent at-least-once delivery and 'apparent' violation of exactly-once, thereby forcing all clients to deal idempotently with messages (and idempotence is something that mid-level devs really struggle with in system implementations) is a great way to make MQTT hard to use. In other protocols, there's the concept of links (a sort of filtered multiplexing), so multiple message delivery w/o separate connections is possible. But it is complex to develop with - most JMS users and their ilk have one consumer for one connection, and keep it as simple as that.

2) AMQP et al
There's the concept of a filter set, which is a logical OR of filters, in the order specified. If the subscription is modified, then the filter-set is re-applied (a link detach-reattach is usually what happens). If one was mapping MQTT to AMQP, then the topic would probably by the AMQP Subject and QoS a delivery annotation or message annotation. Matches would be 'first filter wins'.

3) Summary
Given all this, I like the last sentence suggestion "But even this is...". I'd propose what I think is similar, if not the same, but as a set of rules that apply equally to multiple SUBSCRIBEs as well as repetitions within SUBSCRIBEs:-

- If a later topic string selector EXACTLY matches (wildcards and all, i.e. an exact UTF-8 byte match), then its QoS replaces that used for that selector. If it's higher, it goes higher. It it's lower, it goes lower. If it stays the same - well, it's the Bruce's Play Your Cards Right rules. A variant not yet used on Mornigton Crecent.
- If it doesn't, it gets appended to the bottom of the list
- The list of selectors is resolved as a logical OR, last-to-first

- With an UNSUBSCRIBE, the same exact matching applies. Which means clients couldn't unsubscribe if a QoS was 'mutated' using the above rules. Then again, they've mutated the QoS themselves in any event. In essence, we're applying slightly different filter add / remove logic for topic selectors as opposed to topic QoS. Personally, I don't like this, but we have what we have (I'd have left QoS out of it, and told implementors to just use a different topic for a different QoS need. Lots of topics / queues is always a good architecture in my book as makes clients super-simple. I digress).

Whilst there's substantial merit to all the other approaches, it'd be a complete nightmare to implement some of them - and complex normalisation just slows everything down to a crawl. One day people might be running embedded MQTT brokers locally far more than they do now...

* The difference between 1m messages / second, and 10,000 messages / second.

> How many messages are received for overlapping subscriptions?
> -------------------------------------------------------------
>                 Key: MQTT-58
>                 URL: http://tools.oasis-open.org/issues/browse/MQTT-58
>             Project: OASIS Message Queuing Telemetry Transport (MQTT) TC
>          Issue Type: Improvement
>          Components: core
>            Reporter: Allan Stockdill-Mander 
> A client has subscribed to a/+ with QoS2 and a/b with QoS1. A message is published to a/b. Does the client receive one or two messages? If one which QoS would apply?

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://tools.oasis-open.org/issues/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]