Re: [mqtt] Recharter: Shared subscriptions and $share

[Andrew Schofield <andrew_schofield@uk.ibm.com>]

Hi Nick,
If I've understood your comment correctly,
you are suggesting that a topic starting with the prefix "$shared"
would actually treat all subscriptions as shared subscriptions. Unfortunately,
I think that's a really bad idea. It's gone from N publishers->M subscribers
to N publishers->1 subscription->M consumers. That's not pub/sub
- it's just a queue with multiple consumers.

In a publish/subscribe system, ideally
I want the publishers to be totally unaware of the number and availability
of the subscribers. By having this rule for "$shared", if I want
to use workload balancing among a set of consumers, I have to publish on
a topic starting "$shared" and each message published can be
consumed by at most one consumer. I can no longer dream up another purpose
for the messages and add another subscription - it would just end up getting
a small share of the existing shared subscription.

What I would seek to enable is exactly
what you didn't like: I want subscribers that can take all of the messages
to be able to have them all, and subscribers who cannot keep up with the
message rate to be able to share the stream of messages with like-minded
consumers.

So, I think sharing is a property of
the subscription, not the topic. I prefer having a separate share name
which is not part of the topic name.

Thanks,
Andrew

Andrew Schofield                
                     
        
Chief Architect, Hybrid Cloud Messaging
Senior Technical Staff Member
IBM Cloud Application Services

IBM United Kingdom Limited
Mail Point 211
Hursley Park
Winchester
Hampshire
SO21 2JN

Phone ext. 37248357 (External:+44-1962-818357), DE2J22
Internet mail: andrew_schofield@uk.ibm.com



From:      
 Nicholas O'Leary/UK/IBM@IBMGB
To:      
 mqtt@lists.oasis-open.org
Date:      
 08/04/2016 19:46
Subject:    
   [mqtt] Recharter:
Shared subscriptions and $share
Sent by:    
   <mqtt@lists.oasis-open.org>

---

There are two fundamental approaches here.

1. The client chooses whether a subscription is shared or not
2. The ‘shared’ nature is a property of the topic itself


If the client gets to choose, then you need to define what happens if clients
A and B subscribe to topic foo with the shared flag set, but client C subscribes
without the shared flag set. I think trying to support that type of combination
would overly complicate implementations. We could say that in that scenario
client C’s sub is rejected, but it doesn’t feel like the right approach
to me.

I think it is much cleaner for the shared nature to be a pre-existing property
of the topic - in that way no changes are needed in client implementations
at all, as demonstrated by the fact some broker implementations already
support shared subs within the confines of 3.1.1.

The question is then whether it should be done by virtue of a well-known
topic prefix - $shared, or have it as a property on the topic that can
be administratively set on the broker.

The advantage of the latter is that any topic could be set as a shared
topic - and would be within the constraints of our charter to not define
topic spaces. The downside is it requires administrative action to use
the feature - something that isn’t required anywhere else in the protocol.
To deploy a new application that requires a shared sub would now require
co-ordinated administrative action on the broker - that doesn’t feel lightweight
to me and would fail the Just Works test that so much of MQTT benefits
from.


That brings me to the conclusion that the cleanest solution is to define
any topic that is prefixed with $shared as being a shared topic. We shouldn’t
let the current text of the charter, which we’re currently redrafting
anyway, stop us making the right technical choices for the protocol.


Nick O'Leary
IBM Emerging Technology Services

Twitter: @knolleary

IBM United Kingdom Ltd registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants, PO6 3AU


Brian Raymor --- [mqtt] Recharter: Shared
subscriptions and $share ---

From:	"Brian Raymor" <Brian.Raymor@microsoft.com>
To:	mqtt@lists.oasis-open.org
Date:	Mon, 4 Apr 2016 16:25
Subject:	[mqtt] Recharter: Shared subscriptions and $share

---

Forwarding to the mailing
list for broader awareness and discussion.
 
The conversation is related
to this out-of-scope item in the draft charter:
 
·       The
TC will not identify MQTT topics nor prescribe any mechanism or convention
for classification of MQTT topics or
topic spaces.
 
…Brian
 
From: Andrew Banks [mailto:andrew_banks@uk.ibm.com]
Sent: Friday, April 1, 2016 12:35 PM
To: Ken Borgendale <kwb@us.ibm.com>
Cc: Brian Raymor <Brian.Raymor@microsoft.com>
Subject: Re: MQTT shared subs and charter
 
Ken, using the $share topic prefix seems
the more natural way to go to me.

To avoid using the $ prefix we would still have to encode the sharename
 into the subscribe packet, probably using the same metadata encoding
method as for the Publish Metadata. The existence of the sharename should
signal the shared subscription and so remove the need to use one of the
Qos bits. However, the Subscribe packet can carry multiple subscriptions
and SubAck can carry multiple outcomes so that is where it starts to look
unnatural. 

The $ prefix was reserved in the V3.1.1 specification for uses such as
this, and it is being used in practice,  so I think we should alter
the charter to make its use allowed.


Andrew Banks
Telephone (44) 1962 816123 



From:        Ken Borgendale/Austin/IBM
To:        Andrew
Banks/UK/IBM@IBMGB
Date:        31/03/2016
22:18
Subject:        MQTT
shared subs and charter

---

The issue I have with charter and shared subs in the prohibition in the
charter of defining topics.  I would prefer to keep this charter limitation
but it means that using $shared or any other $ topic to indicate shared
subs would not be allowed.  I think it makes much more sense to use
one of the bits in the QoS byte of SUBSCRIBE for this purpose.  This
is especially true if we also implement nolocal in this way, as there needs
to be an explanation of the intersection of shared subs and nolocal.  The
problem is that there is no corresponding bit on UNSUBSCRIBE but perhaps
we should have an options byte per UNSUBSCRIBE and on UNSUBACK.  This
makes them symetical with SUBSCRIBE and allows for a return status on individual
UNSUBSCRIBES. 

On the issue of shared subs, how you specify shared subs is the simple
issue.  Some of the semantics of shared subs are very messy especially
any text around overlapping subs and shared subs. 





Ken Borgendale  --  kwb@us.ibm.com
   1-207-805-6708  1-207-371-8082
Senior Programmer  --  IBM MessageSight - Architect 


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU