Lily Liu wrote:
I
like your taxonomy that sorts the issues
out nicely.
The
only items I would add to "Determined by Subscriber" category are subscription hints
(such as getting messages by publisher order, or by priority), and
durable subscribers.
I lumped durable subscribers under QoS/Durability in the face of
process failures, but it might be worth calling out specifically. The
subscription hints are very worthwhile additions (and exactly the sort
of thing I'd want to handle via an extension point).
I still have a different
view on "1-of-N delivery" (Issue WSN 2.26) . To me, message queue is not a
special case of pub/sub with only one consumer involved. Message
queue is a different delivery pattern that indicates messages are sent through a single pipe. These
messages can be picked up by a single consumer or multiple consumers.
Multiple consumers of a message queue may get the messages from the
queue by a first come first
served manner, or by a more complicated algorithm. Message queue
enables shared state consumers, load balance, message priority, and other queuing
functions. These are common messaging features
that should be surfaced through the WSN interface.
Funny, that sounds almost exactly like my view of the situation :-).
I think the key here is that there are two equally valid views of
"consumer". As indicated by the phrase "single pipe", at some
point, messages are getting put into a single conduit. On the other
hand, they are getting pulled out of the conduit by multiple parties.
Either the single pipe or the multiple parties (or both) could be
termed "consumers".
The relevant technical question is how to specify this when making a
subscription. I would like to do this in a composable way, so that
notifications could be sent, say
- To consumer A, and also to
- One of
- Consumer B
- Consumer C, and also one of
In other words
- A gets every notification
- B and C together get every notification (and no message goes to
both)
- D and E together get every notification C gets (and no message
goes to both)
This sort of thing is easy to put together with a 1-of-N fanout module,
and viewed this way it's easy to change the picture dynamically. You
could also describe it in terms of dependencies (the in other words
part), but I'm not sure that that will handle dynamic changes as
gracefully.
Also, it seems that such matters of plumbing have nothing at all to do
with topics. The ability to split a sequence of messages among several
recipients seems useful in itself.
One theme I've been harping on lately is the need to separate the
concerns of what data to send from the mechanics of how that data gets
delivered. While I still believe that some form of this distinction
will be essential, I would like to step back a bit and try to get a
broad look at all the various parameters to be decided when
establishing a subscription, whether they are currently written into
WSN or WSE, or are currently considered negotiable by policy, or are
not yet clearly classified at all.
It occurs to me that one criterion for deciding how to treat these
parameters is which party or parties have a say in them. For example,
the NotificationProducer determines what notifications it can possibly
support, while the Subscriber decides what part of that universe it's
actually interested in.
One reason that delivery mechanisms appear to behave differently is
that more than one party may have a say in them. A Consumer may prefer
certain ways of getting data, but the NotificationProducer may not
support the Consumer's favorite means. The Consumer cannot dictate the
means of delivery because it doesn't know what the NotificationProducer
can support. The NotificationProducer could dictate the means of
delivery, but shouldn't, since it doesn't know the Consumer's
preferences. Instead, we need some means of conflict resolution,
whether negotiation, a resolution algorithm, or some combination of the
two. In simpler cases, for example what set of topics does a producer
support, it's enough for the owning party to be able to advertise and
for other parties to be able to make queries.
So here is a preliminary list of variable parameters for a given
notification deployment, with notes as to who the stakeholders are.
I've tried to incorporate the policy issues we've unearthed from the
issues list, along with anything else of interest, but I've almost
certainly missed important items. I'm hoping that presenting a
taxonomy like this will help flush out missing items, and also provide
a place to hang the ones that turn up for whatever reason. I apologize
if this is a little cryptic. Many of these issues have long
discussions and quite a bit of context behind them.
- The universe of possible notifications. Determined by the
NotificationProducer.
- What data to send. Determined by Subscriber
- Topic filtering
- Precondition filtering
- Selector filtering
- Submessage selection (Issue 2.30 from WSDM)
- How to deliver the data. Negotiated by producer and
consumer.
- Pull vs. push delivery (Issue 1.4)
- QoS
- Delivery guarantees
- Queuing and replaying
- Durability in the face of process failures
- Continuity in the face of subscription modifications
(Issue 2.28)
- Security (See note 1 below)
- DDOS mitigation (e.g., double opt-in) (Issue 2.6)
- 1-of-N delivery (Issue 2.26. See note 2 below)
- queuing and replay (Issue 2.27)
- boxcarring of messages
- envelope data (subscription ID, etc. See note 3 below).
- <>message format (Issue 2.13, 3.2. See note 4
below).
- Administration. NotificationProducer has a say. Who else?
- Means of garbage collection (Issue 2.18: scheduled
termination vs. other means).
- Means of notification of subscription life events (See
note 5 below)
- Security of administration (See note 1 below)
- Uncategorized
- GetCurrentMessage vs. Initial value/update vs. stateless
(See note 6 below)
- Authorization to make subscriptions on behalf of a
Consumer.
Notes:
- In the delivery context, security refers to the usual
concerns of encryption, signatures and so forth. There are also
separate issues such as who may make a subscription on whose behalf and
who may delete, pause, resume or modify a subscription, or who may
receive notice of subscription life events.
- I would provisionally regard 1-of-N delivery as delivery to a
single consumer representing the message queue, with the actual
recipients attached to the queue without the NotificationProducer
knowing anything about them, but I realize this issue is still open.
- It is currently configurable whether a Notify message
contains a subscription reference, and even whether Notify is used for
delivery at all.
- This is drawn from the issues list. I'm not entirely sure
what level the issue is at, but it may have to do with choosing between
transport-level options -- as a hypothetical example, whether messages
are to be compressed by some means. Boxcarring is at least a related
issue.
- WSE allows for an explicit callback address for subscription
death. WSN treats this as a special case of notification of a resource
change. I've chosen "subscription life events" as one might be
interested not only in subscription death but also modifications. Some
systems also support notification of subscription creation, that is,
notifications of the form "tell me when anyone creates a subscription
on this topic".
- A while ago I wrote about GetCurrentMessage being a special
case of "get the current state of the resource," with the understanding
that notifications may reflect only state changes and not the entire
state. For example, it may be possible to query a large dataset and
then be notified of presumably much smaller changes to that dataset.
It would be good to be able to advertise this sort of semantic.
|