Hi
David,
Thanks
for your specific example. It kind of helps me
understand your problem, but is also raises another interesting question --
should the Subscriber really know so much about the relationships between the
various brokers on the provider side?
Wouldn't it be a simpler model , where the Subscriber is aware
of a single SubscriptionManager that can handle all of
its requests for Subscription updates. Now the SubscriptionManager itself
may be aware of the different brokers and their interrelationships and may
indeed choose to affect the Subscription update by deleting the Subscription
with one broker and creating a new Subscription with another broker and
also in an atomic manner somehow. However, in this case it will be an
implementation specific issue.
Basically, I still think that the current Subscription update semantics
may be sufficient at the specification level.
Thanks,
Sanjay
Patil, Sanjay wrote:
Hi
David, perhaps it's just me, but I still do not understand the
issue here.
Specifically --
a> how the capability to modify Subscriptions relates to
where the filtering and routnigs are done,
and Suppose distribution is through a tree of
brokers. Each tree node knows what topics its children are
interested in, and subscribes to those topics from its parent, and so on up to
the original producer. Suppose consumers A and B belong to broker 1 and
consumer C belongs to broker 2. Both A and B are subscribed to topic foo, but
nothing belonging to broker 2 is.
Now change the subscription for
consumer A to point to consumer C. We can't just move broker 1's foo
subscription atomically to broker 2, because consumer B still needs foo.
So broker 2 has to open a new subscription (or add foo to the list of topics
on its existing one). Then broker 1 and broker 2 have to coordinate to
be sure that any foo message both get goes to only one of consumer A and
consumer C. Just for fun, suppose that consumer B's subscription is
canceled while this is happening. Broker 1 still can't let go of the
subscription until it knows that broker 2's subscription is
established.
When a tree node has to know about nodes other than its
parent and children, it's no longer in a tree.
Note that these issues
arise from changing the destination of a subscription. Changing the
filtering or policy for an existing subscription, without changing the
consumer, may not be a major issue.
b> how making Subscriptions immutable solves the problem
(which I hope to understand once you answer
a>) It doesn't so much solve the problem as
put it out of scope and make it explicit that duplicate or possibly even
dropped messages are a possibility.
Thanks,
Sanjay
Patil, Sanjay wrote:
I guess modifying an existing Subscription makes life
simpler by guaranteeing atomicity, and by avoiding creating
entirely new subscription when you really wanted to make a
certain small change. (Mozilla just
crashed on me, so I believe I lost a previous reply. Here's another
attempt).
The changes in question are small if you take the view
that all filtering and routing is done by the producer. But, despite
some wording in the introduction, we want to allow the producer to
delegate, either to a broker, or to a traditional system, or perhaps by
some other means. In this case, some or all of the filtering and
routing may be done in a distributed fashion.
This may or may not
actually present practical problems. We'd need to think it
through. Taking the approach that subscription properties are fixed
over the lifetime of a subscription gets around any such problems, at the
expense of exposing simultaneity issues to the outside world.
I'm
not (yet) taking a side on this. At this point I'm just trying to
highlight the issues.
The concerns you raised below seem to be relevant only if one
were to take the alternative approach of canceling a Subscription and
creating a new one , which does not seem to be necessary. Perhaps I am
missing something!
On a side note -- If we had a property such as EffectiveFrom on the
Subscription, it would have been possible to send two separate messages
for Subscribe and Cancel (actually SetTerminationTime) and still
achieve atomicity (assuming that there is sufficient time window
available for the Producer to receive and process both Subscribe and
Cancel messages before the new subscription becomes effective). I am not
suggesting that we need to work on such an alternative protocol for
modifying Subscriptions. It just struck to me as a generally valid use
case for having a property such as -
EffectiveFrom.
Thanks,
Sanjay
As far as I can tell, and correct me if
I'm wrong, the main reason to allow a subscription to be modified, as
opposed to closing it out and creating a new one, is atomicity.
The clearest example of this I can see is useNotify (though I'm at a
loss for a use case). If I change useNotify, then I know that I
will receive every message produced exactly once, and at some
unspecified time in the future the consumer will start seeing messages
in the other format. On the other hand, if I first create a new
subscription and then delete the old, the consumer may get some set of
messages twice, once in the old format and once in the new.
We
need to be careful about simultaneity here. About the only
things we can count are that a message arrives after it is sent, and
that for a given connection, messages arrive in the order they are
sent. But if two different sources send to the same sink, the
exact interleaving of arrivals is undefined. In this context I
am further assuming that if the NotificationProducer receives
(Subscribe new, Cancel old), any notifications produced between these
events will go to both subscribers. But this is not certain, and
we need to nail this down.
Now let's consider the various
properties that can be set:
- ConsumerReference: Atomicity means that every notification
sent (assuming no other properties are changed) will go to either
the new or the old consumer, but not both.
- TopicExpression: Every notification sent to the intersection of
the two topic sets will arrive, regardless of when the switch
actually takes effect. I don't believe there are any
guarantees for topics outside the intersection.
- UseNotify: See above.
- Precondition/Selector. Again, atomicity only matters for
the intersection.
- SubscriptionPolicy: This may depend on the actual policies
involved, but the situation should be essentially similar to
useNotify.
So there are at least two questions here:
Can or should we require any guarantees for the case where the same
producer hears a new subscription creation and then an old one
canceled? In other words, does arrival time at the producer
matter in the context of notifications sent to a consumer?
If
so, how important are the atomicity guarantees for setting
properties? The worst side effect seems to be the possibility of
duplicate messages. And if guaranteeing ordering of
subscriptions/cancellations is not feasible, how feasible are the
atomicity guarantees for setting properties?
My gut feeling
right now is that significantly changing a subscription's properties
in-flight is tantamount to canceling it and creating a new one, and it
would be nice to reflect that without losing useful atomicity
properties. In any case, the key semantic to look at is the
ordering of events in an inherently loosely-ordered
system.
Hope that makes
sense.
|