Patil, Sanjay wrote:
Message
Hi David,
Thanks for your specific example. It kind
of helps me understand your problem, but is also raises another
interesting question -- should the Subscriber really know so much about
the relationships between the various brokers on the provider side?
In no case would the Subscriber know about the relationships between
the brokers (if any). That's currently hidden behind the
NotificationProducer. The point is that if we specify that the
destination of a subscription can change atomically on the fly, then
the specification requires too much coordination in a brokered
implementation (and in other implementations).
Wouldn't it be a simpler model , where the
Subscriber is aware of a single SubscriptionManager that can handle all
of its requests for Subscription updates.
I agree. I also think that's not too far off from what we've got now,
but that's a different issue.
Now the SubscriptionManager itself may be
aware of the different brokers and their interrelationships and may
indeed choose to effect the Subscription update by deleting the
Subscription with one broker and creating a new Subscription with
another broker and also in an atomic manner somehow.
Please be more specific about that "atomic manner somehow." I can't
see any way to do this without either the brokers knowing about each
other's traffic flow, or some external entity (the SubscriptionManger
or whatever) knowing about it. In any case, this is too much for
pieces of a distributed architecture to know about each other.
However, in this case it will be an
implementation specific issue.
The issue here is not whether we can hide the implementation. We can.
The issue is whether the specified behavior of being able to atomically
redirect a subscription is compatible with a distributed brokering
implementation. As far as I can tell it isn't -- at least not without
introducing quite a bit of additional machinery.
Basically,
I still think that the current Subscription update semantics may be
sufficient at the specification level.
Thanks,
Sanjay
Patil, Sanjay wrote:
Hi David, perhaps it's just me, but I
still do not understand the issue here.
Specifically --
a> how the capability to modify
Subscriptions relates to where the filtering and routnigs are done, and
Suppose distribution is through a tree of brokers. Each tree node
knows what topics its children are interested in, and subscribes to
those topics from its parent, and so on up to the original producer.
Suppose consumers A and B belong to broker 1 and consumer C belongs to
broker 2. Both A and B are subscribed to topic foo, but nothing
belonging to broker 2 is.
Now change the subscription for consumer A to point to consumer C. We
can't just move broker 1's foo subscription atomically to broker 2,
because consumer B still needs foo. So broker 2 has to open a new
subscription (or add foo to the list of topics on its existing one).
Then broker 1 and broker 2 have to coordinate to be sure that any foo
message both get goes to only one of consumer A and consumer C. Just
for fun, suppose that consumer B's subscription is canceled while this
is happening. Broker 1 still can't let go of the subscription until it
knows that broker 2's subscription is established.
When a tree node has to know about nodes other than its parent and
children, it's no longer in a tree.
Note that these issues arise from changing the destination of a
subscription. Changing the filtering or policy for an existing
subscription, without changing the consumer, may not be a major issue.
b> how making Subscriptions immutable
solves the problem (which I hope to understand once you answer a>)
It doesn't so much solve the problem as put it out of scope and make it
explicit that duplicate or possibly even dropped messages are a
possibility.
Thanks,
Sanjay
Patil, Sanjay wrote:
I guess modifying an existing Subscription
makes life simpler by guaranteeing atomicity, and by avoiding creating
entirely new subscription when you really wanted to make a
certain small change.
(Mozilla just crashed on me, so I believe I lost a previous reply.
Here's another attempt).
The changes in question are small if you take the view that all
filtering and routing is done by the producer. But, despite some
wording in the introduction, we want to allow the producer to delegate,
either to a broker, or to a traditional system, or perhaps by some
other means. In this case, some or all of the filtering and routing
may be done in a distributed fashion.
This may or may not actually present practical problems. We'd need to
think it through. Taking the approach that subscription properties are
fixed over the lifetime of a subscription gets around any such
problems, at the expense of exposing simultaneity issues to the outside
world.
I'm not (yet) taking a side on this. At this point I'm just trying to
highlight the issues.
The concerns you raised below seem to be
relevant only if one were to take the alternative approach of canceling
a Subscription and creating a new one , which does not seem to be
necessary. Perhaps I am missing something!
On a side note -- If
we had a property such as EffectiveFrom on the Subscription, it would
have been possible to send two separate messages for Subscribe
and Cancel (actually SetTerminationTime) and still achieve atomicity
(assuming that there is sufficient time window available for the
Producer to receive and process both Subscribe and Cancel messages
before the new subscription becomes effective). I am not suggesting
that we need to work on such an alternative protocol for modifying
Subscriptions. It just struck to me as a generally valid use case for
having a property such as - EffectiveFrom.
Thanks,
Sanjay
As far as I can tell, and correct me if I'm wrong, the main reason to
allow a subscription to be modified, as opposed to closing it out and
creating a new one, is atomicity. The clearest example of this I can
see is useNotify (though I'm at a loss for a use case). If I change
useNotify, then I know that I will receive every message produced
exactly once, and at some unspecified time in the future the consumer
will start seeing messages in the other format. On the other hand, if
I first create a new subscription and then delete the old, the consumer
may get some set of messages twice, once in the old format and once in
the new.
We need to be careful about simultaneity here. About the only things
we can count are that a message arrives after it is sent, and that for
a given connection, messages arrive in the order they are sent. But if
two different sources send to the same sink, the exact interleaving of
arrivals is undefined. In this context I am further assuming that if
the NotificationProducer receives (Subscribe new, Cancel old), any
notifications produced between these events will go to both
subscribers. But this is not certain, and we need to nail this down.
Now let's consider the various properties that can be set:
- ConsumerReference: Atomicity means that every
notification sent (assuming no other properties are changed) will go to
either the new or the old consumer, but not both.
- TopicExpression: Every notification sent to the
intersection of the two topic sets will arrive, regardless of when the
switch actually takes effect. I don't believe there are any guarantees
for topics outside the intersection.
- UseNotify: See above.
- Precondition/Selector. Again, atomicity only matters
for the intersection.
- SubscriptionPolicy: This may depend on the actual
policies involved, but the situation should be essentially similar to
useNotify.
So there are at least two questions here: Can or should we require any
guarantees for the case where the same producer hears a new
subscription creation and then an old one canceled? In other words,
does arrival time at the producer matter in the context of
notifications sent to a consumer?
If so, how important are the atomicity guarantees for setting
properties? The worst side effect seems to be the possibility of
duplicate messages. And if guaranteeing ordering of
subscriptions/cancellations is not feasible, how feasible are the
atomicity guarantees for setting properties?
My gut feeling right now is that significantly changing a
subscription's properties in-flight is tantamount to canceling it and
creating a new one, and it would be nice to reflect that without losing
useful atomicity properties. In any case, the key semantic to look at
is the ordering of events in an inherently loosely-ordered system.
Hope that makes sense.
|