Re: implicit prepare - sent on behalf of Savas

[Mark Little <mcl@arjuna.com>]

> I think we've got several interweaving subissues here.

Definitely, and more by the minute!

> 1) the presence of messages or sub-messages in the same packet on the wire
> doesn't necessarily mean they come from the same logical "actor" in the
> sender or will be delivered to a single actor at the recipient.
>
> With any kind of filter/interceptor setup (compare OTS implicit
> propagation), it would seem quite reasonable at least for upcoming Enrolls
> and Votes to get stripped off an application response and delivered to the
> Coordinator.

Here you are assuming that the coordinator and business logic are
co-located. If they are *and* the interceptor knows about both (which it may
not) then this kind of optimisation is possible. However, if they are not
(and at the Mt Laurel face-to-face we agreed that one useful service may
well be this atom-manager/factory) then it's no use for the application
response to "book taxi" to contain information meant for the coordinator.
Obviously the interceptor at the receiving business-logic side could forward
this information on to the coordinator, but that's then 2 hops to deliver
the response, when if the Web Service had sent the message directly to the
coordinator it would have been onle 1.

> By the same token, there is no inherent reason why messages
> from the coordinator could not be sent down with an application request -
> the issue would only be whether the coordinator can know it's appropriate
to
> send the message.

Again you are assuming they are co-located. I do not want to see a protocol
that insists coordinators are co-located with cohesions/business-logic.
That's certainly an optimisation some vendors may take, but it should not be
a necessity to implement BTP. It isn't a necessity in the OTS, for example,
that the transaction manager is co-located with the client.

>
> 2) prepare has heavily overloaded semantics in most transaction protocols.
> Provided the assumptions involved stay the same, this doesn't matter, but
> when we start sawing up things that have previously been united (as with
the
> open-top coordinators), the different aspects of prepare need to be split
> out.
>
> The transit of a prepare message can mean:
> a) there will be no more application data sent (down) to you/your servive
> b) there will be no more application data sent to any service
> c) there can be no more application data received from your service
> (without causing a rollback) d) there can be no more application data
> received from any service (without causing a rollback) e) you are allowed
to
> send heuristics
> f) you are allowed to send a vote
> g) you should attempt to get to a prepared (ready) state
> h) please will you send a vote
> i) propagate this prepare to any subordinates
>
> With a closed-top top-level coordinator, it is impossible to split a, b, c
> and d, and in fact the whole lot are bundled together. In our new world,
> that is no longer true.

As I said in the marked-up version of Alastair's document, we have no
problem with atoms having prepare/confirm/undo (and in fact I thought we had
agreed on this and moved on).

>
> In particular, it would seem reasonable for an initiator to indicate that
> the current request is the last application message that will ever be sent
> to this service (a) - this is something that the initiator (and only the
> initiator application) can know of itself.

Agreed. Also, it is possible that the service knows it only ever has
idempotent, or readonly operations, and would be able to "early prepare".
This is something only the service knows. However, in both situations, this
can be accomplished with application messages (see earlier email) and
cooperation between the web service and its enlisted participant.

> But if the service receives such
> an indication, and has sent what it thus knows to be the last application
> reply, why should it not vote ?

Because it is not the service that votes. It is the participant.

> It will never receive anything that will
> make it more able to get into a ready state.
>
> 3) It is true that only the coordinator knows the true topology of the
atom
> (i.e. what participants are enrolled), and it therefore has the
> responsibility for ensuring the atom is unanimous. However this only means
> that it must ensure it does get a vote from all participants, not that it
> can't accept a vote from a participant well before the initiator has told
> the coordinator to get votes from everyone.

And it is therefore only the coordinator that should send PREPARE messages.
Let's not mix roles in this protocol. As I have indicated, since this is an
application level choice ("I won't send you any more messages"), *and* the
application may not know about atoms/PREPARE/... directly, it is not a clean
model to allow the application to run part of the termination protocol. It
should only deal with the functional aspects of the web services, and let
them interpret them accordingly, e.g., "I won't send you any more messages"
attribute means I can early prepare, so I'll tell my participant to do it.

> Let us allow the initiator to send a marker to a service meaning (a) as in
> 2) above, at any point where the initiator knows this to be true, and from
> that allow the service (after making what data changes it determines are
> appropriate) send a definitive vote with the reply. The vote is stripped
off
> (by some means - effectively part of the initiator/communication boundary)
> and delivered to the coordinator.

As I indicated above, this will not work in a general model. The participant
sends the vote and it should send it to the coordinator, not to the business
logic. If the two are co-located, then optimisations at the message level
are possible using interceptors (for example) at both the client and server.
However, as far as the participant is concerned it should only ever see the
coordinator. Likewise the business logic should only ever see the web
service, and not the participant.

> Eventually the initiator/terminator will ask the coordinator what the
> atom-wide vote is (this will often be intermediated by a cohesion entity,
> but not always). At this point the coordinator should send a prepare (with
> at least sense (h)) to any registered participants that haven't sent a
vote.
> It has then got votes from everyone, and can apply the usual (as for an
> interposed atomic coordinator) rules to work out what its vote is. If more
> than one atom is to be confirmed as part of a cohesion, it is at this
point
> essential that the combined votes are passed to a
(persistent-over-failure)
> composer; if only one atom is to be confirmed, it will in fact work to
have
> the volatile initiator/terminator make the decision. (it is only at
decision
> time that the top-most entity must be persistent).

There obviously needs to be a persistent record of the atoms that have been
prepared, and some record of what the final "outcome" on those atoms is to
be before that outcome is attempted (e.g., confirm A, C and D, but undo B).
This outcome therefore needs to be associated with each atom, unlike in a
transaction system where it on all participants.

> One complication on this would be a case with two services, both with
> participants, but with the first also acting as a context "router".
>
> some ascii art: (J is meant to be a right-angle turn !)
>
> I  --->   S/P(a)  ---->  S/P (b)
>    <-------J              |
>    |                      |
> C <J                      |
>   <-----------------------J
>
>  Posit that (a) sends its enroll AND its vote back directly, but (b) sends
> an enroll directly to the coordinator.

I think that your co-location assumptions are clouding this issue. If we
move away from that then perhaps we can re-start the discussion on even
ground.

> (it must make sure it has had an ack
> to the enroll before it replies on the application message, but that's
> regular checking rules). If the allowed-to-vote marker was propagated to
> (b), it may also vote (on the direct path to C); if not it will just sit
> there, unaware that it has received the full story (imaginably, it hasn't
if
> there are other similar branches in the atom).  If it doesn't vote, the
> coordinator will send it a prepare when asked to by the initiator, and get
> the vote then.
>
> Some implications of this are that it has to be allowable for a
participant
> to receive "prepare" in various forms more than once - certainly that it
> should be prepared to repeat its vote.

Agreed. That's not a problem.

>
> The rule for the Coordinator is that, when asked to prepare its atom, it
> sends prepare (sense h) to all registrees that haven't voted (to its
> knowledge).
>
> I believe that there should be a BTP-defined marker that the initiator can
> put on the application request to say that a vote is permissible.

I disagree. I think such a "marker" is up to the application to send and the
web service to interpret. For example, a ONE_SHOT attribute on the "book
taxi" method.

> Somewhat
> to my own surprise, I now think this should be just a permission (sense
f),
> not a request - it is up to the service+participant to decide when they
can
> vote.

Agreed.

<stuff deleted>

> Without getting some indication from above, the participant doesn't
> generally know whether it can vote - if the application messages + btp
> messages don't include any of senses a-d, the service+participant must be
> able to handle another message  - "hang on, I haven't finished yet".

It's up to the application to determine whether or not it has finished with
the service. How the service interprets this information (assuming it is
provided) is up to that service.

> > I appreciate that there are cases where the Initiator/Terminator
> > (application logic) needs to get a decision from a Participant before
> > deciding on the result of the cohesion ("confirm" or "cancel"). I do not
> > think instructing a Participant to vote and then receiving the vote is
> > the solution. Even if it is an indication that it could vote rather than
> > an instruction, I believe it is wrong. Participants should only send the
> > vote to the Coordinator and not to the Initiator/Terminator and they
> > should only receive a request for a vote from the Coordinator.
>
> why is the latter (especially as just an indication) "wrong" ? This isn't
a
> moral issue :-)

Let's stear clear of meta-physics! It's "wrong" (incorrect) from a model and
implementation stand-point.

>
> > The solution would be to allow the Initiator/Terminator to issue a
> > request for a prepare on the Atom. Since the call to the Web Service
>
> we need to be able to do sequential multiple one-shots in  a single atom -
> posit a transfer where there is a one-shot to get a value from service A
and
> then to deliver the value (or a derivation of it) to service B. That
> requires that A is prepared before B has been invoked.

Application level.

> I think what I've suggested above possibly meets your fears - the
> initiator/terminator doesn't directly deal with participants, but can mark
> webservice invocations such that the participants that the webservice
causes
> to be involved receive signals that allow them to send votes to the
> coordinator.


No, but what you suggest is based on an underlying assumption that
coordinator and initiator are co-located in order that "interception" can
occur.

Mark.

----------------------------------------------
Dr. Mark Little (m.c.little@ncl.ac.uk)
Distributed Systems Research Project
Phone +44 191 2228066
Fax   +44 191 2228232