Re: Open-top coordinators and protocol

[Alastair Green <alastair.green@choreology.com>]

Hi Savas,

Savas Parastatidis wrote:

> (Hoping this message will reach the list :-)

I actually received six copies of this mail, dated from 2.23 to 2.45 am!

>
> Dear all,
>
> A comment on Alastair's document...
>
> I agree with the description of the Initiator-Coordinator interaction as
> a message-set rather than a specific API. However, since there has
> already been a decision on using the Web Services world as a
> proof-of-concept arena, it would be wise to agree on an API as well, for
> those interactions between actors that we agreed to define (WSDL would
> do just fine). If we do not do that, I cannot see our interoperability
> demonstrator ever working :-)

I think there is a distinction between portability and interoperability. If
we wanted to create a portable Java demonstrator (for example), then we
would have to have a standardized API in Java that defined all the
relationships between application components and the BTP protocol (analogous
to the Java or C++ mappings of CORBA being used to create mostly portable
code for OTS implementations). We would then be able to take the Java
implementation of BTP from Choreology and BEA and try rebuilding and
rerunning across the two implementations. There are lots of  issues involved
in getting to this level of portability..

CORBA has standardized algorithmic language mappings from a canonical IDL,
which can be used to create local and distributed objects; we have no
equivalent.

WSDL, as I understand it, is essentially an interface definition language
with bindings that enable you to produce wire formats (encodings) for the
messages described. Therefore, it *might* be reasonable to use WSDL to
define the protocol message sets (although that seems unnecessarily
heavyweight as an interoperability feature, as an XML schema would do the
trick).

But WSDL has no standardized language mappings, unless I'm missing
something. There is no standard way of creating Java objects that map WSDL
definitions, or C# objects or C++ objects, etc etc. Therefore, to use WSDL
to define an API might not be so useful.

After that, you still have to deal with bootstrapping (an area where CORBA
OTS portability breaks down). Common naming conventions for OASIS packages.
Interface/implementation separation to allow different implementations, in
languages without clear interface constructs ...

Interoperability is a simpler, cleaner target, which is also a defined
requirement. The current document refers to the original Choreology
submission, which read:

"Requirement #12: Interoperability

An interoperable protocol will be based on agreed network message formats
for distributed processing and clear contractual commitments on the parties
involved in requesting or receiving the invocation of a computation.

It is assumed as an overriding requirement that message formats will be
described in terms of XML schemas, capable of validation on despatch and
receipt:"

Interoperable implementations use standardized message formats, understand
the correct sequence of messages and the reactions that are required in
response to
receipt of messages. In order to communicate they must have a common
understanding of messaging, in terms of underlying carrier protocol,
addressing of endpoints, and concrete encodings (wire formats).

An interoperability demonstration would allow initiators, services,
coordinators and participants, each created using different implementations,
to interact correctly. The API used to create the language objects that
represent these actors (or actors-in-roles, depending on your view) could be
wildly different from implementation to implementation. The actors would
still interact correctly.

We did decide, way back at a models FTF, that an illustrative API would help
us to discuss and explain the use of the BTP protocol. Then the last FTF
placed heavy emphasis on the need to create distributed coordination sites
(ones separate from the initiator) and this led to the decision to bite the
bullet and define the protocol for the I-C relationship.

Now, it turns out that you have found my very crude illustrative API useful
in discussing the implicit prepare issue. (See further comments below). I
happen to find Java easier to read than WSDL, but that may be my age telling
:-)

>
> Now, some comments on implicit prepare...
>
> While looking again at the latest draft of the spec, I realised that the
> example code included in the document may greatly assist our discussion
> (a trimmed down version is included bellow for illustration purposes).
> In the example, the prepare call is made on the atom. A message is sent
> to the Coordinator rather than the service. The Coordinator knows the
> participants registered with the atom and it is its responsibility to
> collect the votes.
>
> If my understanding of Alastair's proposal is correct,

Well, this actually goes back to the BEA submission. I agree with Sanjay and
disagree with Mark Little that there is no proven or likely need for
one-shotness. I listen to BEA's experience with an actual product out in the
market. It is backed up by comments made by Mark Potts, whose company also
has deployment experience in this field. In principle, it seems obvious that
many web service interactions will be based on something similar to
"getQuote". Practical experience in many environments also makes me allergic
to extra messages (and _relatively_ indifferent to payload size).

> the
> sendApplicationMessage() and prepare() calls should be replaced with a
> sendApplicationMessageAndAllowPrepare() single call. This will result to
> a (number of) participant(s) being registered with the coordinator of
> the atom and at the same time, those who wish, they send their vote. But
> where do they send the vote? In its original form, the example bellow
> suggests that the Coordinator will return the result of the prepare (via
> the atom interface). If we are to replace the two calls with just one
> (as described above), who is going to receive the vote? If the result
> goes to the Initiator, then we do not honour the spec that specifies the
> interactions between actors (the participant sends the vote to the
> initiator).

The vote would always be returned to the Coordinator. The service receives
the CONTEXT, containing the coordinator address. It sends back an ENROLL to
that address (in this case piggybacked on the app response) which contains
the participant address, swiftly followed by a VOTE. The interception code
which would be required to achieve this strips the ENROLL and VOTE and
delivers them to the Coordinator. The net result is no different than
ENROLL, followed by a PREPARE (sent by Coordinator to Participant), followed
by a VOTE (sent by Participant to Coordinator). The vote has been received.

The only difference between the one-shot case, and the long-winded approach,
is that messages are boxcarred via interception, rather than being sent
out-of-band (asynchronously to the application traffic). For this to be
achieved the protocol actors (Coordinator, Participant) decide to _delegate_
message transmission to the application actors (Initiator, Service).

Delegation is not something that the standard can prohibit (other than by
some odd, ex cathedra pronouncement). How an implementation sends a message
from actor A to actor B is not our business. How it generates "return
addresses" for others to use in future communications is not our business.
So it is somewhat idle to argue about the possibility or desirability of
delegation. [There are some additional issues when you deal with security,
but that's another pot of honey altogether]

The next issue here is: should the protocol allow boxcarring? Should we
allow messages (any messages) to be concatenated on the wire, so that an
interoperable receiver can ingest several messages at once? I can see no
good reason to prevent that kind of optimization.

If we tolerate delegation (which is hidden from us), and permit boxcarring
then you can't stop the one-shot deal being implemented. Then the debate
resolves to the issue PREPARE or PERMISSION_TO_PREPARE, which is another
subject.

> If the vote goes to the Coordinator, then the Initiator does not know the
> result of the vote. It needs to talk to the Coordinator to
> find out.

This is not new. All that has happened is that the initiator used some API
to say <boxcar PREPARE with this message>, and the VOTE was returned.

[all CAPS like this means "BT protocol message"; <inanglequotes> means "API
call". "I" means "initiator", "C" is "coordinator", "P" is "participant",
"S" means "service", "INT" means "interceptor", *REQ" means app request,
*RESP" means app response]

In the "standard" case, you see this happen:

I -- GET_CONTEXT -- C
C -- CONTEXT -- I
I -- CONTEXT -- INT
I -- <sendrequest> -- INT
INT -- *REQ* + CONTEXT -- S    [net message 1]
S -- ENROLL -- C                           [net message 2]
S -- *RESP* -- I                               [net message 3]
I -- <prepare> -- C
C -- PREPARE -- P                         [net message 4]
P -- VOTE -- C                                [net message 5]
C -- <outcome> -- I

In the optimized case, you see this happen:

I -- GET_CONTEXT -- C
C -- CONTEXT -- I
I -- CONTEXT -- INT
I -- <sendrequestwithprepare> -- INT
INT -- *REQ* + CONTEXT + PREPARE -- S  [net message 1]
S -- ENROLL + VOTE + *RESP* -- INT          [net message 2]
INT -- ENROLL + VOTE -- C
INT -- *RESP* -- I
I -- <prepare> -- C
C -- <outcome> -- I

In the optimized case the prepare API call just gets back the vote result as
held locally.

2 net messages between I and S, instead of 5 in the standard case.

I assume for this discussion that I, C and INT are colocated, and S and P
are colocated. That way we see the real likely impact.

> It has to call prepare on the atom. Granted, the prepare may
> return faster in case the participants returned a vote at the time of
> their registration.

Which is part of the point of all of this.

>
> void cohesionComposer() // an application method
> {
>    Atom orderGoods = new Atom();
>
>   // application work
>
>   Quote quoteForGoods =
>     orderGoods.sendApplicationMessage ("quoteForGoods", arg, arg .)
>
>   // ensure that the quotes are guaranteed (may be folded into app
> messages)
>   orderGoods.prepare();                // no exception, so it is ready
>   orderGoods.confirm();
> }
>
> Having said all the above, I can see the rational behind this approach
> :-)
> I would be happier, though, if the optimisation utilised the fact that
> the atom (or atom context or atom ID) need to be propagated to the
> participants. Since the prepare is part of the atom's interface, we
> could configure the atom so that it contains the information required
> for participants to see that an implicit prepare is allowed. So the code
> would like this:
>
> void cohesionComposer() // an application method
> {
>    Atom orderGoods = new Atom(AllowImplicitPrepare);
>
>   // application work
>
>   Quote quoteForGoods =
>     orderGoods.sendApplicationMessage ("quoteForGoods", arg, arg .)
>
>   // Check the status
>   orderGoods.prepare();
>   orderGoods.confirm();
> }

This is API design, which is interesting (for implementers) but not strictly
relevant for specification writers.

>
> Again, the same problem arises. The initiator does not know about the
> vote so it needs to call prepare which could be faster. I think I am
> happier with this though :-) This approach clearly shows that the
> implicit prepare is part of the atom's context and has nothing to do
> with application messages.
>
> Lastly, if an implicit prepare was to be sent together with an
> application message, how would the service know which participants must
> receive it? Some of the participants may have already been registered
> because of earlier application messages on the same service.

That's an interesting question. My default answer would be that PREPARE on
an app message goes to whoever it is destined for, i.e. addressed to. And I
assume that a PREPARE in a one-shot case has no addressee, because the
participant is not yet known, so it should only be "delivered" to newly
registered participants. It would be legitimate therefore to see a response
message

S -- *RESP" + ENROLL(Pw) + ENROLL(Pz) + VOTE (Pw) + VOTE (Pz) -- INT

The termination scenario might then look like this: Assume that C has
already received "early" VOTEs from participants W and Z, but only
ENROLments from X and Y (as a result of earlier app messages)

[VOTEs in for W and Z]
I -- <prepare> -- C
C -- PREPARE -- Px
C -- PREPARE -- Py
Px -- VOTE -- C
Py -- VOTE -- C
C -- <outcome> -- I

Yours,

Alastair

>
begin:vcard 
n:Green;Alastair
tel;cell:+44 795 841 2107
tel;fax:+44 207 670 1785
tel;work:+44 207 670 1780
x-mozilla-html:FALSE
url:www.choreology.com
org:Choreology Ltd
version:2.1
email;internet:alastair.green@choreology.com
title:Managing Director
adr;quoted-printable:;;13 Austin Friars=0D=0A;London;;EC2N 2JX;
fn:Alastair Green
end:vcard