Re: [ws-caf] ACTION for optimization of registration

[Mark Little <mark.little@arjuna.com>]

I found more time ;-)

>
>2. OTS has a concept of "reply checking". This states that one must
>count back the application responses that match each application
>request. This is crude and excessive. 
>
>OTS states that checking in a distributed tree is tantamount to ensuring
>that all transactional work of all leaf nodes has been completed. 
>
>In fact: 
>
>a) an application response may be the first or nth of a conversational
>stream, and its arrival cannot be construed as equalling completion of
>anything other than its role in the application's own protocol; 
>
>b) the danger of unchecked behaviour arises, not from attempting confirm
>processing until all transaction work has been accomplished, but from
>attempting confirmation before all participants are enrolled. 
>
>A one-way message-based transaction completion protocol can begin to
>pass two-phase requests downtree long before all leaf node work is
>completed. The key point is that, while a PREPARE downtree may stall
>waiting for a PREPARED (wait for work to complete), we must not send
>PREPARE to only some of the required participants. If we do then we risk
>leaving them in a solitary state of preparedness (a state from which
>they can never escape short of a brutal backstop timeout on their
>attempts to retry obtaining a second-phase instruction). The terminator
>application should be able to leave transaction completion to occur
>without regard to its own lifecycle.
>
>3. BTP correctly addresses these two problems a) and b). It introduces a
>signal that means: "all enrolments stemming from an application message
>have been processed". This signal is a BTP message CONTEXT-REPLY. It can
>be sent independently of application responses or as part of an
>application response message (as a header element). This signal
>logically terminates the stream of 0..n ENROL signals that may arise as
>a result of a service receiving an inbound CONTEXT message:
>
>CONTEXT
>ENROL/ENROLLED
>ENROL/ENROLLED 
>...
>CONTEXT-REPLY
>
>This sequence could be thought of as the "checking sub protocol".
>
>(This is a simplification: the reply can be used to communicate failure
>of required enrolments, and there can be a stream of context replies to
>indicate to the app that it's in business, but that the protocol level
>checking hasn't been completed.)
>  
>
This is standard checked transaction semantics and nothing to do with 
the issue at hand. Please refrain from mixing the subjects/issues. Let's 
keep this on the subject of OPTIMIZING REGISTRATIONS, which is what the 
subjectline says. If you want to discuss checked transactions, then I 
respectfully ask that you do so within a different subjectline so we can 
track and manage these things independently.

>4. The point I have been trying to get across in the last week or so is
>that the need satisfied by CONTEXT-REPLY is separate from the issue of
>"boxcarring". 
>  
>
And I've been trying to get through the point that REGISTRATION 
OPTIMIZATION is separate from checked transactions!

>BTP has all kinds of (useful) network traffic optimizations. It can
>concatenate any number of protocol messages together in a wrapper
>element holding BTP messages ("bundling"). And it can also "group"
>messages to create compound semantics (e.g. CONTEXT-REPLY & ENROL &
>PREPARED). 
>
>Mark says:
>
>  
>
>>>(ii) a "boxcar" (a la BTP) of enlistment messages that accompany the 
>>>one-way "response" message. The precedent for this is BTP.
>>>
>>>Now it seems to me that (ii) could be dealt with by the underlying 
>>>SOAP infrastructure - that's always been the argument against putting
>>>      
>>>
>
>  
>
>>>boxcarring into the protocol. Whereas (i) is a specific approach for 
>>>a very specific problem.
>>>      
>>>
>
>Mark then goes on to explain that his approach i) -- so sedulously
>distinguished from BTP -- is in fact to have a bunch of enlistment
>messages that accompany a one-way response message called ... a response
>context, which sounds awfully like approach ii).
>  
>
I was careful to make the distinction between general boxcarring, which 
I certainly credit to BTP, with a specific form for this problem alone. 
Call it BTP-boxcarring-lite if you want, I really don't care.

>Without vectorization we cannot group the context reply ("checked") with
>the enlistments or registrations that must succeed to achieve the
>checked state. 
>
I really don't want to get into this discussion about checked 
transactions in this issue because we can and should deal with these 
issues separately. Start a different thread.

>The checked message has no magical power on its own to
>achieve vectorization.
>
>5. Mark is right to say there are two ways of achieving vectorization:
>by relying on the infrastructure, and by doing it yourself. I have to
>smile when I read that the SOAP infrastructure will do it for us. I have
>been hearing this for four years. In what decade, and through which
>well-known protocol nearing maturity within the WS architecture will
>this occur? 
>
>In BTP we did three wise things as a TC on this kind of question --
>refused to reference incomplete specs, defined all the facilities we
>needed abstractly, and defined a "today's technology" binding that
>concretely expressed them -- while defining how a "tomorrow's
>technology" binding might be created when the time came. The abstract
>EPR/WS-A issue. Are these WS-CAF specs to be capable of implementation
>this or next year? If so then I suggest we need to define how boxcarring
>is going to be done, or drop the attempt to support it altogether. 
>  
>
I agree on dropping the boxcarring issue entirely, but I think this 
specific problem can be dealt with independently of that. It was you who 
brought up the issue of BTP and how we tackled things in that TC. If we 
rollback that conversation to before BTP was even mentioned, then I 
think we can have some text that at least addresses this issue and we 
can proceed to a vote. I'm happy to accept whatever the outcome of such 
a vote might be.

>6. Which brings me back to Mark's proposed text. Boxcarring is achieved
>by stating that the context will contain enlistments as extension
>elements. Mark comments:
>
>  
>
>>This all presupposes something along the lines of (i) above. [AG:
>>    
>>
>having a context reply by using an extended context] 
>  
>
>>I'm going to continue to recommend that approach because I think
>>    
>>
>general 
>  
>
>>boxcarring as outlined in (ii) [BTP boxcarring in a context reply] is 
>>something that the runtime infrastructure will take care of. 
>>However, that still leaves open the 
>>issue of what is meant by "context" in the provided text. Is is a) a 
>>WS-Context as I original indicated, or is it b) the specific 
>>addParticipant messages, i.e., a very specific form of boxcarring for 
>>this very specific problem? As I've already said, I prefer the 
>>conceptual model of contextualising all of the one-way messages, so a)
>>    
>>
>
>  
>
>>probably edges out b) for me, but at the moment I can't think of any 
>>technical reasons for one over the other. I think how the information 
>>is conveyed is less important than agreeing to convey that information
>>    
>>
>
>  
>
>>in the first place.
>>    
>>
>
>I summarize this as:
>
>Approach i) [Nothing to do with OTS, original Mark proposed text]
>  
>
Wrong. See previous email.

>= have a context response and put a number of enlistment elements within
>it.
>
>Approach ii) [BTP] 
>
>= put a number of enlistment elements within a wrapper, and then
>optionally carry that freestanding or grouped with a context reply.
>
>Approach iii) [Mark musing on an alternative]
>
>= "have addParticipant (enlistments) messages in a vector". 
>
>On iii) How, where? 
>
My preference is for within a reverse flow WS-Context context. I "mused" 
about another approach to try to bridge the divide between our 
approaches. I kind of assumed if we had agreement on the aim, then we 
could follow up with an agreed upon substance. Since you're not happy 
with that either then I'm more than happy to ignore it.

>As free-standing header elements (which implies they
>must run alongside application messages (which is not required)? As
>children of an element specifically for bundling WS-CAF protocol
>messages -- exactly what <btp:messages/> does in BTP? This would allow
>transmission alone or piggybacked as a header element on an app message.
>This leads to a wrapper, and thence straight to approach ii), as in BTP.
>I think it would be brain-bending to use the context itself as such a
>wrapper, but someone might come up with that suggestion.
>  
>
I fail to understand this "brain-bending" issue. In WS-CF we already 
have the capability to have participants within the context. This was 
originally in WS-Context, but was moved to where the TC believed it more 
accurately belonged. This is the wscf:participant-service element within 
the WS-CF context. What I am suggesting would build on this.

>[Incidentally I cannot see why one would restrain oneself from using the
>technique in i) as a technique for *any* bundle of messages. And I also
>cannot see why you would hold back from allowing a wrapper to contain
>anything CAFish.]
>
>I simply do not understand what "contextualizing all the one-way
>messages" has to do with this. The one-way messages of the application
>must *all* be contextualized, even if they have no need? What if the
>protocol semantics have nothing to do with app messages and are
>out-of-band to the app? The one-way messages of the protocol(s) must all
>be contextualized? What would that mean?   
>  
>
We say nothing about when application messages are contextualised, 
beyond statements that the context must be available in order to be 
used! From a model perspective, there is nothing wrong with sending the 
context on all messages in the same way that a WS-A header element must 
accompany all one-way messages. I think the same argument that myself, 
Greg and others have been making for the utility of WS-Context over WS-A 
stands, whether the one-way message is a "request" or a "response".

>7. WS-CF and transactions: how do they interrelate in this area?
>
>Finally, to pull it all together -- WS-CF needs boxcarring to allow
>vectorized tree-building or graph-painting. WS-TXM (and it won't just be
>WS-Acid, which raises the need for a general TXM level protocol), needs
>to have a way of saying "checked, enrolments done". 
>
>They may or may not be combined in a single message, but they are
>semantically distinct. 
>  
>
I agree with Greg's observation that this issue (optimization of 
registration) should occur within WS-CF. With that in mind, I'll produce 
an updated text for discussion and possible vote.

>If they are combined then they need a bucket to travel in together. An
>augmented context can be used as the bucket. It needs to have an
>extension element for checked that belongs to WS-TXM (and undefined
>message in the current spec). It needs to be able to contain multiple
>addParticipants for WS-CF. 
>
>But a context is not usually thought of as a body element, and it is
>unclear why context is being used just as a bucket, aka a wrapper. And
>it is unclear why the checked message should have to travel with an app
>message, as a header etc.
>
I said nothing about "checked message". In fact, I've tried very hard to 
keep these things separate.

> And it is unclear why addParticipants should
>have to travel in something called a context to achieve multiple
>registrations in one go. 
>
>This points to a WS-CAF bucket for multiple messages, that can be used
>as a header element, as a body element, as an extension element of
>context if need be etc. This is what BTP does. It works. It addresses
>all use-cases.
>  
>
And is not used by any other WS specification in existance, to the best 
of my knowledge. I agree that it's nice in theory, but it tackles a 
wider problem than this specific issue. If memory serves, boxcarring was 
not something that just happened overnight in BTP - it took us quite a 
while to agree on, because of the general aspects. So I would prefer to 
consider this optimization for WS-CF in isolation and if there is a 
wider need later, make further choices. Why expend effort for 
unquantifiable gain?

>8. Checking can also be achieved by counting of identified enrolled
>inferiors. This can be dealt with separately, but it should not be
>forgotten. It is a WS-TXM-general need, not specific to WS-Acid. 
>
>* * *
>
>This entire discussion begs the question: why are we trying to reinvent
>the wheel of BTP in these areas? I venture to suggest that the same
>unavoidable question will plague most other discussions as we proceed
>with WS-TXM. This is exactly what I meant by "needless variation". 
>
>We would perhaps do ourselves and the wider audience a favour if we
>frankly set out to copy BTP as faithfully and as literally as makes
>sense given the layering of WS-Context and WS-CF. That way we would end
>up with a workable and proven solution of equal power, in a short space
>of time, on this issue, and on many others. 
>  
>
Sorry. That argument has been considered and voted against. Let's close 
it and move on.

Mark.

>Alastair
>
>
>
>Alastair J. Green
>CEO and CTO 
>Choreology Ltd
>68 Lombard Street
>London EC3V 9LJ
>www.choreology.com
>
>+44 870 739 0050
>+44 870 739 0051 (fax)
>+44 795 841 2107 (mobile)
>
>
>-----Original Message-----
>From: Greg Pavlik [mailto:greg.pavlik@oracle.com] 
>Sent: 19 July 2005 18:44
>To: Mark Little
>Cc: ws-caf
>Subject: Re: [ws-caf] ACTION for optimization of registration
>
>Mark Little wrote:
>
>  
>
>>Conscious of the telecon today, I'd like to keep this rolling.
>>
>>I think the original issue of optimizing registration calls should be 
>>orthogonal to checked transactions. As was discussed at the f2f, this 
>>issue is only concerned with reducing the number of cross-domain 
>>registration calls. With that in mind ...
>>
>>    
>>
>yes, I agree. This has been the one problem point in the discussion so 
>far. To clarify my early statement on WS-CF: I'm in favor of optimizing 
>registration at that level but not in adding any additional semantic 
>implications: my view is that CF  provides the building blocks for 
>organizing the participant graph but nothing more.
>
>  
>
>>>Actually I think that we have two different ways of approaching the 
>>>same problem.
>>>
>>>
>>>
>>>Pros and cons both ways.
>>>      
>>>
>>We have to make a choice. Fairly obviously, there's an implied (iii) 
>>in the above, which is "do nothing", i.e., drop the issue. Assuming we
>>    
>>
>
>  
>
>>don't want to go that route just yet (afterall, we're still in the 
>>discussion phase), let's return to the original text which I had the 
>>AI for generating:
>>
>>"A service that receives a transaction context on an application 
>>request  SHOULD enlist participants with the corresponding activity 
>>group. The transaction service implementation associated with the 
>>service MAY decide to use the following protocol to optimize remote 
>>registration invocations: this optimization MAY occur transparently to
>>    
>>
>
>  
>
>>the service. Rather than register participants directly, the EPRs of 
>>the participants (and the protocols they should be enlisted with) are 
>>retained by the service-side transaction service component until the 
>>response to the original invocation is sent; in which case a context 
>>containing these EPRs and associated data is propagated back with the 
>>response.
>>
>>If a receiver of a response obtains this context it MUST either be 
>>able to perform the enlistment itself or, if it does not support this 
>>optimization, send back a wsacid:CannotOptimizeEnlistment fault code 
>>message and mark the transaction so that its only outcome is to 
>>rollback. If enlistment fails then the transaction MUST be rolled 
>>back; an implementation MAY retry transient registration failures.
>>
>>If the sender of the response receives an indication that there was a 
>>non-recoverable failure (e.g., in delivery of the message or 
>>registration of the participant EPRs) then it MUST rollback the work 
>>it has performed in the scope of that transaction. When using this 
>>optimization, the service MUST still see a wscf:participantAdded 
>>message in order to be compliant with WS-CF. However, this message 
>>SHOULD be generated by the service-side transaction component for this
>>    
>>
>
>  
>
>>optimization to work."
>>
>>
>>Mark.
>>
>>    
>>
>
>  
>

-- 
Mark Little
Chief Architect
Arjuna Technologies Ltd
(www.arjuna.com)