Re: [wsbpel] Issue 147 - Proposal For Vote

["Yaron Y. Goland" <ygoland@bea.com>]

My concern with your first proposed change is that it will make portable 
behavior impossible. For example, what happens in the following case:

foreach iteratorVariableName="I" iteratorVariableType="e:foo"\
         parallel="no" validate="yes"
    expression expressionLanguage="xpath"
       $ick/blah
    sequence
       assign
          copy
             from
                staticValue
                   bar
                      blah
             to variable="ick"
       assign
          copy
             from variable="I"
             to variable="output"

The variable ick contained an element blah which itself contained 5 
nodes. So the output set in this case has 5 values. On the first 
iteration the first value is assigned to the iterator variable I. 
However the first thing that happens in the iterator is that the 
variable ick has its value changed so that all the children of the 
element blah (remember, those are the values that make up the input set) 
are deleted.

So what value does output end up with?

With the language in my proposal the answer is clear - no value, I 
becomes uninitialized and a fault occurs. But if we don't state that 
clearly then there will be no common definition for what happens to I. 
That's why I believe the behavior must be specified.
 > Again, you're telling someone how to implement their elephant.  But the
 > part about "if the referred to thing disappears, then all of a sudden
 > don't iterate over it" will lead to completely surprising results.  I
 > wonder as to why you've taken the trouble to think about these edge
 > cases, and then come up with the results you did.  I'd prefer in this
 > edge case at least, to err on the side of semantics, rather than
 > implementation details.  The iterator should iterate over the values of
 > the set at the time they were returned.

In the case of BEA's customers that is exactly the behavior they don't 
want. In the vast majority of cases our users are iterating over a XML 
structure for the purpose of editing that structure. If the outcome of 
that editing is that part of that structure should disappear then they 
don't want that value still in their result set as they would then be 
asked to edit a value that no longer exists. Similarly, if during 
editing the programmer causes some part of the variable to change in 
value that want that changed value to show up in all further iterations. 
Aliasing is absolutely essential to provide the core functionality that 
programmers manipulating large XML structures depend on.

I'd also point out that non-aliased behavior provides some deeply 
bizarre semantics. For example if someone is iterating through a 
purchase order in order to update data in that purchase order they won't 
be able to make any changes to the iterator variable because those 
changes will be lost when the iterator exits. This means that the 
for-each is useless to the programmer. They will have to invent their 
own for-each out of while and a counter (and lose parallel capabilities) 
in order to be able to do simple iterative structure editing. That 
rather defeats the whole purpose of having a for-each.

	Yaron

Danny van der Rijn wrote:
> Yaron Y. Goland wrote:
> 
>  > > -1 to the notion of spelling out the "returned set" and removing
>  > > elements from it.  Why specify to such detail how the iterator works?
>  >
>  > Can you please explain how it is possible to portably define for-each
>  > without specifying this behavior?
>  >
> Change:
> 
>  > The first
>  > entry in the sequence is an assign that assigns an EII from the
>  > returned set, in document order where possible, in random order where
>  > not, to the scope's only variable. Once an EII from the return set is
>  > assigned to the local variable that EII is removed from the return
>  > set. The next activity in the sequence is then the activity specified
>  > in the body of the foreach activity. If the virtual scope exits
>  > without error and if there are remaining EIIs in the returned set then
>  > the process repeats until the returned set is empty.
> 
> 
> To:
> 
>  > The first
>  > entry in the sequence is an assign that assigns an EII from the
>  > returned set, in document order where possible, in random order where
>  > not, to the scope's only variable. The next activity in the sequence
> is then the activity specified
>  > in the body of the foreach activity. If the virtual scope exits
>  > without error and if there are remaining EIIs in the returned set then
>  > the process repeats until the returned set is empty.
> 
> i.e. just remove the sentence about removing the EII from the set.  Then
> there need not be such a formal notion of the set as if it were a data
> structure.  We don't need to (and shouldn't) tell people how to build
> their elephants.
> 
> 
> 
> 
>  > > -1 to the above paragraph.
>  >
>  > Um... o.k. Would you like to explain why? And how you would replace it
>  > and still have for-each work efficiently?
> 
> 
> Again, you're telling someone how to implement their elephant.  But the
> part about "if the referred to thing disappears, then all of a sudden
> don't iterate over it" will lead to completely surprising results.  I
> wonder as to why you've taken the trouble to think about these edge
> cases, and then come up with the results you did.  I'd prefer in this
> edge case at least, to err on the side of semantics, rather than
> implementation details.  The iterator should iterate over the values of
> the set at the time they were returned.
> 
>  >
>  > > All this talk of references/aliases is going to expand the scope of the
>  > > spec.  I'm sure you have a proposal in the wings, but why not alias
>  > > variables?
>  >
>  > I think that introducing a generic mechanism to alias variables would
>  > expand the scope of the spec and not necessarily in a way useful
>  > enough to be justified for a 1.0 spec (no matter what we named the
>  > silly thing). On the other hand what is proposed in for-each is a very
>  > limited mechanism that exactly replicates the behavior of two
>  > processes manipulating the same variable.
> 
> I agree with you on the generic mechanism.  I think, though, that the
> detailed discussion in the proposal leads us too far in the same
> direction.  And again, I think that if I want to think about the case, I
> would want it to be specified so that the values don't change once the
> set is returned from the expression.  If you absolutely need to, specify
> copy-on-write semantics, no matter how twisted it would be to implement,
> or say that in that case, the results are indeterminate, leaving people
> free to implement how they wish.  Just don't introduce the alias notion,
> no matter how limited, to the spec.
> 
> I'm confused.  What is the behavior of 2 processes manipulating the same
> variable, and how did it get into this conversation?
> 
>  >
>  >     Yaron
>  >
>  > Danny van der Rijn wrote:
>  >
>  >>
>  >> Yaron Y. Goland wrote:
>  >>
>  >>  > 147 - Serial and Parallel For-Each
>  >>  >
>  >>  > Proposal: Introduce a foreach to BPEL with both serial and parallel
>  >>  > semantics.
>  >>  >
>  >>  > Rationale: Just check the issue list for issue 147. There is a whole
>  >>  > series of mails attesting to how important this activity is.
>  >>  >
>  >>  > Introduce a section 12.6
>  >>  >
>  >>  > Note: This proposal assumes that issue 126 is adopted.
>  >>  >
>  >>  > 12.6 - Foreach
>  >>  >
>  >>  > <foreach iteratorVariableName="ncname" iteratorVariableType="qname"\
>  >>  >            parallel="xs:Boolean" validate="yes|no"?
>  >> standard-attributes>
>  >>  >      standard-elements
>  >>  >      <expression expressionLanguage="anyURI">...</query>
>  >>  >      activity
>  >>  > </foreach>
>  >>  >
>  >>  > A foreach activity is used to iterate over a variable, either
>  >> serially
>  >>  > or in parallel. The foreach activity specifies an expression that is
>  >>  > to return a set of element information items (EIIs). The foreach also
>  >>  > specifies a variable name in the iteratorVariableName attribute and a
>  >>  > variable type in the iteratorVariableType attribute. All the EIIs
>  >>  > returned by the expression MUST be of the type specified by the
>  >>  > variable type. By default however a BPEL engine is not required to
>  >>  > validate variable types, if the programmer wishes to guarantee that
>  >>  > the output of the expression will be validated to be of the
>  >>  > appropriate type then the validate attribute should be set to 'yes'.
>  >>  > If the validate attribute is set to 'yes' and the output of the
>  >>  > expression are not all EIIs of the type specified by
>  >>  > iteratorVariableType then a mismatchedAssignment failure MUST be
>  >>  > thrown. Note that variable validation, if activated, is performed
>  >>  > before any of the 'notional' virtual scopes described below are
>  >> created.
>  >>  >
>  >>  > If parallel="no" then the foreach is a serial foreach. In that case,
>  >>  > conceptually, a new 'notional' virtual scope is created that does not
>  >>  > contain a fault handler nor a compensation handler. In fact, this
>  >>  > 'notional' virtual scope cannot be said to exist for purposes of
>  >>  > determining which scopes are children of other scopes. Please
>  >> refer to
>  >>  > section 13.5.1 for a full description of the compensation handle of
>  >>  > this 'notional' virtual scope. This 'notional' virtual scope contains
>  >>  > a single variable declaration with the name given in
>  >>  > iteratorVariableName and the type given in iteratorVariableType. The
>  >>  > virtual scope conceptually contains a virtual sequence. The first
>  >>  > entry in the sequence is an assign that assigns an EII from the
>  >>  > returned set, in document order where possible, in random order where
>  >>  > not, to the scope's only variable. Once an EII from the return set is
>  >>  > assigned to the local variable that EII is removed from the return
>  >>  > set. The next activity in the sequence is then the activity specified
>  >>  > in the body of the foreach activity. If the virtual scope exits
>  >>  > without error and if there are remaining EIIs in the returned set
>  >> then
>  >>  > the process repeats until the returned set is empty.
>  >>
>  >>
>  >> -1 to the notion of spelling out the "returned set" and removing
>  >> elements from it.  Why specify to such detail how the iterator works?
>  >>
>  >>  >
>  >>  > The EIIs in the returned set are aliases to real EII instances
>  >> held in
>  >>  > BPEL variables. It is possible for one of the EIIs being pointed to,
>  >>  > to be deleted while the for-each is executing. Such a deletion would
>  >>  > occur if the EII being pointed to was removed from the BPEL variable
>  >>  > that contained it. In that case the EII disappears from the returned
>  >>  > set and is not iterated over.
>  >>
>  >>
>  >> -1 to the above paragraph.
>  >>
>  >>  >
>  >>  > The foreach virtual scope local variable in the foreach iteration
>  >>  > instance is also an alias to an actual EII in a BPEL variable. While
>  >>  > the iteration instance is executing the EII could have its value
>  >>  > changed. If that happens then the situation is similar to what
>  >> happens
>  >>  > if a BPEL variable is shared by two simultaneously executing sections
>  >>  > of a BPEL, the change made in one section is instantly visible to the
>  >>  > other. If, however, the EII instance is deleted then the virtual
>  >> scope
>  >>  > local variable will become null. It is legal to assign to the virtual
>  >>  > scope local variable after it becomes null but the value will be lost
>  >>  > when that iteration ends, just as with any other scope local
>  >> variable.
>  >>  > To prevent other parts of a BPEL from altering the return set or the
>  >>  > scope local variable the foreach can be placed in a serialized scope.
>  >>
>  >>
>  >> All this talk of references/aliases is going to expand the scope of the
>  >> spec.  I'm sure you have a proposal in the wings, but why not alias
>  >> variables?
>  >>
>  >>  >
>  >>  > If the parallel attribute is set to 'yes' then a parallel foreach is
>  >>  > created. In a parallel foreach a virtual flow is created that
>  >> contains
>  >>  > a series of virtual scopes, one for each EII in the returned value
>  >>  > set, defined as given previously. As with a serial foreach the local
>  >>  > variables in the parallel scopes are aliases to actual BPEL values
>  >>  > with the same behavior as given above. While the possibility of
>  >>  > interference can be reduced by placing a parallel foreach into a
>  >>  > serialized scope this cannot prevent parallel virtual scopes
>  >> generated
>  >>  > by the foreach from interfering with each other since they all exist
>  >>  > inside of the same serialized scope. A parallel foreach ends using
>  >> the
>  >>  > same logic that controls any flow's behavior. That is, either all the
>  >>  > scopes in the flow successfully exit or one of the scopes throws an
>  >>  > unhandled fault and the flow is terminated.
>  >>  >
>  >>  > Addition to Appendix A:
>  >>  >
>  >>  > mismatchedAssignment  Thrown from a for-each if the returned iterator
>  >>  > set is validated and contains entries of a type other than that
>  >>  > specified in iteratorVariableType.
>  >>  >
>  >>  >
>  >>  > ---------------------------------------------------------------------
>  >>  > To unsubscribe, e-mail: wsbpel-unsubscribe@lists.oasis-open.org
>  >>  > For additional commands, e-mail: wsbpel-help@lists.oasis-open.org
>  >>  >
>  >>  >
>  >>
>  >
>  >
>