Issue - 160 - Supplementary note

[Alex Yiu <alex.yiu@oracle.com>]

Hi,  all,

As promised in F2F, here is a supplementary note to illustrate how 
existing <assign>/<copy> can create data which are invalid to XSD.

Food for thoughts ... Thanks!  :-)


Regards,
Alex Yiu

Title: Supplementary Note for Issue 160 - Validation Boundary

Supplementary Note for Issue 160 - Validation Boundary

Here are some examples to illustrate how existing
<assign>/<copy> operation can violate XML Schema. Those
situations are typically difficult or impossible to be detected just
doing static analysis: 

[1] Key/KeyRef - Key/KeyRef is similar to the primary key and foreign key of referential integrity facture in RDBMS

Consider this XSD sample for Orders and Suppliers and the key/keyref constraints between them:

<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://foo.com"
targetNamespace="http://foo.com" elementFormDefault="qualified"
attributeFormDefault="unqualified">
    <xs:complexType name="tSupplier">
        <xs:sequence>
            <xs:element name="supplierID" type="xs:short"/>
            <xs:element name="name" type="xs:string"/>
        </xs:sequence>
    </xs:complexType>
    <xs:complexType name="tOrder">
        <xs:sequence>
            <xs:element name="orderID" type="xs:short"/>
            <xs:element name="lineItem" maxOccurs="unbounded">
                <xs:complexType>
   
           
    <xs:attribute name="partId" type="xs:short"
use="required"/>
   
           
    <xs:attribute name="partDesc" type="xs:string"
use="required"/>
   
           
    <xs:attribute name="supplierId" type="xs:short"
use="required"/>
   
           
    <xs:attribute name="quantity" type="xs:int"
use="required"/>
   
           
    <xs:attribute name="unitPrice" type="xs:double"
use="required"/>
                </xs:complexType>
            </xs:element>
        </xs:sequence>
    </xs:complexType>
    <xs:element name="OrderAndSuppliers">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="order" type="tns:tOrder"/>
   
            <xs:element
name="supplier" type="tns:tSupplier" maxOccurs="unbounded"/>
            </xs:sequence>
        </xs:complexType>
        <xs:key name="supplierID">
            <xs:selector xpath="./tns:supplier"/>
            <xs:field xpath="tns:supplierID"/>
        </xs:key>
        <xs:keyref name="OrderToSupplier" refer="tns:supplierID">
            <xs:selector xpath="./tns:order/tns:lineItem"/>
            <xs:field xpath="@supplierId"/>
        </xs:keyref>
    </xs:element>
</xs:schema> 

Say, we have the following XML data which is valid:

<OrderAndSuppliers xmlns="http://foo.com">
    <order>
        <orderID>123</orderID>
        <lineItem partId="223" partDesc="some parts" quantity="33" unitPrice="32" supplierId="1002"/>
    </order>
    <supplier>
        <supplierID>1002</supplierID>
        <name>Supplier 1002</name>
    </supplier>
</OrderAndSuppliers>

Now we have the following assign/copy operation to update a supplier: 

<assign>
    <copy>
        <from>
        <supplier xmlns="http://foo.com">

            <supplierID>1003</supplierID>

            <name>Supplier 1003</name>

        </supplier>
        </from>
        <to variable="x" query="/foo:OrderAndSuppliers/foo:supplier" /> 
    </copy>
</assign> 

Now, the data becomes the following, which is invalid because of the mismatch between key and keyRef:

<OrderAndSuppliers xmlns="http://foo.com">

    <order>

        <orderID>123</orderID>

        <lineItem partId="223" partDesc="some parts" quantity="33" unitPrice="32" supplierId="1002"/>

    </order>

    <supplier>

        <supplierID>1003</supplierID>

        <name>Supplier 1003</name>

    </supplier>

</OrderAndSuppliers>


[2] Regular Expression -
Regular Expression can used to restrict string values to fit certain
pattern. (e.g. SSN,  phone number  package tracking number
pattern and etc.)

    <xs:simpleType name="SSNStr">
        <xs:restriction base="xs:string">
            <xs:pattern value="\d{3}-\d{2}-\d{4}"/>
        </xs:restriction>
    </xs:simpleType>

The following regular expression will produce a string value valid to the SSN regular expression. 

<assign>
    <copy> 
        <from><expression>'123-45-678'</expression></from>
        <to variable="mySSNStrVar" /> 
    </copy> 
</assign> 

BPEL 2.0 spec is depending on XPath 1.0 as our default expression
language. It has only the generic string concept and does not have the
concept of restricted string type, we simply have no means to enforce
the compatibility between restricted SSN string type and a generic
XPath expression. Consider the following assign/copy operation may or
may NOT produce a valid SSN string. We cannot determine it by just
doing static analysis.  

<assign>
    <copy> 
        <from><expression>foo:someBarFunction(...)</expression></from>
        <to variable="mySSNStrVar" /> 
    </copy> 
</assign> 

Even after we move to XPath2.0/XQuery1.0, by following the same design
principle of XQuery 1.0 specification, it is considered unpractical to
require that :

• only expressions of SSNStrType can be used to produce values for SSNStrType variables; OR;
  
• runtime validation  happens on every operation that produces value for SSNStrType.

[3] xsi:type - "xsi:type" is an XML Schema standard attribute to denote the type of an element:

Consider this XSD definition:

    <xs:complexType name="employeeType">
        <xs:sequence>
            <xs:element name="firstName" type="xs:string"/>
            <xs:element name="lastName" type="xs:string"/>
        </xs:sequence>
    </xs:complexType>
    <xs:complexType name="managerType">
        <xs:complexContent>
            <xs:extension base="tns:employeeType">
                <xs:sequence>
   
           
    <xs:element name="approvalLimit"
type="xs:double"/>
                </xs:sequence>
            </xs:extension>
        </xs:complexContent>
    </xs:complexType>
    <xs:element name="employeeList">
        <xs:complexType>
            <xs:sequence>
   
            <xs:element
name="employee" type="tns:employeeType" maxOccurs="unbounded" /> 
            </xs:sequence>
        </xs:complexType>
    </xs:element>
    <xs:element name="mgrList">
        <xs:complexType>
            <xs:sequence>
   
            <xs:element
name="employee" type="tns:managerType" maxOccurs="unbounded" /> 
            </xs:sequence>    
        </xs:complexType>
    </xs:element>

[3.1] Dealing with xsi:type at runtime: 
In this XSD, we define an "employeeType" complex type which is the base
of "managerType" extension. We also define an "employeeList" which is a
list of employees of "employeeType" and "managerType". The type of the
employee is identified by xsi:type. E.g.:

<employeeList xmlns="http://foo.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <employee>
        <firstName>John</firstName>
        <lastName>Doe</lastName>
    </employee>
    <employee xsi:type="managerType">
        <firstName>Mary</firstName>
        <lastName>Jane</lastName>
        <approvalLimit>1234.00</approvalLimit>
    </employee>
</employeeList>

We also define "mgrList" which consists of employee who are managers only. 

Say, we have the following <assign> operation to copy a manager employee from a employee list to a manager list. 

<assign>
    <copy>
            <from variable="employeeListVar"
                query="/foo:employeeList/foo:employee[2]" />
            <to variable="mgrListVar" query="/foo:mgrList/foo:employee[1]" /> 
    </copy>
</assign>

With static analysis, we cannot guarantee that the above assign
operation will actually copy a manager employee or it may just copy a
non-mangerial employee instead. If the latter case, it will produce an
invalid "mgrList". Again, following the same design principle from
XQuery, it is not practical to force runtime validation on every
manipulation operation on "mgrList". 

[3.2] Changing xsi:type

    <foo:employee xmlns:foo="http://foo.com" xsi:type="foo:employeeType">

        <foo:firstName>John</foo:firstName>

        <foo:lastName>Doe</foo:lastName>

    </foo:employee>

By using <assign>/<copy>, we can change the type of the employee from a plain employee to a manager employee. E.g.: 

<assign> 
    <copy>
        <from>foo:managerType</from> 
         <to variable="myEmployeeVar" query="/foo:employee/@xsi:type" /> 
    </copy> 
</assign> 

After changing the xsi:type, the employeeVar becomes invalid, because the approvalLimit is missing. 

[4] xsd:any  

The xsi:type attribute basically allows dynamic runtime nature of the
type of a piece of XML data. That is: (a) one cannot tell the exactly
type of the data during static analysis time (b) one can change the
type of the data during runtime by changing xsi:type attribute

"xsd:any" another common used construct in XSD poses a similar
challenge to static type analysis of assignment operation. When
"xsd:any" is used in a schema, the exact type of the content is not
known until run-time. 

[5] Local element declaration: 
XML Schema allows the same element name (QName) to be used as a local
element declaration with different types. For example, consider this
XSD fragment:

<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://foo.com"
targetNamespace="http://foo.com" elementFormDefault="qualified"
attributeFormDefault="unqualified">
    ... 
    <xs:element name="customer">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="checkingAccount">
                        ... 
                </xs:element>
                <xs:choice>
   
           
    <xs:element name="consumerCustomer">
   
           
        <xs:complexType>
   
           
           
<xs:sequence>
   
           
           
    <xs:element name="id"
                   
                type="tns:consumeridType" />
   
           
           
    <xs:element name="firstName"
                   
               
type="xs:string" /> 
   
           
           
    <xs:element name="lastName"
                   
               
type="xs:string" /> 
   
           
           
</xs:sequence>
   
           
        </xs:complexType>
                    </xs:element>
   
           
    <xs:element name="smallBusinessCustomer">
   
           
        <xs:complexType>
   
           
           
<xs:sequence>
   
           
           
    <xs:element name="id"
                   
                type="tns:businessidType" />
   
           
           
    <xs:element name="name" 
                                   
type="xs:string" /> 
   
           
           
</xs:sequence>
   
           
        </xs:complexType>
                    </xs:element>
                </xs:choice>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
</xs:schema>


It defines a customer based on a choice of consumer customer and small
business customer. Both kinds of customer have "foo:id" field. E.g.: 

<foo:customer xmlns:foo="http://foo.com">
    <foo:checkingAccount>...</foo:checkingAccount>
    <foo:consumerCustomer>
        <foo:id>123</foo:id>
        <foo:firstName>Peter</foo:firstName>
        <foo:lastName>Smith</foo:lastName>
    </foo:consumerCustomer>
</foo:customer>

This <foo:id>123</foo:id> is of "consumeridType".

<foo:customer xmlns:foo="http://foo.com">
    <foo:checkingAccount>...</foo:checkingAccount>
    <foo:smallBusinessCustomer>
        <foo:id>A234-67</foo:id>
        <foo:name>Small Pizza Oven</foo:name>
    </foo:smallBusinessCustomer>
</foo:customer>

This <foo:id>A234-67</foo:id> is of "businessidType".

If the following <assign> operation is used:

<assign>
    <copy>
        <from variable="customerVar" query="/foo:customer//foo:id" /> 
        <to variable="..." /> 
    </copy>
</assign> 

then, we will know the type of "foo:id" only during runtime (similar to
the cases of "xsd:any" and "xsi:type"). Therefore, only limited static
type analysis can be done in this case. Again, by following the design
principle of XQuery 1.0, it is not practical do runtime validation at
every single step of operation of this kind. 

[6] What is XQuery doing on this front?

XQuery does not validate any data at runtime by default. It will only
validate data only when the "validate" is explicit used in their
XQuery: 

validate { ... expression / variable ... } 

See: http://www.w3.org/TR/xquery/#id-validate

END