Re: [ubl-comment] Ammendment Re: Suggestion for a basis for subsetprofile conformance clauses

[Stephen D Green <stephengreenubl@gmail.com>]

Hi Jon

 

Writing the test assertions does not have to be part of writing

the subset profile spec, though it help to do it at the same time.

It is more a QA step for the writing of the profile spec, to ensure

that the conformance clause actually allows conformance tests.

If there is another approach to making sure the profile can be

conformance tested then fine (though a test assertions method

gets some OASIS traction and traction from industry perhaps). 

 

I would point to the 'UBL 1.0 SBS' subset and its 'must under-

stand'/'may ignore' kind of conformance clause. At the time

this seemed to be the closest fit to the business requirements

but I hadn't been able to establish whether it could actually be

tested. How does a conformance test bed test whether software

has 'understood' a BIE? How does it test whether or not software

ignores a BIE? I now think it has to be abandoned as a basis for

a profile's conformance clause and something more easily tested

used instead.

 

To get to something testable I would first look at the target system

as at some stage a system under test and ask what kinds of tests

are feasible. What overall test results of such tests could be made

a criterion for conformance QA? People tend to want conformance

to be a 'yes'/'no' rather than 'this yes', 'this no', 'this yes'. So there

needs to be an overall way to summate the results of a set of tests

and it needs to be clear what those tests should be for conformance.

This is where test assertions can help as they sit tie spec statements

to resulting conformance tests.

 

For a subset there seems to me to be a need to go further than just

schema conformance (validity). For the overall UBL the validity is

about as far as you can go because you have to allow for all kinds

of subsets between the system and UBL conformance which add

their own requirements. So the UBL standard requirements have to

be vague: You don't have to have all these BIEs in a document

but any you do have have to look like this. For a subset profile on the

other hand you have scope (and perhaps a duty) ot be more

specific about what BIEs a document can contain and what BIEs,

perhaps, a system MUST be 'able' to put in a document or read

from a document. So here there is, I suggest, the opportunity to be

more precise than just 'must conform to schema X' because you can

(and perhaps should) go beyond the document to place requirements

on the sending and (less so, by Postel's law) receiving systems. So

you will therefore want to have a spec which covers these systems.

So the spec should lend itself to testable test assertions for these

systems. So it helps to include the test assertion writing in the spec

writing process because it is ultimately the normative statements

which determine the test assertions and the confromance tests are

going to be based, if possible, on the test assertions as testable

renderings of those normative statements.

 

Best regards

----

Stephen D Green

On 1 August 2011 17:49, Jon Bosak <bosak@pinax.com> wrote:

Hello Stephen,

I like the idea of a basic subset (always have), and I'm happy to have
the benefit of your work on this.

A question about the test assertion approach, however.  I'm having
trouble seeing how these assertions go beyond just saying "you have to
validate against the subset schema, and in addition, these optional UBL
elements are mandatory."  (In which case, of course, you could just
modify the subset schema to that effect.)  Could you help me understand
this?

Jon

Stephen D Green wrote:

I would refine the previous comment to say that the previous receiving system
conformance clause might not have been fully testable (since not every possible
document received will be tested). I could refine this to say that a receivng system
MUST be able to receive a document which contains every element in the subset
and every multiple cardinality element twice or more times in the document.
 I would suggest that continuing along the lines of a conformance test centric subset profile there might be consideration of test assertions for such a
profile, again considering testability of the assertions and the normative
statements from which they are derived.
 Here is an example of a test assertion set for, say, a conformance clause relating to
the invoice document type (there might be a clause for each type to allow
implementations to implement just one or more document types and still
be conformant).
 It is based on an example profile which somehow lists every element (ignoring
attributes for conformance requirements except insofar as they are ever mandatory
for the UBL standard schema validity for that document type)
 e.g.
 Where the profile for the subsets contains statements like this:
...
Statement INV004: A subset sending system which can send a subset invoice MUST be able to send a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing element /in:Invoice/cbc:ID
 Statement INV005: A subset sending system which can send a subset invoice MUST be able to send a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing element /in:Invoice/cbc:CopyIndicator
... one such statement for each element in the subset or a table to the same effect or some other
blanket statement to this effect including reference to an overall schema for the subset
...
 The conformance clause for subset invoice sending systems would require conformance to
all these statements. Another clause for receiving systems might require conformance to
a statement that receiving systes receiving a subset invoice be able to receive one with all
elements in the subset (including multiple occurences where the schema includes multiple
occurences).
 Test assertions could be presented in the form of markup like this
-- here using OASIS (in progress) Test Assertion Markup Language:


<!-- one set of normative statements and corresponding test assertions for every document type in the subset and for each of these sets one corresponding conformance clause for sending systems (for that document type) and one for recieving systems (for that document type) -->
<testAssertionSet xmlns="http://docs.oasis-open.org/ns/tag/taml-201002/" setid="ubl-subset.example.ta-set.1" setname="Invoice Subset">
<common>
...
<!-- need some bindings for prefixes used in the XPath expressions -->
</common>
...
<testAssertion id="ubl-invoice-subset.example.ta.s4">
<!-- the actual nomative source might be a table or might rely on a schema to list all elements in the subset (attributes too but these might be optional) -->
<normativeSource><derivedSourceItem documentId="..." resourceProvenanceId="..." uri="...">A subset sending system which can send a subset invoice MUST be able to send a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing element /in:Invoice/cbc:ID</derivedSourceItem></normativeSource>
<target>subset sending system</target>
<prerequisite>can send a subset document valid according to the OASIS standard UBL 2.1 Invoice schema</prerequisite>
<predicate>can send a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing element /in:Invoice/cbc:ID</predicate>
<prescription level="mandatory"/>
</testAssertion>
<testAssertion id="ubl-invoice-subset.example.ta.s5">
<normativeSource><derivedSourceItem documentId="..." resourceProvenanceId="..." uri="...">...</derivedSourceItem></normativeSource>
<target>subset sending system</target>
<prerequisite>can send a subset document valid according to the OASIS standard UBL 2.1 Invoice schema</prerequisite>
<predicate>can send a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing element /in:Invoice/cbc:CopyIndicator</predicate>
<prescription level="mandatory"/>
</testAssertion>
<testAssertion id="ubl-invoice-subset.example.ta.s6">
<normativeSource><derivedSourceItem documentId="..." resourceProvenanceId="..." uri="...">...</derivedSourceItem></normativeSource>
<target>subset sending system</target>
<prerequisite>can send a subset document valid according to the OASIS standard UBL 2.1 Invoice schema</prerequisite>
<predicate>can send a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing element /in:Invoice/cbc:UUID</predicate>
<prescription level="mandatory"/>
</testAssertion>

<!-- one normative statement and corresponding test assertion for every element in the subset for that document type -->
...

<testAssertion id="ubl-invoice-subset.example.ta.r1">
<normativeSource><derivedSourceItem documentId="..." resourceProvenanceId="..." uri="...">...</derivedSourceItem></normativeSource>
<target>subset receiving system</target>
<prerequisite>can receive a subset document valid according to the OASIS standard UBL 2.1 Invoice schema</prerequisite>
<predicate>can receive a document valid according to the OASIS standard UBL 2.1 Invoice schema and containing every single-occurence element in that subset once and multiple-occurence element in that document twice or more</predicate>
<prescription level="mandatory"/>
</testAssertion>
...

</testAssertionSet>

 Best regards
 ----
Stephen D Green

On 30 July 2011 18:40, Stephen D Green <stephengreenubl@gmail.com <mailto:stephengreenubl@gmail.com>> wrote:

   Suggestion for a basis for subset profile conformance clauses
             In the UBL 1.0 Small Business Subset were clauses which
   I found later to be very difficult to test precisely and this
   might make conformance testing and conformance claims
   problematic. The introduction of conformance clauses in
   recent standard specifications is to aid interoperability and
   promote adoption through the clarification of what it means
   for an implementation to conform to a standard specification.
        To promote adoption of subsets for the OASIS Universal
   Business Language it is important to include a conformance
   clause of set of clauses and I would suggest that a subset
   conformance clause should target more than just the UBL
   documents (Invoice, Order, etc) themselves but also there
   should be a conformance clause for a sending system, one
   for a receiving system instead of or in addition to the clause
   for conformance of the documents themselves. I would like
   to suggest as a basis for the clause for the conformance
   target of sending system conformance to a set of statements
   which amount to the sending system being able to send a
   certain set of BIEs in a given document, as defined by the
   subset schema for that document. I would suggest that a
   basis for a clause for the target of a receiving system would     be a conformance clause requiring that the system MUST
   NOT reject a document merely because of the presence in
   it of any of the BIEs as defined by the subset schema (or
   list of BIEs, e.g. given as a set of XPath expressions).
        As a simplistic example, if a subset contains BIEs X,Y,Z of
   which Y and Z are mandatory and X is optional, there could
   be a set of specification requirements to the effect that the
   sender system MUST be able to send all BIEs X, Y and Z
   in sending that particular document (even though only Y
   and Z are mandatory in any given document of that type).
   A conformance clause for the target of the sending system
   would mandate these particular statements as necessary
   for the conformance of that system. A set of statements that
   target the receiving system would require that the system
   MUST NOT reject a document of that type merely because
   it contains BIE X or Y or Z. A conformance clause for the
   receiving system would make the statements mandatory
   for conformance by such a system. There would be a set of     statements for each conformance target that the documents
   of that type MUST contain BIEs Y and Z because these are
   mandatory.
        Such conformance clauses and this focus on testability may
   help to promote adoption of any given subset and contribute
   to successful adoption and interoperable implementations.
             Best regards
        ----
   Stephen D Green