RE: [tag] Test Assertion Modeling - comments, etc

[stephen.green@systml.co.uk]

Hi. Thanks Jacques and David and for comments.

Yes, the testing of this item for the SBS profile for UBL
which relates to the Sender (there is another rule which
relates to the Receiver) could be like the following:

If there is any element in a message from a Sender which
is 'external' to the subset profile (one of the elements
the profile gives a property of 'external to the subset',
say) then there need to be tests to determine if the
sender is permitting that it be ignored. This is indeed
testable but a little unusual compared to API tests.

If the initial or any subsequent communication made a
reference to information in the initial message which was
external to the subset then a warning could be issued.
So one test would be to monitor all other parts of the
message and all subsequent messages which might relate to it
for such eventualities which would have to result in a
warning.

Another test would be to cycle through all communications
and requirements prior to the message to check that these
did not act as impositions from the Sender on the Receiver
to process external parts of the message.

Essentially it is a business rule, albeit of an unusually
technical nature but there are still tests which can be
applied. Some such tests might be best applied by technical
auditors though, even by legal experts in some cases.

Not sure how these test requirements would be expressed as
test assertions though since maybe the target audience of the
test assertion would be a technical auditor or legal expert,
even a legal court (if say there was a large sum unpaid
because the receiver had ignored some payment terms or tax
amounts which were external to the subset) eventually.

Regarding structure I'd imagine a better way than prose to
relate the whole subset, including this rule, might be to have
a structured test assertion in mark-up perhaps a bit like
docbook in design with sections within sections, etc where
the artefact near the top level could be either the original
spec of UBL or the formal definition of the subset/profile.
There might be just a kind of overall property or event-behaviour
statement at that level a bit like a conformance clause then
levels below could be each tied to individual elements of the
XML messages - with the artefact being the element and a property
being some term like 'external to subset'. There sould need to
be a way to define at top level (or even a definition in the
actual TAG model spec for such and other common terms/patterns)
what 'external to subset' means in terms of test assertions.
This would be a potential basis for a TA template or type for
subsets and maybe profiles in general. Some ready made values
for potential artefacts might be 'profile', 'specification',
'module', 'level' and 'item'. Some ready made values for
their properties might be 'normative', 'informative', 'optional';
plus some like 'deprecated' or 'warn against' (not sure what term
here) which the SBS might use instead of 'external to subset'.

I'd thought of adding this idea to my presentation but it doesn't
really fit the topic I'd chosen on merging the model types (cp
model types in UML).

Regards

-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting "Durand, Jacques R." <JDurand@us.fujitsu.com>:

> Comments inline, sorry for late reply
>
> Cheers,
> -Jacques
>
>
> -----Original Message-----
> From: stephen.green@systml.co.uk [mailto:stephen.green@systml.co.uk
> <mailto:stephen.green@systml.co.uk> ]
> Sent: Monday, July 30, 2007 4:46 PM
> To: TAG
> Subject: [tag] Test Assertion Modeling - comments, etc
>
> I've had a few thoughts on the material on the wiki before the meeting
> which I wasn't confident about putting on the wiki as yet so I thought
> I'd send a few to the list.
>
> It strikes me that only simpler spec items can be represented by the
> Predicative or the Event-behaviour models and TCs might be challenged
> too much with more complex items. I was considering the UBL SBS profile
> as an example and it has two normative rules which are succinct in prose
> but would, I think, be very long and complex if expressed as a list of
> TAs and even then could not be so expressed using the models, I fear.
> They would need something more like an ontology even though they are
> short. Rule one states:
>
> "Parties sending UBL documents to parties specified as receivers of a
> UBL 1.0 Small Business Subset (SBS) document or documents SHOULD NOT
> require the same receiving party to process any part of the UBL document
> which is external to the specified SBS."
> http://docs.oasis-open.org/ubl/cs-UBL-1.0-SBS-1.0
> <http://docs.oasis-open.org/ubl/cs-UBL-1.0-SBS-1.0>  - section 2.1.1
>
>
> <jacques> one problem with above statements (and what makes them of
> interest for our TA guide too) , is that they are not prescriptive of a
> real requirement,  but to the contrary, are kind of
> "requirement-waivers" (relaxing one party from doing something). Or, put
> in other words: I see two parties involved here: sender and receiver.
> The requirement is more on the Sender side, to tolerate Receiver not
> doing some processing.
> NOTE: the statement above seems to me stronger than "The Receiver MAY
> ignore..." because its about a Sender that SHOULD NOT require the
> Receiver to..." So it sounds in fact as a requirenment for the Sender -
> subject to a TA - while in the former case, it would be hard to derive a
> TA (or more context would be needed from the rest of the spec).
>
> My first concern then, for writing a TA, would be: is there an
> observable sign that a Sender has been "accepting"  of a Receiver NOT
> doing the processing? That sends me back to the assumed [abstract] Test
> Environment here. For this to be "testable" even abstractly by a TA, the
> test environment assumes:
> - that there are two communicating parties, Sender and [SBS] Receiver.
> - that there exists some operation, or output, supported by a Sender
> under test, that makes it possible to observe that it "tolerates" a
> Receiver that does not process parts external to the SBS.
> I believe here that the target for the above UBL requirement, is the
> Sender. The Receiver is free to do what it wants (process or not
> process). A system under test implementing a test case for this TA could
> simulate Receiver behavior to check how the Sender behaves.
>
>
>
> So it seems to me there need to be onotological statements defining
> party sender subset specified external ... etc even before a list of
> test assertions can be modelled for this rule (and such test assertions
> would be, after all, just formal rules which could then be turned into
> actual software tests).
>
> <jacques> isn't the specification itself supposed to be explicit on how
> to define "parts external to an SBS"? If there remains any ambiguity or
> doubt on the practicality to check this in a TA, the test environment
> again should suggest some way to control / check this.
>
> Then there are issues like - how to model 'SHOULD', can there be
> artifacts of artifacts, properties of properties, and can this
> complexity even be modelled as a set of predicatives, etc?
>
> <jacques> one way to handle SHOULD or RECOMMENDED reqs, is to treat them
> like MUST in the TA, but in case of failure, produce a "warning"
> instead of a "fail"...
>
> Maybe this sort of item just needs to be prose with minimal structure
> (list?).
>
> I did start to get an idea of what a set of predicatives might look like
> though. I modelled them as generalaised XML nodes, looking at two
> factors
>
> 1. elements vs attributes - mainly elements?
> 2. nesting vs listing - with XOR, etc as attributes?
>
> and one way to model the predicative features, mapping these to the XML
> is:
>
> <talist>
>         <preCondition OR>
>                 <preCondition/>
>         </preCondition>
>         <preCondition AND>
>                 <preCondition/>
>         </preCondition>
>         <ta>
>                 <preCondition XOR>
>                         <preCondition/>
>                 </preCondition>
>                 <preCondition/>
>                 <preCondition/>
>                 <preCondition/>
>                 <artifact OR>
>                         <property/>
>                         <property/>
>                         <property/>
>                         <artifact XOR>
>                                 <property/>
>                                 <property/>
>                                 <artifact AND>
>                                         <property/>
>                                         <artifact>
>                                                 <property/>
>                                                 <property/>
>                                         </artifact>
>                                 </artifact>
>                         </artifact>
>                 </artifact>
>         </ta>
>         <ta>
>                 <preCondition XOR>
>                         <preCondition/>
>                 </preCondition>
>                 <preCondition/>
>                 <preCondition/>
>                 <preCondition/>
>                 <artifact OR>
>                         <property/>
>                         <property/>
>                         <property/>
>                         <artifact XOR>
>                                 <property/>
>                                 <property/>
>                                 <artifact AND>
>                                         <property/>
>                                         <artifact>
>                                                 <property/>
>                                                 <property/>
>                                         </artifact>
>                                 </artifact>
>                         </artifact>
>                 </artifact>
>         </ta>
> </talist>
>
> In its simplest form, either model (predicative or event-behaviour)
> seems just to be a bulleted list or perhaps with some nesting and
> distinction of a few types of bullets such as artifact, property and
> maybe precondition (being borrowed from event-behaviour?)
>
> <jacques> there are quite a few markups out there for logical
> expressions that we should consider reusing (RuleML, I think OCL too)
>
> I've lots more thoughts, issues, comments, etc but maybe I'm so off
> track that they are best kept in hand.
>
> Best regards
>
> --
> Stephen Green
>
> Partner
> SystML, http://www.systml.co.uk <http://www.systml.co.uk>
> Tel: +44 (0) 117 9541606
>
> http://www.biblegateway.com/passage/?search=matthew+22:37
> <http://www.biblegateway.com/passage/?search=matthew+22:37>  .. and
> voice
>
>
>
>
>
>
>
>
>