RE: [tag] Test Assertion Modeling - comments, etc

["Durand, Jacques R." <JDurand@us.fujitsu.com>]

Comments inline, sorry for late reply

Cheers,
-Jacques


-----Original Message-----
From: stephen.green@systml.co.uk [mailto:stephen.green@systml.co.uk]
Sent: Monday, July 30, 2007 4:46 PM
To: TAG
Subject: [tag] Test Assertion Modeling - comments, etc

I've had a few thoughts on the material on the wiki before the meeting which I wasn't confident about putting on the wiki as yet so I thought I'd send a few to the list.

It strikes me that only simpler spec items can be represented by the Predicative or the Event-behaviour models and TCs might be challenged too much with more complex items. I was considering the UBL SBS profile as an example and it has two normative rules which are succinct in prose but would, I think, be very long and complex if expressed as a list of TAs and even then could not be so expressed using the models, I fear. They would need something more like an ontology even though they are short. Rule one states:

"Parties sending UBL documents to parties specified as receivers of a UBL 1.0 Small Business Subset (SBS) document or documents SHOULD NOT require the same receiving party to process any part of the UBL document which is external to the specified SBS."
http://docs.oasis-open.org/ubl/cs-UBL-1.0-SBS-1.0 - section 2.1.1

 

<jacques> one problem with above statements (and what makes them of interest for our TA guide too) , is that they are not prescriptive of a real requirement,  but to the contrary, are kind of "requirement-waivers" (relaxing one party from doing something). Or, put in other words: I see two parties involved here: sender and receiver. The requirement is more on the Sender side, to tolerate Receiver not doing some processing.

NOTE: the statement above seems to me stronger than "The Receiver MAY ignore..." because its about a Sender that SHOULD NOT require the Receiver to..." So it sounds in fact as a requirenment for the Sender - subject to a TA - while in the former case, it would be hard to derive a TA (or more context would be needed from the rest of the spec).

 

My first concern then, for writing a TA, would be: is there an observable sign that a Sender has been "accepting"  of a Receiver NOT doing the processing? That sends me back to the assumed [abstract] Test Environment here. For this to be "testable" even abstractly by a TA, the test environment assumes:

- that there are two communicating parties, Sender and [SBS] Receiver.

- that there exists some operation, or output, supported by a Sender under test, that makes it possible to observe that it "tolerates" a Receiver that does not process parts external to the SBS.

I believe here that the target for the above UBL requirement, is the Sender. The Receiver is free to do what it wants (process or not process). A system under test implementing a test case for this TA could simulate Receiver behavior to check how the Sender behaves.

So it seems to me there need to be onotological statements defining party sender subset specified external ... etc even before a list of test assertions can be modelled for this rule (and such test assertions would be, after all, just formal rules which could then be turned into actual software tests).

<jacques> isn't the specification itself supposed to be explicit on how to define "parts external to an SBS"? If there remains any ambiguity or doubt on the practicality to check this in a TA, the test environment again should suggest some way to control / check this.

Then there are issues like - how to model 'SHOULD', can there be artifacts of artifacts, properties of properties, and can this complexity even be modelled as a set of predicatives, etc?

<jacques> one way to handle SHOULD or RECOMMENDED reqs, is to treat them like MUST in the TA, but in case of failure, produce a "warning"  instead of a "fail"...

Maybe this sort of item just needs to be prose with minimal structure (list?).

I did start to get an idea of what a set of predicatives might look like though. I modelled them as generalaised XML nodes, looking at two factors

1. elements vs attributes - mainly elements?
2. nesting vs listing - with XOR, etc as attributes?

and one way to model the predicative features, mapping these to the XML is:

<talist>
        <preCondition OR>
                <preCondition/>
        </preCondition>
        <preCondition AND>
                <preCondition/>
        </preCondition>
        <ta>
                <preCondition XOR>
                        <preCondition/>
                </preCondition>
                <preCondition/>
                <preCondition/>
                <preCondition/>
                <artifact OR>
                        <property/>
                        <property/>
                        <property/>
                        <artifact XOR>
                                <property/>
                                <property/>
                                <artifact AND>
                                        <property/>
                                        <artifact>
                                                <property/>
                                                <property/>
                                        </artifact>
                                </artifact>
                        </artifact>
                </artifact>
        </ta>
        <ta>
                <preCondition XOR>
                        <preCondition/>
                </preCondition>
                <preCondition/>
                <preCondition/>
                <preCondition/>
                <artifact OR>
                        <property/>
                        <property/>
                        <property/>
                        <artifact XOR>
                                <property/>
                                <property/>
                                <artifact AND>
                                        <property/>
                                        <artifact>
                                                <property/>
                                                <property/>
                                        </artifact>
                                </artifact>
                        </artifact>
                </artifact>
        </ta>
</talist>

In its simplest form, either model (predicative or event-behaviour) seems just to be a bulleted list or perhaps with some nesting and distinction of a few types of bullets such as artifact, property and maybe precondition (being borrowed from event-behaviour?)

<jacques> there are quite a few markups out there for logical expressions that we should consider reusing (RuleML, I think OCL too)

I've lots more thoughts, issues, comments, etc but maybe I'm so off track that they are best kept in hand.

Best regards

--
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice