Re: [tag] Test Assertion Modeling - comments, etc

[stephen.green@systml.co.uk]

How about this as a test assertion for the SBS rule #1

The TA is targeted at / relevant to (maybe it should say so)
an application which sends a UBL SBS document

An application conforming to the SBS which sends a UBL document,
if it allows the document to include elements not defined in the
SBS specification as being required, it SHOULD provide a warning
that such an element might not be processed by or visible to the
receiver of the document.

-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting stephen.green@systml.co.uk:

> Having to rethink the idea of test assertions for this SBS rule -
>
> The document which conforms to the spec, a UBL SBS document,
> is a kind of spec itself - it specifies some business, whether
> a payment that needs to be made in conformance to an invoice
> or a supply of goods or services in conformance to an order.
> The spec is kind of saying with this SBS rule that certain
> specified parts of that document should be regarded as
> informative only. Now how on earth can you test for that? :-)
>
> Maybe you can't in terms of software at all. Maybe it's just
> a legal thing and not for testing at all. How do you test
> that something is being treated as normative or informative
> in a particular implementation? In this case it is a matter
> of testing that say an order is part informative and part
> normative and that the informative part, if present at all,
> does not require normative treatment. That can only be a
> matter or business and legal vigilance can't it - not a
> software testing thing at all perhaps. A lesson seems to be
> here though that not everything need be tested for it to be
> be worth writing in test assertion formal language - at least
> not tested with typical software tests. The test assertion
> technique might be used to facilitate business and legal
> implementation of a business document and software which
> produced it in a way that software testing itself cannot do.
>
> -- 
> Stephen Green
>
> Partner
> SystML, http://www.systml.co.uk
> Tel: +44 (0) 117 9541606
>
> http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice
>
>
>
> Quoting stephen.green@systml.co.uk:
>
>> Thanks. Excellent.
>>
>> Of course, the tests can be limited to a business process
>> of which the document under test is a part. In a way this
>> binds the testing not only to the business document but
>> also to it's business process - an interesting aspect of the
>> SBS and other business language profiles perhaps.
>>
>> So here the TA has to take into account not just the subset
>> but also the business process of which it participates. So
>> the spec can't really include the TAs as such unless it can
>> generalise them for all anticipated business processes or
>> somehow make the assertions independant of the processes.
>>
>> I did imagine there will be quite a few layers to test and
>> also that these might inter-relate in some way - is that a
>> problem though? inter-relating layers of validation? A test
>> in one layer might depend on results from another layer.
>> This means there is a sequence in testing with dependencies
>> even crossing between layers of testing. What happens if
>> any tests are mutually dependant or if two layers of tests
>> are mutually dependant? Is it necessary to declare all depend-
>> encies somehow in the TAs? What if dependencies existed
>> between different sets of assertions (if the assertions were
>> split for different audiences as I think you suggest) but
>> not all test sets were used as supplied. In that case making
>> dependencies explicit would show up the mismatches. But
>> if the test assertions were put into the same file then ID
>> and IDREF could maybe be used to assure of the integrity of
>> dependencies between id and ref to some extent when
>> validating the TA document. Else XInclude or the like might be
>> used if dependencies crossed file boundaries.
>>
>> -- 
>> Stephen Green
>>
>> Partner
>> SystML, http://www.systml.co.uk
>> Tel: +44 (0) 117 9541606
>>
>> http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice
>>
>>
>>
>> Quoting Dave Pawson <dave.pawson@gmail.com>:
>>
>>> Wow. Quite a test.... or more correctly a test sequence?
>>>
>>> On 14/08/07, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote:
>>>> Hi. Thanks Jacques and David and for comments.
>>>>
>>>> Yes, the testing of this item for the SBS profile for UBL
>>>> which relates to the Sender (there is another rule which
>>>> relates to the Receiver) could be like the following:
>>>>
>>>
>>>
>>>> Essentially it is a business rule, albeit of an unusually
>>>> technical nature but there are still tests which can be
>>>> applied. Some such tests might be best applied by technical
>>>> auditors though, even by legal experts in some cases.
>>>>
>>>> Not sure how these test requirements would be expressed as
>>>> test assertions though since maybe the target audience of the
>>>> test assertion would be a technical auditor or legal expert,
>>>> even a legal court (if say there was a large sum unpaid
>>>> because the receiver had ignored some payment terms or tax
>>>> amounts which were external to the subset) eventually.
>>>
>>> Layering needed?
>>> A single test (pass or fail).
>>> A test group (again pass or fail with test results)
>>> If test group passes, output the message in an appropriate format?
>>> Appropriate to the audience that is.
>>> Groups layered into supergroups as needed,
>>> culminating in a complete application.
>>> The end resut is the summation (done automatically, not collated by the
>>> application, which cheats) of the test groups. Again either pass or fail.
>>>
>>>
>>>
>>> Nasty:
>>> Writing tests with built in debug.
>>> I.e. run it with the debug flag set and all the techie garbage flows.
>>> Switch it off and the legal eagle sees pass, or fail, test number 1,290.
>>>
>>> If you've multiple audiences, the test application layer needs parameters.
>>>
>>>
>>> regards
>>>
>>>
>>>
>>>
>>> --
>>> Dave Pawson
>>> XSLT XSL-FO FAQ.
>>> http://www.dpawson.co.uk
>>>