Re: [tag] Test Assertion Modeling - comments, etc

[stephen.green@systml.co.uk]

Having to rethink the idea of test assertions for this SBS rule -

The document which conforms to the spec, a UBL SBS document,
is a kind of spec itself - it specifies some business, whether
a payment that needs to be made in conformance to an invoice
or a supply of goods or services in conformance to an order.
The spec is kind of saying with this SBS rule that certain
specified parts of that document should be regarded as
informative only. Now how on earth can you test for that? :-)

Maybe you can't in terms of software at all. Maybe it's just
a legal thing and not for testing at all. How do you test
that something is being treated as normative or informative
in a particular implementation? In this case it is a matter
of testing that say an order is part informative and part
normative and that the informative part, if present at all,
does not require normative treatment. That can only be a
matter or business and legal vigilance can't it - not a
software testing thing at all perhaps. A lesson seems to be
here though that not everything need be tested for it to be
be worth writing in test assertion formal language - at least
not tested with typical software tests. The test assertion
technique might be used to facilitate business and legal
implementation of a business document and software which
produced it in a way that software testing itself cannot do.

-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting stephen.green@systml.co.uk:

> Thanks. Excellent.
>
> Of course, the tests can be limited to a business process
> of which the document under test is a part. In a way this
> binds the testing not only to the business document but
> also to it's business process - an interesting aspect of the
> SBS and other business language profiles perhaps.
>
> So here the TA has to take into account not just the subset
> but also the business process of which it participates. So
> the spec can't really include the TAs as such unless it can
> generalise them for all anticipated business processes or
> somehow make the assertions independant of the processes.
>
> I did imagine there will be quite a few layers to test and
> also that these might inter-relate in some way - is that a
> problem though? inter-relating layers of validation? A test
> in one layer might depend on results from another layer.
> This means there is a sequence in testing with dependencies
> even crossing between layers of testing. What happens if
> any tests are mutually dependant or if two layers of tests
> are mutually dependant? Is it necessary to declare all depend-
> encies somehow in the TAs? What if dependencies existed
> between different sets of assertions (if the assertions were
> split for different audiences as I think you suggest) but
> not all test sets were used as supplied. In that case making
> dependencies explicit would show up the mismatches. But
> if the test assertions were put into the same file then ID
> and IDREF could maybe be used to assure of the integrity of
> dependencies between id and ref to some extent when
> validating the TA document. Else XInclude or the like might be
> used if dependencies crossed file boundaries.
>
> -- 
> Stephen Green
>
> Partner
> SystML, http://www.systml.co.uk
> Tel: +44 (0) 117 9541606
>
> http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice
>
>
>
> Quoting Dave Pawson <dave.pawson@gmail.com>:
>
>> Wow. Quite a test.... or more correctly a test sequence?
>>
>> On 14/08/07, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote:
>>> Hi. Thanks Jacques and David and for comments.
>>>
>>> Yes, the testing of this item for the SBS profile for UBL
>>> which relates to the Sender (there is another rule which
>>> relates to the Receiver) could be like the following:
>>>
>>
>>
>>> Essentially it is a business rule, albeit of an unusually
>>> technical nature but there are still tests which can be
>>> applied. Some such tests might be best applied by technical
>>> auditors though, even by legal experts in some cases.
>>>
>>> Not sure how these test requirements would be expressed as
>>> test assertions though since maybe the target audience of the
>>> test assertion would be a technical auditor or legal expert,
>>> even a legal court (if say there was a large sum unpaid
>>> because the receiver had ignored some payment terms or tax
>>> amounts which were external to the subset) eventually.
>>
>> Layering needed?
>> A single test (pass or fail).
>> A test group (again pass or fail with test results)
>> If test group passes, output the message in an appropriate format?
>> Appropriate to the audience that is.
>> Groups layered into supergroups as needed,
>> culminating in a complete application.
>> The end resut is the summation (done automatically, not collated by the
>> application, which cheats) of the test groups. Again either pass or fail.
>>
>>
>>
>> Nasty:
>>  Writing tests with built in debug.
>>  I.e. run it with the debug flag set and all the techie garbage flows.
>>  Switch it off and the legal eagle sees pass, or fail, test number 1,290.
>>
>> If you've multiple audiences, the test application layer needs parameters.
>>
>>
>> regards
>>
>>
>>
>>
>> --
>> Dave Pawson
>> XSLT XSL-FO FAQ.
>> http://www.dpawson.co.uk
>>