Re: [tag] Test Assertion Modeling - comments, etc

[stephen.green@systml.co.uk]

Mmm. I guess we are tending to a conclusion that some aspects
of the SBS example are not likely to become test assertions.
On the other hand, some aspects might still benefit from an
attempt to consider how to reword the spec item to facilitate
the writing of a test assertion. At some point that rewording
starts to look a bit like a test assertion but not enough like
one to actually be regarded as one. Then to make it into a TA
it is necessary to structure it according to an accepted TA
pattern and, here I get a bit lost, maybe do a bit more than
just structure it...?

Does the pattern structure provide enough to make a spec item
into a TA? If not what else is needed (given this SBS example)?
But if so then maybe the spec item could be so structured anyway
couldn't it? Wouldn't that improve the spec item in some way?
So could the TA structure we come up with (hopefully) not also
take on a role in spec production a bit like docbook (which at
least UBL TC uses for its specs of late)? Maybe it would be a
matter of even taking docbook and adding a TA extension or
profile (if docbook allows for that). I think OASIS has some
XSLT to turn docbook into a spec. Maybe that could be adapted
to recognise TA inclusions in the docbook XML and handle it
appropriately.

Back to the SBS example though, I'm a little apprehensive about
the way a necessarily vague conformance clause might impact on
TAs. I guess there are all sorts of way of implementing a markup
standard for instance (look at all the types of products which
variously implement HTML, XHTML or docbook for example - browsers,
XSLT apps, mappers, editors, pdf generators, office apps, etc)
and each would perhaps need a different conformance clause and
consequently different TA lists to test such conformance. Or the
conformance could be defined just as schema validity and the
products left on their own to work out how they should otherwise
conform and produce TAs themselves with the potential loss of
interoperability, etc. I'm a bit worried that a trend to make TAs
as part of standard design might either be biased toward just the
most obvious types of implementation and ignore future innovation
and that it might therefore make conformance too specific and rigid
with such applications in mind. Not a problem so much for APIs though
where the conformance is almost exclusively a matter for applications
and therefore TAs are more predictable and clear cut. Maybe this
is a groundless concern though - after all XForms seems to be suited
to its W3C test suite despite XForms being a markup language and not
an API.

-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting Dave Pawson <dave.pawson@gmail.com>:

> On 14/08/07, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote:
>> Having to rethink the idea of test assertions for this SBS rule -
> From your comments it appears that you are trying to use a test assertion
> for all levels. I'm clear that's not right.
>
> At some level a Turing complete language will be required.
> It's unreasonable to lump all this into a declarative assertion statement(s).
>
>
>
>>
>> The document which conforms to the spec, a UBL SBS document,
>> is a kind of spec itself - it specifies some business, whether
>> a payment that needs to be made in conformance to an invoice
>> or a supply of goods or services in conformance to an order.
>> The spec is kind of saying with this SBS rule that certain
>> specified parts of that document should be regarded as
>> informative only. Now how on earth can you test for that? :-)
>
> If it's informative it's not tested? Tests only test firm requirements.
> Further, only requirements which are verifiable.
>
>
>
> At some point a clear scoping document/paragraph/section will be
> needed to draw a line above which TAG is not interested.
>
> From this thread I'd suggest that the line is close, but below, the
> application layer?
>
>
> regards
>
>
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
>