Re: [tag] Test Assertion Modeling - comments, etc

[stephen.green@systml.co.uk]

Thanks David

It's probably important to balance these two views (and I could happily
accept yours that the markup and schema validity can provide the mainstay
of a standard, less so tool validity). OASIS standards traditionally major
on markup ("structured information") so it especially applies to OASIS.

Tool creators do need their own validity test but I gather you are
proposing that the essense of such tests should be schema validity of
XML output where applicable. This concept is kind of growing on me :-)

The concepts of event-behaviour need to be overlaid on this though for
standards such as messaging ones where the system behaviour is of the
essense. APIs will likely fall into the latter camp too of course.

Maybe we now need some examples to balance the SBS one where there is no
requirement or applicability of an emphasis on instance validity.

Best regards
-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting Dave Pawson <dave.pawson@gmail.com>:

> On 17/08/07, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote:
>
>> I'll try to defend my stance on validation of both tool and markup.
>
> My opinion is that tool validation is out of scope.
>
>> Here's a quote from the html spec for 4.0:
>
>> Lots of description of tool use here.
>
> Tool use. Now find an application validator? No. The validation
> is by schema on the (x)html produced.
>
>>
>> The browser presenting the html can surely be validated in how
>> it does so as well as the html itself being validated against the DTD.
>
> Find one. I don't think you will. It's like judging how you won the race.
> The result is won/2nd etc.
>
> The focus is generally on what is produced.
>
>
>
>> With UBL there is little point just describing the markup although
>> an attempt has really been made to make conformance a matter of validity
>> of an instance against the schema.
>
> I.e. the conformance is of an instance to the schema.
>
>
> If my app insists on using an order
>> number as the invoice number element content I can't claim my app is
>> compliant even though the instances will still validate. So UBL has
>> a focus on saying the definitions of the elements are normative too.
>
> Again Schema based, on what is produced. Data types are included in
> the schema.
>
>> I don't think a test suite can validate the content in those terms though
>> apart from schema validity and maybe some calculation business rule
>> validity too
>
> You just need different tools for business rules. Have a look at Schematron.
>
>
>
>>
>> This kind of illustrates that not only must the document be valid
>> according to the spec in its structure but maybe with regard to its
>> semantics (where there are normative rules, etc)
>
> I don't know of any semantics check on xml.
> Thats generally where people come in.
>
>  and moreover that an
>> end user can be using the standard in a way which is valid or not.
>> So the app has to test user input according to test assertions, say,
>> and a test suite might test the app according to the same or other
>> test assertions perhaps and the test suites can be tested using test
>> assertions too.
>
> You're getting too abstract for me here. Any examples? And how far
> back into meta meta land do we go?
>
> The documents can be tested and its tests tested too.
>
> Usually against a test plan. Answer the  question, 'does this test
> prove compliance to the requirement'.
>
>> Lots of tests and test assertions and layers and so on. If the standard
>> is well written it will foster all of this to ensure best implementation
>> and end user experience. A tall order though.
>
>
> I'll close on this thread now. I think what has been made clear is the
> need for a clear scope.
> AFAIK, there isn't one yet.
>
> regards
>
>
>
>
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
>