Re: [tag] Test Assertion Modeling - comments, etc

[stephen.green@systml.co.uk]

Hi David

I'd better own up to having been the one mixing app and markup
requirements - Serm was quoting me.

I'll try to defend my stance on validation of both tool and markup.

[p.s. Sorry it's a bit lengthy - I just wanted to get a few thoughts down.]

Here's a quote from the html spec for 4.0:
"The body of a document contains the document's content. The content  
may be presented by a user agent in a variety of ways. For example,  
for visual browsers, you can think of the body as a canvas where the  
content appears: text, images, colors, graphics, etc. For audio user  
agents, the same content may be spoken."
http://www.w3.org/TR/1998/REC-html40-19980424/struct/global.html#h-7.5.1

Lots of description of tool use here.

The browser presenting the html can surely be validated in how
it does so as well as the html itself being validated against the DTD.

So the W3C working group had to assume certain essentials about
what the apps/tools would do with the product (html) of the standard.

Take XForms: the working group hasn't just produced an XML schema for
XForms to enable validation and formal description of the instances
but also has produced a nice and thorough test suite for the browsers
supporting it - where they assume a browser is the main app to use it
(while OOo also uses it to some extent and that isn't the target, I
think, of the tests).
http://www.w3.org/MarkUp/Forms/Test/XForms1.0/Edition2/front_html/XF102edTestSuite.html

With UBL there is little point just describing the markup although
an attempt has really been made to make conformance a matter of validity
of an instance against the schema. If my app insists on using an order
number as the invoice number element content I can't claim my app is
compliant even though the instances will still validate. So UBL has
a focus on saying the definitions of the elements are normative too.
I don't think a test suite can validate the content in those terms though
apart from schema validity and maybe some calculation business rule
validity too (where normative rules exist, say in an 'implementation guide'
- read 'profile' or 'level' I guess).

This kind of illustrates that not only must the document be valid
according to the spec in its structure but maybe with regard to its
semantics (where there are normative rules, etc) and moreover that an
end user can be using the standard in a way which is valid or not.
So the app has to test user input according to test assertions, say,
and a test suite might test the app according to the same or other
test assertions perhaps and the test suites can be tested using test
assertions too. The documents can be tested and its tests tested too.
Lots of tests and test assertions and layers and so on. If the standard
is well written it will foster all of this to ensure best implementation
and end user experience. A tall order though. This means test assertions
of all kinds and aimed at all sorts of targets will be good as a basis
for quality assurance of the standard spec but could lead to information
overload and some shaky assumptions too among the standard architects
perhaps. So moderation has to be exercised, one never knows all the uses
in advance I guess.

Regards

-- 
Stephen Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice



Quoting Dave Pawson <dave.pawson@gmail.com>:

> On 16/08/07, Serm Kulvatunyou <serm@nist.gov> wrote:
>> ----------------
>> Back to the SBS example though, I'm a little apprehensive about the way a
>> necessarily vague conformance clause might impact on TAs. I guess there are
>> all sorts of way of implementing a markup standard for instance (look at all
>> the types of products which variously implement HTML, XHTML or docbook for
>> example
>
> Please don't get into validating applications when what you want to validate
> is the markup generated by the application.
> The standard is the XML instance, not the way it has been produced.
>
>
>  - browsers, XSLT apps, mappers, editors, pdf generators, office
>> apps, etc) and each would perhaps need a different conformance clause and
>> consequently different TA lists to test such conformance.
>
> A mixed bag of applications with no common specification?
> What has this to do with TAG?
>
> regards
> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
>