Re: [tag-discuss] Re: Reasons to wait on convening a TC

["Dave Pawson" <dave.pawson@gmail.com>]

Request please David.
Would you use plain text.
I can barely read the HTML.

On 28/11/06, david_marston@us.ibm.com <david_marston@us.ibm.com> wrote:

> Back on the main line of skepticism about TAs in XML, I think a blunt
> statement might be in order:
> The Spec Guidelines tell editors to "write testable sentences" and to a
> great extent, spec editors already do that. If the TA TC is formed and all
> they do is tell spec editors how to write more-refined forms of testable
> sentences, it wasn't worth it.

Doesn't that pre-judge the output of an as yet unformed TC?



>
> Right now, test planners and writers of test cases can do a reasonably good
> job of using prose in the specs to design a set of test cases by the
> "manual" methods of understanding the sentences and joining multiple
> constraints in their minds. If the TA TC is formed and they issue a standard
> for machine-processable TAs, then we have a significant advance in
> conformance testing, and the work of the TC was worth it.

My own personal scepticism says that's a good vision. The machine processable
tests I mean.


>
> DP>It would appear you have an end to end solution mapped out.
>
> Well, yes and no. I think the conformance testing community is aware that
> some aspects of their individual solutions are hard to quantify in planning
> and might be due for enhanced technology.
I don't understand that sentance.


Test case metadata is on the list
> of technology enhancements, above TAs in priority,
Nowhere have I seen any rationale for test metadata, nor suggestions as to
what it migh contain.




and we know that work is
> underway to put test case metadata into XML.
Which, prior to the formation of the TC, seems (from an Oasis viewpoint) surely
wasted effort? Do you mean for other purposes or organisations?



Because it's XML, it can be
> reformatted to different XML if future developments change the details.

Two concerns. Unaddressed as yet.
What transforms, for what purpose within the scope of this proposed TC.
What if the tested spec working group are not XML geeks. Who writes
the tests and this metadata, in XML for them?



In my involvement with XSLT testing
> (since early 1998), planning always has to assume that a hard-to-quantify
> but large amount of time is spent in just reading the specs very carefully,
> building up implications from testable sentences that are far apart in the
> documents, and figuring out the correct behavior and how to test it.
My two Euros says that won't change. Understanding complex specs
is plain hard work. Translating that into clear XML, worse into mc executable
tests won't be any easier.



Machine
> processable TAs would act as a catalyst for tooling to help in this process.
> It is appropriate to convene a TC to fill a hole identified by many, without
> presuming that the hole exists in just one end-to-end process.

If they are a reality, I'd agree.


>
> Not mentioned so far, but also worth considering: machine-processable TAs
> might also help developers write correct code.
Do you mean code, or XML?



>
> DM>>If we design TAs to be amenable to machine processing (e.g., by
>  >>expressing them in XML), then there is hope that early efforts at
>  >>TAs will become more valuable in the future as new tools emerge.
>
>  DP>A test matrix will do that.
>
> Intriguing thought! Can you elaborate? To me, the term "test matrix"
> suggests a display of selected test case metadata.


Simple matrix of test spec para vs test addressing that para.
A simple means of checking test completeness.





>
> DP>What of a WG that doesn't have any XML expertise? Wouldn't
> >[defining the TA format as an XML vocabulary]
> >be making it unecessarily hard for them?
>
> It's possible that TAs will only apply to a subset of the type of specs
> issued by OASIS, W3C, OMG, and the like. (BTW, XML knowledge is effectively
> mandatory for spec editors at the W3C.)
But only effectively. It is widely loathed in other quarters :-)

Is this a scoping issue for the TC? Testing may not be wholly applicable?
Isn't this TC supposed to be more focussed on Oasis specs?
Can anyone clarify please?



This gets back to the "testable
> sentences" remark I made earlier. I'm pretty sure that the hard part about
> composing well-formed TAs is formulating the contingencies and conclusion
> (or whatever the various parts will be named) to be suitably rigorous.

Wholly agree. Which is why I'm concerned that an XML vocabulary
won't be flexible enough.

> On
> the other side, there may already be vocabularies for "rules" that have
> already solved all aspects of the TA problem other than TA-specific markup.

ATLAS, in 1973 :-)

Unsure on how this last para should be interpreted David. Sorry.

regards
-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk