Re: Reasons to wait on convening a TC

[david_marston@us.ibm.com]

Responding mainly to Dave Pawson:
DM>> The Spec Guidelines tell editors to "write testable sentences" and
>>to a great extent, spec editors already do that. If the TA TC is
>>formed and all they do is tell spec editors how to write more-refined
>>forms of testable sentences, it wasn't worth it.

DP>Doesn't that pre-judge the output of an as yet unformed TC?

It may, but not unduly so. I was responding to Jacques' suggested
possible deliverables of the unformed TC. On 11/27, he wrote:
>1. A [simple] guide for writing test assertions, the goal of which is
>to provide some basic guidance to most groups and TCs working on a
>specification....Some on this forum have stated that this guide
>alone has value.
I contend that it wouldn't have enough value to justify forming a TC.
The QA Interest Group at the W3C has already discussed TAs, and they
could set up a Wiki page to facilitate the production of the guide
contemplated above.

Repeating myself: a lot of the people involved in W3C QAIG and this
list are the same people who have been working on a family of
guidelines and tools for several years. The choice of consortium in
which to do further work on TAs is secondary. BTW, who will be at the
XML 2006 conference next week? Maybe we could have a meal together?

DM>I think the conformance testing community is aware that some aspects
>of their individual solutions are hard to quantify in planning and
>might be due for enhanced technology....
>It is appropriate to convene a TC to fill a hole identified by many,
>without presuming that the hole exists in just one end-to-end process.

Let me give that thought some new words. Dave Pawson had asked if I
was picturing a spec for machine-processable TAs as part of a big
end-to-end solution for automated testing. (Whether this was just
about conformance testing of specs was not stated, but read on.) I
think there are many pictures of end-to-end solutions in which the
current technology leaves a black hole in the vicinity of written
specs getting converted into test cases. If you are estimating the
workload for testing a given spec, you have to estimate how long it
will take a team of humans to understand the spec deeply enough to
generate a test plan populated with cases. Machine-processable TAs
offer new hope for simpler quantification of that phase of the
proverbial end-to-end process.

I should also add that I, and probably several others on this list,
have some picture of an end-to-end solution by which consortia like
OASIS write their specs and get the specs accepted by the Web or
e-commerce communities at large. Better TAs are a part of *that*
end-to-end picture!

DM>>Not mentioned so far, but also worth considering:
>>machine-processable TAs might also help developers write correct code.
DP>Do you mean code, or XML?
I mean code, like Java, C, etc. I think the discussion of TAs arose in
a community that is more focused on deriving test cases from specs, but
there are plenty of people interested in deriving code from specs.

There is also a question of whether TAs will be harder to write than
prose that purports to convey the same meaning. I think that's likely
to be the case. It might also be harder to decide how many TAs need
to be written on a given topic, compared to the number of "testable
sentences" in current practice. I think these difficulties arise just
as a consequence of greater formality, whether or not the form of
expression is machine-processable.
.................David Marston