RE: [tag-discuss] the role of an XML mark-up for TAs

["Durand, Jacques R." <JDurand@us.fujitsu.com>]

<JD2>

-----Original Message-----
From: david_marston@us.ibm.com [mailto:david_marston@us.ibm.com] 
Sent: Tuesday, November 21, 2006 3:03 PM
To: tag-discuss@lists.oasis-open.org
Subject: Re: [tag-discuss] the role of an XML mark-up for TAs

JD>...So the design of a TA mark-up may vary depending on these [goals].
>Also, the grouping of TAs in larger entities (e.g. conformance
>profiles or levels) is another aspect of the mark-up, that is more or
>less important to address depending on above options.

I think that test case metadata cites the TAs. Test suite execution is
driven by test case metadata, so with TAs in there, you can make the
test harness exercise particular TAs. The test case metadata also has
profiles, levels, or other Dimensions of Variability (DoV) noted, but
TAs probably need them also. 

<JD2> Putting on my hat of
the-guy-who-tries-to-figure-out-the-scope-of-work-to-do (translate:
trying to get to a consensual TC charter ;-) let me try to outline a
possible scope, and see how folks react:
- TA metadata is in, yet separate from more detailed test case metadata,
which we know will refer to TAs, but which we leave out for now (I
suspect there is more out there on test case metadata than on TAs, e.g.
ATML / TPML, etc). 
- description of test set-up assumed by these TAs, is in (not sure how
much beyond a description narrative can be done here - metadata?)
- TAs are likely to need "containers" such as conformance profiles.
Metadata for these is in. Here, trouble may start with the semantics of
these groupings (see distinction between modules and profiles in "W3C QA
Framework"): spec writers may want to organize them by functional
modules following the spec flow, which is not the same as conformance
profiles or usage profiles.
</JD2>



I am suggesting that one test case cites
more than one TA regarding setup of the premise, and possibly more than
one TA as a test target, so a test case arises from the interaction of
several related TAs.

<JD2> fair to support this - but noting that from a metadata
perspective, does not have to affect TA meatadata? </JD2>


I want to submit a Goal 4, higher up than the 1-3 from Jacques.
4. Test case creation driver - TAs can be loaded en masse into a tool
that will produce design elements of each test case needed, which will
be the test case metadata and some specifics of the content of the
test case. I picture this as being analogous to a tool that takes a
grammar expressed in BNF and produces valid instances of the grammar.
This is where the interaction of TAs is really put to good use.

<JD2> seems ambitious... I wouldn't rule that out but hard to figure for
how many target specifications (and types of TAs) this goal would be
reachable. Would need some amount of contextual input e.g. a processable
representation of the test set-up. More work...</JD2>

Between (3) and (4), we should also be able to address the coverage
afforded by a test suite. It might not be airtight, but the TAs and
test case metadata should be able to be combined into a picture of
how well the test suite covers either (1) any single TA or (2) the
set of all testable situations that can be derived as interactions
among the TAs. Again, I think that coverage is not a direct goal, but
rather something that is enabled by aspiring to the other goals.

<JD2> can you better explain your (2) above? another aspect, is coverage
of the target specification by TAs. Certainly coverage of TA itself by
test case is important too - but I hope we don't have to worry about it
within the scope of designing TA metadata. </JD2>

.................David Marston

---------------------------------------------------------------------
To unsubscribe, e-mail: tag-discuss-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: tag-discuss-help@lists.oasis-open.org