OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

tag-discuss message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: RE: [tag-discuss] Re: Test case metadata...


I am responding to those parts of Serm Kulvatunyou's message (of 12/2)
that I can address without a detailed reading of the ebXML documents.

SK>There are a lot of terminology mismatches around here.

Agreed.

SK>Having read some software testing literatures, the terminology in
>the IIC specification seems more common.

Please provide some citations. I think this is one of the reasons I
wanted to suggest a workshop, to try harder to find out what has been
done elsewhere.

SK>...Note that in IIC, a 'test case' contains everything from
>relationship to the TR, machine interpretable test execution steps,
>inputs, and verification conditions/expected result, and more.

The only terminology issue here is that I and my metadata colleagues
treat the "test case" as an abstraction and the "test case metadata"
as a real entity (an XML file) that has the data. From the above, it
seems that we agree about the set of data that describes a test case.

SK>...some elements in the [W3C Note on] test case metadata are
>defined to be machine processable.... On this basis, perhaps it is why
>some of the list participants are thinking of some automations based
>on the TA, if it includes such elements.

This gives me a chance to clarify something about automation in
testing. There are two thoughts commingled above.
1. Automation to run a set of test cases against a test subject has
been and will continue to be facilitated by having test case metadata
that is platform-independent and describes the phases (setup, test
execution, comparison of results, cleanup) in platform-independent
terms. TAs are beneficial for labeling and coverage analysis, but are
not essential.
2. Automation could be used (and maybe is already) to create parts
of a test suite. TAs are much more central to this kind of work, and
being machine processable is where true advances in the field could
arise.

Perhaps I make the notion of machine processable TAs sound too
ambitious. If we assume an XML notation for a moment, I see the
difference as something like this.
XML, but not machine processable in any meaningful sense:
<assertion>A sentence in English prose.</assertion>
XML that supports to machine processing:
<assertion>
  <contingency>fixed wording for a contingency</contingency>
  <contingency>fixed wording for a contingency</contingency>
  <contingency>fixed wording for a contingency</contingency>...
  <stimulus>fixed wording describing how triggered</stimulus>
  <result>fixed wording describing the result</result>
</assertion>
To a great extent, the "fixed wording" parts use wording specific to
the domain of the spec, and may use additional XML sub-elements or
attributes to isolate certain details. A set of TA guidelines would
try to describe the desirable qualities of the fixed wording. One
such desirable quality is that each sub-element could be processed by
automation to select and configure building blocks of testing. A
separate ambition is to have the contingencies of more complex test
assertions build on the results of simpler ones, which would allow
TAs to be compounded, in which case the test cases they inspire
could also be compounded.

SK>...the TA or TR spec should include only descriptive info, i.e.,
>it should not be much different from those already included in the
>IIC spec for TR.

Serm, you seem to think that a TC should be formed, despite the IIC
spec and what you've read in the literature. Can you provide more
justification?

I think that if a TC is formed, it must at least be mindful of how
much could be done to machine-process TAs, if the format supports it.
.................David Marston


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]