RE: [tag-discuss] Re: Test case metadata...

["Serm Kulvatunyou" <serm@nist.gov>]

Hi, have been following the discussion for a while.

Yes. There are a lot of terminology mismatches around here. 

A number of us on this list are familiar with the terminologies in OASIS
ebXML IIC Test Framework spec and the OASIS conformance definition.

I saw that the W3C published note has different terminology, particularly of
confusion here is the "test case metadata" (on the W3C note web page, I
think, it is called just "test metadata". I will use the term test case
metadata in this message since it has been used in this discussion group).

Having read some software testing literatures, the terminology in the IIC
specification seems more common.

My understanding so far as having read all the messages in the TAG
discussion is as follows.

- Test Assertion (TA) is in fact Test Requirement (TR) plus Test Profile
(test profile from the functional module perspective, not from usage profile
perspective) in IIC, and maybe a few more additions (e.g., relationships b/w
TRs). Note that TA is an element of the Test Requirement. In the IIC specs,
only descriptive info is contained these entities, which is, perhaps, why a
number of us cannot quite grasp the idea of auto-generate something from
here including test cases or implementation code.
- In the W3C note on test case metadata, there seems to be overlaps into the
Test Case as defined by the IIC. This is because some elements in the test
case metadata are defined to be machine processable (e.g., the Inputs and
Expected Results. On this basis, perhaps it is why some of the list
participants are thinking of some automations based on the TA, if it
includes such elements. Note that in IIC, a test case contains everything
from relationship to the TR, machine interpretable test execution steps,
inputs, and verification conditions/expected result, and more.

My opinion is as follows:
- The big question is how far does a TA or TR spec goes, whether including
just descriptive info or also machine interpretable instructions.
- I think if the (business) objective of the TC is to create something that
a lot of Standard TC can chew on, the TA or TR spec should include only
descriptive info, i.e., it should not be much different from those already
included in the IIC spec for TR. And 'yes' as one of Dave Marson's email
suggested, let the test expert and/or implementer with the help of the
target standard expert translates that into executable TC (as defined by the
IIC). I personally think that such TA or TR spec is useful and of sufficient
value b/c although it is unlikely to provide value in automation in
subsequent steps it is divide the target standard into atomic
features/functions that make test coverage analysis (matrix as termed by
Dave Pawson) more clear. In addition, it would definitely make test case
writing easier as well. 
- I think for standards that are very well defined via some sort of formal
models, autogenerating TAs/TRs as well as TCs maybe at least partially
possible. But I think that should not be the subject of concern for this TC
(it should be an interest of research labs). The produce of this TC would
provide a standard medium for the auto-generation algorithm to publish
result. Ie. This TC focus can be only defining TA/TR data structure such
that it facilitates test coverage analysis, test case authoring, and
understanding of functions within the target standards.

I have witnessed values and how people appreciate the availability of such
descriptive TA/TR.

Having said that I would recommend that we used the terms defined in the IIC
spec for discussion (in which case I think we should have used the term TR
in place of TA), and the length of work for this TC shouldn't be more than 6
months.
 
Thanks for reading this long message...I think I deserve one since I have
read tons of messages before sending one :).


Serm Kulvatunyou
National Institute of Standards and Technology


-----Original Message-----
From: Dave Pawson [mailto:dave.pawson@gmail.com] 
Sent: Friday, December 01, 2006 8:59 AM
To: tag-discuss@lists.oasis-open.org
Subject: Re: [tag-discuss] Re: Test case metadata...

The terminology issues is very rife round here.
Please could the TC take some of the jargon and perhaps develop a glossary.
Simple statement.
My definitions are contra those of David M's and I find it hard
to comprehend his terminology.


regards


-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk

---------------------------------------------------------------------
To unsubscribe, e-mail: tag-discuss-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: tag-discuss-help@lists.oasis-open.org