Name and =
abbreviation
Test Assertions Guideline =
TC, or TAG=20
TC (?)
Purpose
The design of Test =
Assertions (TAs)=20
associated with a specification or standard has the following recognized =
benefits: (i) it improves the quality of this specification during its =
design,=20
and (ii) it reduces the lead time necessary to create a test suite for =
this=20
specification. =20
A test assertion (TA), =
also=20
sometimes defined as test specification, is understood in this charter =
with the=20
following general meaning: it describes the expected output or behavior =
for an=20
implementation under some specific operation conditions, in a way that =
can be=20
measured or tested. Each test assertion is (as far as possible) an =
independent,=20
complete, statement of how to verify that an implementation satisfies a=20
requirement in the target specification. Test assertions are generally =
different=20
from test cases, which are more detailed and contingent to a concrete =
test=20
framework: TAs are the basis to write test cases, and relate the latter =
to the=20
narrative of the target specification. NOTE: this candidate definition in a =
charter is=20
only intended to reflect a general level of agreement about what is it =
we work=20
on. It may be refined and improved in a final=20
speciification.
The general objective =
served by this=20
TC is to facilitate the creation and usage of test assertions by any =
group=20
involved in designing a specification or standard of which software=20
implementations are expected to be developed, with a focus on OASIS =
technical=20
committees.
The first step in =
achieving this is=20
to establish a common and reusable model, metadata, methodology and=20
representation for TAs.
This is aligned with the =
intent of=20
the former OASIS Conformance TC, although the focus in the current =
initiative is=20
not on the various aspects of quality and conformance, but on a specific =
one,=20
namely test assertions.
This TC will submit its =
deliverables=20
to the OASIS Technical Advisory Board (TAB) for recommendation to the =
OASIS=20
Board. The TC would accept feedback and recommendations from the TAB and =
OASIS=20
Board, so that its output can be considered by the OASIS Board for =
inclusion in=20
a future revision of the OASIS general TC process, aiming at improving =
the=20
quality and adoption of OASIS output.
Scope
The scope of activity for =
this TC=20
must be within the following topics:
- =20
A=20
model for designing Test Assertions (TA model), that is independent from =
any=20
particular target specification or standard, but that may recognize =
different=20
types of test assertions, and may accommodate these in a specific way.=20
- =20
Test assertions may be for =
verifying=20
conformance of an implementation to a specification, or interoperability =
between=20
implementations of the same specification.
- =20
The TA model may address =
any useful=20
relationship identified between TAs, such as pre-requisites or =
pre-conditions.=20
It may include support for grouping several TAs - or grouping entities =
-, but=20
will not pretend to fully model such entities as conformance profiles,=20
specification modules or implementation roles.
- =20
A=20
methodology to make use of this model. This may include examples derived =
from=20
applying the above TA model to particular specifications or=20
standards.
- =20
An XML=20
representation for TAs and – if appropriate – for their =
grouping entities.=20
Additional notations supporting the modeling of TAs (e.g. UML) are also =
within=20
scope. The intent of the XML representation is left at the discretion of =
the TC.=20
It could be intended as an exchange format for editing tools, or as a =
source for=20
rendering/publishing, or as an input for a test tool, or a combination =
of=20
these.
- =20
Within scope is a =
characterization=20
of the test environment or test harness assumed by the test assertions, =
stating=20
the expected properties and mode of operation required by the =
verification of=20
the TAs. Such characterization may be seen as some of the requirements =
for a=20
real test harness intended to process test cases based on these TAs=20
.
- =20
Within scope is the =
selection and/or=20
refinement of definitions of concepts expected to be related to TAs, =
even if not=20
directly targeted by the modeling and methodology work of this TC. Such =
concepts=20
may include: test case, conformance profile or level, test environment / =
harness, test execution.
- =20
It=20
is in the scope of this TC work to examine current practices in other =
OASIS TCs=20
or other organizations which have already written test assertions for =
their=20
specifications.
- =20
Within scope are efforts =
to liaise=20
with other organizations than OASIS, and cooperation with external =
contributors=20
and groups will be considered, as well as joint=20
submissions.
Documents that will be =
considered as=20
contributing material to this TC, i.e. that will deserve prime attention =
from=20
the TC assuming their IP status is compatible with the IPR mode of the=20
TC:
- =20
the Test=20
Assertion Guideline draft, (originally initiated within OASIS TAB,=20
2004-2005)
- =20
the=20
Conformance guideline (OASIS)
- =20
QA=20
Framework: Specification Guidelines (W3C, November =
2004)
http://www.w3.org/TR/qaframe-spec/
- =20
Variability in =
Specifications, WG=20
note (W3C, 2005)
- =20
Test=20
Assertion Documents for WS-I profiles (2003-2004).
- =20
Others?
Explicitly=20
Out-of-scope:
- Defining test case =
metadata or=20
representation. (Several developments are occurring in this=20
area.)
- Modeling or methodology =
of=20
dimensions of variability such as conformance profiles / levels / =
modules /=20
discretionary items. Substantial prior work exists in these areas. =
However the=20
TC may use this prior work as a foundation and may also accommodate =
these=20
constructs when designing an XML mark-up for TAs.
Deliverables
The TC will produce the =
following=20
deliverables, in this order: (tentative =
date?)
(a) Test =
Assertions=20
Guideline document, including definitions, abstract model / structure =
not=20
excluding the use of a modeling notation (e.g. UML), methodology and =
examples=20
showing how to extract TAs from a target =
specification.
(b) A representation =
of TAs in=20
XML, that refers to and supports the TA model of (a). =
IPR =
Mode
- =20
Royalty-Free on limited =
terms=20
Audience
- =20
members=20
of any OASIS tech committees
- =20
members=20
already involved in defining specifications/standards/tests in other=20
organizations.
Language
- =
English
References