OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

tag-discuss message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: surviving the winter break

charset="us-ascii" Content-Transfer-Encoding: quoted-printable
I hope = all of you=20 are still on board, after the winter break... which leads me to wish you = a happy=20 New Year  (even if this is still premature to some of=20 us).
The = discussion list=20 has been up for almost 2 months, and it has clearly revealed interest in = the=20 topic. As you know, this list is primarily intended for testing the = waters for=20 TC creation - and is not a forum for doing the real tech work: more time = for=20 this in a future TC assuming we could agree on a TC platform. This = becomes now=20 the main thing to focus on - and hopefully we should all agree this = month on a=20 charter draft.
Here = is a new=20 iteration on a charter draft - please review and comment. For = the sake=20 of finding consensus, in case you disagree with a statement in the = draft, please=20 indicate:
- if = it is a=20 show-stopper for your participation (can't tolerate = it)
- if = you can live=20 with it though not happily ever after.
Of = course, feel free=20 to comment on this even if you/your company is not sure yet of its = committment=20 to a TC.

Name and = abbreviation


Test Assertions Guideline = TC, or TAG=20 TC (?)




The design of Test = Assertions (TAs)=20 associated with a specification or standard has the following recognized = benefits: (i) it improves the quality of this specification during its = design,=20 and (ii) it reduces the lead time necessary to create a test suite for = this=20 specification. =20


A test assertion (TA), = also=20 sometimes defined as test specification, is understood in this charter = with the=20 following general meaning: it describes the expected output or behavior = for an=20 implementation under some specific operation conditions, in a way that = can be=20 measured or tested. Each test assertion is (as far as possible) an = independent,=20 complete, statement of how to verify that an implementation satisfies a=20 requirement in the target specification. Test assertions are generally = different=20 from test cases, which are more detailed and contingent to a concrete = test=20 framework: TAs are the basis to write test cases, and relate the latter = to the=20 narrative of the target specification. NOTE: this candidate definition in  a = charter is=20 only intended to reflect a general level of agreement about what is it = we work=20 on. It may be refined and improved in a final=20 speciification.


The general objective = served by this=20 TC is to facilitate the creation and usage of test assertions by any = group=20 involved in designing a specification or standard of which software=20 implementations are expected to be developed, with a focus on OASIS = technical=20 committees.

The first step in = achieving this is=20 to establish a common and reusable model, metadata, methodology and=20 representation for TAs.


This is aligned with the = intent of=20 the former OASIS Conformance TC, although the focus in the current = initiative is=20 not on the various aspects of quality and conformance, but on a specific = one,=20 namely test assertions.


This TC will submit its = deliverables=20 to the OASIS Technical Advisory Board (TAB) for recommendation to the = OASIS=20 Board. The TC would accept feedback and recommendations from the TAB and = OASIS=20 Board, so that its output can be considered by the OASIS Board for = inclusion in=20 a future revision of the OASIS general TC process, aiming at improving = the=20 quality and adoption of OASIS output.






The scope of activity for = this TC=20 must be within the following topics:


-         =20 A=20 model for designing Test Assertions (TA model), that is independent from = any=20 particular target specification or standard, but that may recognize = different=20 types of test assertions, and may accommodate these in a specific way.=20


-         =20 Test assertions may be for = verifying=20 conformance of an implementation to a specification, or interoperability = between=20 implementations of the same specification.


-         =20 The TA model may address = any useful=20 relationship identified between TAs, such as pre-requisites or = pre-conditions.=20 It may include support for grouping several TAs - or grouping entities = -, but=20 will not pretend to fully model such entities as conformance profiles,=20 specification modules or implementation roles.


-       =20 A=20 methodology to make use of this model. This may include examples derived = from=20 applying the above TA model to particular specifications or=20 standards.


-       =20 An XML=20 representation for TAs and – if appropriate – for their = grouping entities.=20 Additional notations supporting the modeling of TAs (e.g. UML) are also = within=20 scope. The intent of the XML representation is left at the discretion of = the TC.=20 It could be intended as an exchange format for editing tools, or as a = source for=20 rendering/publishing, or as an input for a test tool, or a combination = of=20 these.


-         =20 Within scope is a = characterization=20 of the test environment or test harness assumed by the test assertions, = stating=20 the expected properties and mode of operation required by the = verification of=20 the TAs. Such characterization may be seen as some of the requirements = for a=20 real test harness intended to process test cases based on these TAs=20 .


-         =20 Within scope is the = selection and/or=20 refinement of definitions of concepts expected to be related to TAs, = even if not=20 directly targeted by the modeling and methodology work of this TC. Such = concepts=20 may include: test case, conformance profile or level, test environment / = harness, test execution.


-         =20 It=20 is in the scope of this TC work to examine current practices in other = OASIS TCs=20 or other organizations which have already written test assertions for = their=20 specifications.


-         =20 Within scope are efforts = to liaise=20 with other organizations than OASIS, and cooperation with external = contributors=20 and groups will be considered, as well as joint=20 submissions.



Documents that will be = considered as=20 contributing material to this TC, i.e. that will deserve prime attention = from=20 the TC assuming their IP status is compatible with the IPR mode of the=20 TC:


-       =20 the Test=20 Assertion Guideline draft, (originally initiated within OASIS TAB,=20 2004-2005)

-       =20 the=20 Conformance guideline (OASIS)

-       =20 QA=20 Framework: Specification Guidelines (W3C, November = 2004)


-       =20 Variability in = Specifications, WG=20 note (W3C, 2005)

-       =20 Test=20 Assertion Documents for WS-I profiles (2003-2004).

-       =20 Others?




Explicitly=20 Out-of-scope:


- Defining test case = metadata or=20 representation. (Several developments are occurring in this=20 area.)


- Modeling or methodology = of=20 dimensions of variability such as conformance profiles / levels / = modules /=20 discretionary items. Substantial prior work exists in these areas. = However the=20 TC may use this prior work as a foundation and may also accommodate = these=20 constructs when designing an XML mark-up for TAs.




The TC will produce the = following=20 deliverables, in this order: (tentative = date?)


(a)  Test = Assertions=20 Guideline document, including definitions, abstract model / structure = not=20 excluding the use of a modeling notation (e.g. UML), methodology and = examples=20 showing how to extract TAs from a target = specification.


(b) A representation = of TAs in=20 XML, that refers to and supports the TA model of (a). =



IPR = Mode


-       =20 Royalty-Free on limited = terms=20




-       =20 members=20 of any OASIS tech committees

-       =20 members=20 already involved in defining specifications/standards/tests in other=20 organizations.




- = English



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]