AI: Grouping

["Stephen Green" <stephen.green@documentengineeringservices.com>]

Here is some candidate text for the Grouping sections (see also in
wiki). It doesn't include yet the AI of providing text for a new
glossary item 'Test Assertion Document' but I've added a subsection by
that title to the Lists section.

<start/>

3.5 Test Assertion Grouping

When writing test assertions, as the specification is being analysed,
it is usual to group certain test assertions together, either as
having a special status, such as all accredited test assertions for a
given specification, or as sharing a particular characteristic, such
as a common category of test assertion target.

A special kind of grouping is the container of all test assertions
which belong to a particular specification or profile. Here the
container may be the specification document itself if it includes
within it the test assertions to be associated with it.

Ways to group test assertions assertions include two of special note:
- explicit listing of test assertions by their identifiers (section
3.5.1) and a more implicit grouping by a common but not unique
property such as the test tags names or tag values assigned to the
test assertions (section 3.5.2).


3.5.1 Lists (Dimensions of Variability)

To explicitly identify a group of test assertions they can be listed
explictly by the unique test assertion identifiers. This makes it
clear once and for all which assertions belong to that particular
group and which do not. In addition to such a list, the logical reason
which determines whether a test assertion is a member or not of that
list will need to be stated, at least to help with understanding and
maintenance of the list.

For example:

Test Assertion List Id: A001
List Description: all assertions describing 'Size' requirements
List members: TA001, TA002, ....., TA008

Note that although, in this example, we have avoided enumerating each
and every test assertion identifier by using an ellipsis ('...'), such
methods introduce a possible weakness. It might be overlooked during
maintenance of the test assertions or the test assertion list if a
later test assertion is given an ID of TA002a and therefore is
implicitly rather than explictly made a member of this particular
list. This may be a mistake since the new test assertion TA002a might
not relate to the list description of 'Size' requirements. A list is
completely explicit about every test assertion member when every
member test assertion is listed explicitly by its test assertion
identifier.

For example:

Test Assertion List Id: A001
List Description: all assertions describing 'Size' requirements
List members: TA001, TA002, TA003, TA004a TA004b, TA005, TA006, TA007, TA008

Such a list may be regarded as 'fixed', 'frozen' or 'closed'. A test
assertion added later with identifier TA005a, if it is to be included
in this list, would require a change to the list (with possible
version or change control implications) or the creation of a new list
(with a new list identifier, if a list identifier is included with the
list).


Test Assertion Document

A list of test assertions related to either conformance or
interoperability testing will need special care with respect to
version control and change management and therefore it will need to be
clear what criteria are being used to determine which test assertions
are members of the list and which are not. The special case of a
container for all test assertions related to a given specification or
profile, say, is a special example of a most explicit list, although
here the method used to define such a list may involve the use of
inclusion of the test assertion itself (rather than just its
identifier) within a special document or package. One way to create
such a list is to include all such related test assertions within a
document, which we might call a 'Test Assertion Document'. Other
synonymous terms might be 'Test Assertion List'. 'Specification
Analysis' or 'Test Assertion Set'. Note that the container of this
complete set of test assertions might instead be the document of the
specification or conformance profile itself, when test assertions are
included, say, within the text of the actual specification or profile.


3.5.2 Tags (Test Assertion Metadata)

Another way to define a group of test assertions is to use a
non-unique property of such assertions rather than just using their
unique identifiers in a list or containing the test assertions
themselves in a document. To this end, test assertions may be assigned
metadata in the form of non-unique 'tags' or 'labels'.

For example, test assertion 'widget-TA100-7' might be tagged as
'Size-Property-Descriptive':

TA id: widget-TA100-7
Target: widget
Normative Source: specification requirement 104
Predicate: [the widget] is from 5 to 15 centimeters long in its longer
dimension.
Prescription Level: medium-size:mandatory
Tag: Size-Property-Descriptive

Then it might be included in a list of test assertions related to
'Medium Size' requirements, along with other assertions, say, tagged
'Size-Related' but 'not' say with test assertions
'Small-Size-Related'.

Test Assertion List Id: A002
List Description: all assertions involved in describing 'Medium Size
Widget' requirements
List members: All test assertions with Tag 'Size-Property-Descriptive'
AND Tag 'Size-Related' AND NOT Tag 'Small-Size-Related'

This we have called a 'List' but it is in fact defined rather more
implicitly than if every member were listed by its identifier, as
described in the previous section (section 3.5.1). In fact a more
explicit and well-defined list might combine both tags and identifiers
to group the assertions:

For example:

Test Assertion List Id: A002
List Description: all assertions involved in describing 'Medium Size
Widget' requirements
List Description: All test assertions with Tag
'Size-Property-Descriptive' AND Tag 'Size-Related' AND NOT Tag
'Small-Size-Related'
List members: TA001, TA002, TA003, TA004a TA004b, TA005, TA006, TA007, TA008

So a tag is a further, optional test assertion element useful in
grouping test assertions. It may sometimes be useful to create tags as
name-value pairs. For example, tagging a test assertion:

Tag: WidgetSize=Medium

This would allow all test assertions related to requirements for
medium sized widgets to be grouped to facilitate, say, testing of just
the medium sized widgets or a conformance profile relating just to
medium-sized widgets.

Several such filters can be applied to the same set of assertions and
any given assertion can appear in more than one grouping.

Special consideration when using tags for grouping is to be given to
the stages in the workflow of test assertion authoring and maintenance
and subsequent use at which changes might be made to tags and their
values. Specially there may be the addition of new tags, perhaps by
adding metadata which is separate to the documented test assertion. If
metadata for test assertions is defined and maintained separately from
the test assertions it may be subject to an entirely different set of
version and change control rules and methodologies. In this case, a
distinction might need to be made between lists based tags which were
part of the original test assertion and those whose list membership
might be different to that which was known or expected at the tiome
the list was defined.

For example, consider a list defined using tags but without explictly
listing test assertion identifiers:

Test Assertion List Id: A002
List Description: all assertions involved in describing 'Medium Size
Widget' requirements
List Description: All test assertions with Tag
'Size-Property-Descriptive' AND Tag 'Size-Related' AND NOT Tag
'Small-Size-Related'

If TA004a is originally tagged 'Size-Related' but the workflow allows
it to be subsequently tagged 'Small-Size-Related', then there will
need to be rules defined which determine whether the test assertion is
still a member of List 'A002'.



<end/>


-- 
Stephen D. Green

Partner
SystML, http://www.systml.co.uk
Tel: +44 (0) 117 9541606
Associate Director
Document Engineering Services
http://www.documentengineeringservices.com

http://www.biblegateway.com/passage/?search=matthew+22:37 .. and voice