Re: [tag] Groups - TA Anatomy V0.4 (AnatomyTA-v04.doc) uploaded

["Dave Pawson" <dave.pawson@gmail.com>]

Apologies. Finger trouble. continued below

On 20/09/2007, Dave Pawson <dave.pawson@gmail.com> wrote:
> On 20 Sep 2007 00:24:55 -0000, jdurand@us.fujitsu.com
> <jdurand@us.fujitsu.com> wrote:
> > material for "Anatomy of a TA"  section, updated after F2F and mail list
> > comments.
>
> Further comments:
>
> Characterize the item under test (or IUT) -
>
> 'characterize' needs definition.  None of the examples are explicit.
> Suggest. Define the item under test sufficiently to enable its
> selection, e.g. Part title, Part number, build specification, version
> number.
>
> performed on the item under test or on the test environment,
>
> Clarification on 'test environment'. Used without definition.
>
> Although two tests are actually involved, a single TA is appropriate,
> as these tests are about the handling of a same feature (the radix) by
> the same function.
>
> Unclear, confusing, suspect English. The word radix appears out of context.
>
> (e.g. it should not tell where to find the argument values to be used
> – this is the role of a test case).
>
> Where is the differentiation between test assertion and test case?
> Isn't the latter an
> instantiation of the former? Clarify please.
>
> Conversely, in many cases, it is acceptable to be less explicit than above.
>
> This is supposed to be a defining document. This only confuses.
>
> A test assertion ignores the keywords MUST, SHOULD, MAY because its
> focus is on the feature to be verified on the item under test.
>
> I quite disagree with this statement. IMHO a test assertion MUST contain hard
statements, hence MUST contain MUST statements to avoid lack of clarity in
interpretation. It isn't a case of ignoring them, more a case of
responding to them.
In that response the person designing the tests must be given clear
actions, which
will include the word MUST, e.g. must measure between 10 and 12.2 volts.

The pre-condition (optional). This condition acts as a qualifier for
the item under test (IUT): it defines which instances of the IUT are
qualified for this test, or it defines in which state the test
environment must be for an IUT to undergo this test.

Suggest separation be given to the UUT and the test environment. UUT
types/variants
are not part of the environment, they form a clear definition of the
item being tested.
A pre-condition may be a physical set of requirements. E.g. clean room
conditions,
air temp this, humidity that. The current definition is too constrained.

If the pre-condition is false, the TA does not apply. Its outcome is:
"not applicable".
Confusing. A pre-condition is a pre-requisite for the test to be
carried out. Otherwise it is not a pre-condition? If the conditions
are not in place
the consequence is that the test cannot be carried out.

The term not applicable is used. Perhaps 'the test is not executed' is
clearer. A test
is always applicable within a given test suite.

The test effect. This is an observable behavior (action, event) or
quality (expresses as a predicate) of the IUT – or the combination
{IUT, test environment},

Two comments. Not always a predicate? Qualitative assessment could be
measured on
a linear scale. Agreed pass./fail will be determined by the point on
that scale, but the test
effect (test outcome is clearer) is that an operator assessed X and
marked it as 5/10 or
similar. The test outcome is a pass due to the requirement being to be
marked as 4/10
or higher. The measurement requires differentiation from the test outcome, which
is binary. Predicate is too posh for a simple yes/no. The rest of that
para seems
superfluous. Indeterminate simply implies that the test was badly
designed. Positive
and negative affects? Plain English please. Pass fail is clearer and
preferable.

This condition characterizes the final state of the test environment
and/or IUT, after the test is complete. If for any reason, the
post-condition is "false",

Again mixing the UUT and the test environment. This all forms part of
the post condition.
It cannot be false. It is fact, no more. No less. That para is very
confusing. Post conditions
are not tested, the UUT is. It may be that a test screws up the test
environment, e.g. blows
the test lab up. the UUT may fail the test, but the post conditions
are as they are,
and the next failure is an anability to establish the pre-conditions
of the next test.
If the test environment needs testing, it isn't the test environment,
it's a different UUT.


must use the approved format for product identifiers.
Unclear example? approved format is undefined hence untestable.

Test Assertion: TA-101b
Specification requirement: < req 101>
Item under test: A business endpoint.

'A business endpoint'  Too vague for an example? Seems to be an XML instance.
The test trigger isn't a trigger, it reads as a sequence of test executions.


regards









> --
> Dave Pawson
> XSLT XSL-FO FAQ.
> http://www.dpawson.co.uk
>


-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk