>
>>
>> [2]- def of Test Case : could include a
mention that "Test Cases
>> usually are grouped in a Test
Suite."
>>
>> --------------------------------
>>
Section 3.1:
>>
>> [3]- Informal names for TA parts:
qualifying the names with "Test Assertion"
>> (e.g. Test Assertion
Target) seems redundant, and inconsistent: we
>> only do that for Test
Assertion Identifier and Test Assertion Target
>> (why?). Suggest to
remove.
>
> OK, but we have this kind of thing in the Guidelines
where it seems
> more appropriate (e.g. we have 'TA Id', not simply 'Id').
I have had a
> few tussles with where 'Test Assertion' is redundant and
wheere it is
> necessary to eliminate ambiguity and vagueness.
<JD> I guess the Guidelines can be a little more
lax on this... and make an exception for "TA Id" where ID is such a generic
term.
>
>
>>
>> [4]- Inside the def of Target:
suggest to add a "note: a test
>> assertion may concern more than one
[part of] implementation. It is
>> recommended however to select only
one of these as the Target."
>
> Not really sure I agree. I think it
may be an oversimplification in
> some ways to only have a single target -
in logical terms.
> I've read opinions that a predicate can have more than
subject or
> target when the predicate describes a relationship between
two
> subjects/targets:
> e.g.
> [A] is less than
[B]
>
> I agree we have in the markup simplified it by rolling all
targets
> into one and for the sake of tools convenience decided that in
some
> implementations just one target can be denoted as 'primary'.
However,
> I could imagine other ways to implement the model which it
seems
> unfair to restrict in this way. I'm thinking some
representations
> could be optimised for scenarios where relationships
between two or
> more targets are the focus, e.g. TAs about tables/fields
in a
> relational database or classes in an API. I'm not even sure we make
a
> normative requirement about this in the markup. Multiple targets
I'm
> OK with but not sure we should insist on one being denoted
'primary'
> (which seems to be arbitrary and related to specific tools
which could
> probably deal with any limitation they have by taking the
first target
> mentioned as 'primary' by default).
>
<JD> I think we misunderstood each
other:
The problem we have to address is: so far we allow only
one Target in our model !
And I totally agree with you that some TAs are of the
kind: [A] is less than [B]
So we need to tell the user this is still OK: they can
decide that "A" is the Target, and "B" becomes an accessory object (e.g.
referred by a Variable).
I now realize this is an important point that we have
not addressed at all - but that can be done easily.
Probably a note in the Guidelines would help too, but I
think the model needs to mention it too, so that people don't think the model is
not adequate.
> Have I misunderstood?
>
>>
>> [5]- Def of
Prerequisite: its las sentence is a repeat (said before).
>
>
Agreed. Good catch
>
>>
>> [6]- Def of
"Tag":
>> - remove the (s)
>> - remove the references to the
reader ("these tags provide you
>> with..." --> "these tags provide
an
>> opportunity...")
>> ("they enable you to group..."
--> "they allow for grouping") Same
>> for the "Variables"
definition --> Variable.
>
> OK
>
>>
>>
--------------------------------
>> Section
3.2:
>>
>> [7]- same as [3], for the names in the
table.
>
> OK
>
>
>>
>> [8]- in the
testAssertion formal def: paragraph before last: it seems
>> to suggest
that several prerequisites are OK ("... together or
>> multiple
prerequisites...
>> A test assertion may have prerequisite(s)". Remove
these allusions.
>> Confusing as we clearly have just one
Prereq.
>
> Well. I kind of agree but with reservations. In the
markup we have a
> policy that we combine prerequisite components into one
expression. In
> modeling terms though I think the distinction with,
say,
>
> TA(A) = 'pass' AND TA(B) = 'pass'
>
> between
calling "TA(A) = 'pass'" one of two prerequisites here and
> calling
"TA(A) = 'pass' AND TA(B) = 'pass'" a single prerequisite is
> so fine a
distinction that I don't think we can make using one or
> other of these a
requirement. Some could say that "TA(A) = 'pass' AND
> TA(B) = 'pass'" is
a combination of two prerequisites and that they
> could indeed be
combined using an expression, as here, or using some
> other construct in
the representation. I just wouldn't see it as the
> place of the model
spec to say whether prerequisites should always be
> seen as one
prerequisite combining the separate parts.
>
> Still, I'll have a
look at the details of what changes would be
> required in the
text.
>
<JD> So that looks like a terminology issue:
"our" Prerequisite element is unique in tthe model. We can then say that it
still can compose several "prerequisites" in the common sense given to this term
(i.e. prerequisite condition). How about replacing:
"A test assertion may also declare a prerequisite
(either as a single entity comprising all prerequisites together or multiple
prerequisites), a prescription level and any number of tags. For this the
testAssertion class has optional associations to classes named
prerequisite, prescription and tag.
The tag
association has optional, multiple occurrence. A test assertion
may have prerequisite(s),
a prescription level and tags,
either implicitly or explicitly."
..."
With:
(note that I replace "declare" with
"define" : when commenting what real test assertions are made of, "define" seems
more appropriate, while the formal testAssertion class "declares" its
internal elements.)
"A test assertion
may also define (a) a prerequisite element, the content of
which is a logical expression that may compose multiple simpler prerequisite
conditions, (b) a prescription level and (c) any number of tags. For this the
testAssertion class has optional associations to classes named
prerequisite, prescription and tag.
The tag
association has optional, multiple occurrence. "
I removed the last sentence which seems
redundant.
</JD>
>>
>> [9]- Formatting: each new construct that is
introduced as : "Formal
>> Definition:"
>> is not highlighted
enough: only the root constructs are, e.g.
>>
normativeSource.
>> I wish its parts would also be highlighted in some
way, e.g.:
>> Formal Definition for refSourceItem:", or with a title,
even if at
>> same level as normativeSource (for
refSourceItem,
>> derivedSourceItem...)
>
>
Agreed.
>
>>
>> [10]- when introducing textSourceItem
and derivedSourceItem: it is
>> said they are "alternatives" to other
options, suggesting exclusiveness.
>> Make it clear they could be "an
alternative - or a complement - to.."
>
>
Agreed.
>
>
>>
>> [11]- derivedSourceItem: why
does it have a complex structure like
>> refSourceItem?
>>
Should be much simpler like textSourceItem.
>
>
> I think we
may need to discuss this a bit. I hadn't envisiaged both a
> refSourceItem
and a derivedSourceItem or both a textSourceItem and a
> derivedSourceItem
bing used for the same requirement unless the
> requirement consisted of
several sources of different kinds. Therefore
> to properly model a single
source item it doesn't seem right to me to
> use two distinct source item
types - not unless they are in fact two
> separate source items. So for
this reason I thought we would need the
> derivedSourceItem to be able to
say what source item(s) it is derived
> from, without having to combine it
with a refSourceItem. It therefore,
> I had thought, would have the
structures needed to make reference(s),
> as well as the structure to add
some derived text.
>>
<JD> I see. I guess either way would work. Maybe
the "interpretation" would do what I had in mind.
>> [12]- definition of "interpretation":
>> We could
add: "It provides additional clues about how the Predicate
>>
(or
>> Prereqisite)
>> relates to the normative
source."
>
> I'm OK with something like
that
>
>>
>> [13]- Target def: "The Target shall be the
subject of the test assertion..."
>> Because of [4] above, say rather:
"The Target shall be the primary
>> subject of the test
assertion.."
>
> Again, as above, I'm not sure about this
one
>
<JD> I think its OK to keep "the subject".
>>
>> [14]- def of Variable:
>>
replace:
>> "...using a notation such as $variable1"
>>
with:
>> "...e.g. using a notation such as $variable1"
>>
Also, add that " a variable may also contain a logical expression,
>>
e.g. a subset of a Prrequisite or Predicate."
>
> Well, strictly
speaking the 'notation' part would be treated as
> normative but not the
example so to move the 'notation' into the
> example does have the affect
of making it vaguely informative.
> Perhaps this is the intention. Perhaps
it should be removed altogether
> if it shouldn't be normative.
<JD> OK to remove. This looks more like a best
practice that the Guidelines could mention.
> Some kind of notation is needed in our markup to use a variable
by
> name within an expression.
> Do we not want to state this as a
requirement?
>
<JD> Mmmh.. not sure we need to even suggest a
notation. Should not be normative by any means, as that really affects the
content (values) of the elements, which depends on a representation of the
model. And even so, our mark-up does not impose this notation...
We could say that the notation used for these variables
in the content of Predicate / Prerequisite, etc. is left to implementations of
this model.
>>
>> [15]- def of Report:
>>
Replace:
>> "the circumstances which shall be true for the report to
be
>> generated. It shall contain an expression which evaluates to true
or
>> false."
>> with:
>> "the circumstances in which
this report item should be generated. It
>> shall contain an expression
which evaluates to true or false, or
>> refer to such an expression
(e.g. as contained in some variable) ."
>
> No longer relevant as we
are removing 'report'
>
>>
>>
--------------------------------
>> Section 3.3
(Semantics)
>>
>> [16]- replace:
>> "As a test
assertion has parts that can be evaluated over a Target
>> instance
(i.e. the prerequisite and the predicate),..."
>> with:
>> "As
a test assertion has parts that can be evaluated over a Target
>>
instance (i.e. the prerequisite, the predicate, and possibly some
>>
variable containing expressions),..."
>
> OK (though I would think
that resolving the variables would be part of
> the interpretation of the
predicate and prerequisite and would happen
> before evaluation of the
TA).
>
>>
>> [17]- use bold font for the semantic
outcomes ("Target not
>> qualified", etc)
>
>
Agreed
>
>>
>>
--------------------------------
>> Section 4:
>>
>>
[18]- as a gneral rule, there should be a short introductory
sentence
>> between the sub-Titles and the formal def of the construct
they
>> announce:
>> E.g. below testAssertionSet and its
formal def, add:
>> "Test Asserions may be grouped using a
testAssertionSet class."
>> - Same for testAssertionRef: move up the
sentence :
>> "A test assertion set may refer to one or more test
assertions by
>> their test assertion identifiers rather than include
the test
>> assertions literally within the set."
>> - Same
for testAssertionResource:
>> "testAssertionResource is used when test
assertions are contained in
>> another external
document."
>
> OK
>
>>
>>
--------------------------------
>> Section 5:
>>
>>
[19]- Conformance clause:
>> - in (2) and especially (3): should we
replace "test assertion parts"
>> with "test assertion constructs",
because in section 4 these are not
>> TA parts per say, but more
generally higher level constructs as well.
>
> I
agree
>
>> - Should we also consider a second kind of conformance
target, beside
>> "representations":
>
> As per last
meeting, I'm not sure the model spec should give
> conformance clauses
about conformance to all representation specs. I
> would leave that to the
representation spec writers. The users of a
> markup or other
representation might not even have access to this
> model spec's
conformance clause.
<JD> OK
>
> I do appreciate, however, that in some cases there may not
be a
> representation spec but here it would be difficult to claim
>
conformance anyway to the model (and maybe not important).
>
> I'm
open to discussion though - maybe in view of any public review
>
comments.
>
>> By extension, instances of Test Assertions can be
said conforming to
>> this model, if:
>> - if they use a "TA
representation" that is itself conforming.
>> - or even if they use a
simple informal notation not defined by a
>> formal representation,
they must use this notation in a way that is
>> unambiguously mapped to
this model and its class system, e.g.
>> respecting cardinalities and
using unambiguous names.
>> (I am thinking of the informal notation we
introduce in the
>> Guidelines, for
>>
examples...)
>>
>