Re: [tag] comments on the TA model

[Stephen Green <stephen.green@documentengineeringservices.com>]

response inline


Best regards

Steve
---
Stephen D Green




2010/1/19 Jacques R. Durand <JDurand@us.fujitsu.com>:
>
> Here is some late (sorry..) comments on the TA model: only minor things.
>
> Nothing I see that would justify waiting any longer for a CD vote and PR.
>
> However, if we have time to go through (dome of) these in the meeting,
> resolution is obvious and
> could easily be left to the Editor, if we agree.
> Overall, I feel very good about this document - very crisp and precise.
>
> As a priority, I'd focus on these two that may be more than editorial:
>
> [11]
> [19]
>
>
> Regards,
> Jacques
>
>
> --------------------------------
> Section 1:
>
> [1]- def of "association" is not very clear... it seems recursive.

I agree, will change to:

The term “association” is used of an entity which is grouped by a
class and is itself based on a class (rather being based on a
primitive or simple datatype).

>
> [2]- def of Test Case : could include a mention that "Test Cases usually are
> grouped
> in a Test Suite."
>
> --------------------------------
> Section 3.1:
>
> [3]- Informal names for TA parts: qualifying the names with "Test Assertion"
> (e.g. Test Assertion Target) seems redundant, and inconsistent: we only do
> that for
> Test Assertion Identifier and Test Assertion Target (why?). Suggest to
> remove.

OK, but we have this kind of thing in the Guidelines where it seems
more appropriate (e.g. we have 'TA Id', not simply 'Id'). I have had a few
tussles with where 'Test Assertion' is redundant and wheere it is
necessary to eliminate ambiguity and vagueness.


>
> [4]- Inside the def of Target: suggest to add a "note: a test assertion may
> concern
> more than one [part of] implementation. It is recommended however to select
> only
> one of these as the Target."

Not really sure I agree. I think it may be an oversimplification
in some ways to only have a single target - in logical terms.
I've read opinions that a predicate can have more than subject
or target when the predicate describes a relationship between two
subjects/targets:
e.g.
[A] is less than [B]

I agree we have in the markup simplified it by rolling all targets
into one and for the sake of tools convenience decided that
in some implementations just one target can be denoted as
'primary'. However, I could imagine other ways to implement
the model which it seems unfair to restrict in this way. I'm
thinking some representations could be optimised for scenarios
where relationships between two or more targets are the focus,
e.g. TAs about tables/fields in a relational database or classes
in an API. I'm not even sure we make a normative requirement
about this in the markup. Multiple targets I'm OK with but not
sure we should insist on one being denoted 'primary' (which
seems to be arbitrary and related to specific tools which could
probably deal with any limitation they have by taking the first
target mentioned as 'primary' by default).

Have I misunderstood?

>
> [5]- Def of Prerequisite: its las sentence is a repeat (said before).

Agreed. Good catch

>
> [6]- Def of "Tag":
> - remove the (s)
> - remove the references to the reader
> ("these tags provide you with..." --> "these tags provide an
> opportunity...")
> ("they enable you to group..." --> "they allow for grouping")
> Same for the "Variables" definition --> Variable.

OK

>
> --------------------------------
> Section 3.2:
>
> [7]- same as [3], for the names in the table.

OK


>
> [8]- in the testAssertion formal def: paragraph before last: it seems to
> suggest that
> several prerequisites are OK ("... together or multiple prerequisites...
> A test assertion may have prerequisite(s)". Remove these allusions.
> Confusing as we clearly have just one Prereq.

Well. I kind of agree but with reservations. In the markup we
have a policy that we combine prerequisite components into
one expression. In modeling terms though I think the distinction
with, say,

TA(A) = 'pass' AND TA(B) = 'pass'

between calling "TA(A) = 'pass'" one of two prerequisites here
and calling "TA(A) = 'pass' AND TA(B) = 'pass'" a single
prerequisite is so fine a distinction that I don't think we can
make using one or other of these a requirement. Some could
say that "TA(A) = 'pass' AND TA(B) = 'pass'" is a combination
of two prerequisites and that they could indeed be combined
using an expression, as here, or using some other construct
in the representation. I just wouldn't see it as the place of the
model spec to say whether prerequisites should always be
seen as one prerequisite combining the separate parts.

Still, I'll have a look at the details of what changes would be
required in the text.

>
> [9]- Formatting: each new construct that is introduced as : "Formal
> Definition:"
> is not highlighted enough: only the root constructs are, e.g.
> normativeSource.
> I wish its parts would also be highlighted in some way, e.g.:
> Formal Definition for refSourceItem:", or with a title, even if at same
> level as normativeSource (for refSourceItem, derivedSourceItem...)

Agreed.

>
> [10]- when introducing textSourceItem and derivedSourceItem: it is said they
> are "alternatives" to other options, suggesting exclusiveness.
> Make it clear they could be "an alternative - or a complement - to.."

Agreed.


>
> [11]- derivedSourceItem: why does it have a complex structure like
> refSourceItem?
> Should be much simpler like textSourceItem.


I think we may need to discuss this a bit. I hadn't envisiaged
both a refSourceItem and a derivedSourceItem or both a
textSourceItem and a derivedSourceItem bing used for the
same requirement unless the requirement consisted of
several sources of different kinds. Therefore to properly
model a single source item it doesn't seem right to me to
use two distinct source item types - not unless they are in
fact two separate source items. So for this reason I thought
we would need the derivedSourceItem to be able to say what
source item(s) it is derived from, without having to combine
it with a refSourceItem. It therefore, I had thought, would
have the structures needed to make reference(s), as well as
the structure to add some derived text.
>
> [12]- definition of "interpretation":
> We could add: "It provides additional clues about how the Predicate (or
> Prereqisite)
> relates to the normative source."

I'm OK with something like that

>
> [13]- Target def: "The Target shall be the subject of the test assertion..."
> Because of [4] above, say rather: "The Target shall be the primary subject
> of
> the test assertion.."

Again, as above, I'm not sure about this one

>
> [14]- def of Variable:
> replace:
> "...using a notation such as $variable1"
> with:
> "...e.g. using a notation such as $variable1"
> Also, add that " a variable may also contain a logical expression, e.g. a
> subset
> of a Prrequisite or Predicate."

Well, strictly speaking the 'notation' part would
be treated as normative but not the example so
to move the 'notation' into the example does
have the affect of making it vaguely informative.
Perhaps this is the intention. Perhaps it should
be removed altogether if it shouldn't be normative.
Some kind of notation is needed in our markup
to use a variable by name within an expression.
Do we not want to state this as a requirement?

>
> [15]- def of Report:
> Replace:
> "the circumstances which shall be true for the report to be generated. It
> shall contain an expression
> which evaluates to true or false."
> with:
> "the circumstances in which this report item should be generated. It shall
> contain an expression
> which evaluates to true or false, or refer to such an expression (e.g. as
> contained in some variable) ."

No longer relevant as we are removing 'report'

>
> --------------------------------
> Section 3.3 (Semantics)
>
> [16]- replace:
> "As a test assertion has parts that can be evaluated over a Target instance
> (i.e. the prerequisite and the predicate),..."
> with:
> "As a test assertion has parts that can be evaluated over a Target instance
> (i.e. the prerequisite, the predicate, and possibly some variable containing
> expressions),..."

OK (though I would think that resolving the variables would
be part of the interpretation of the predicate and prerequisite
and would happen before evaluation of the TA).

>
> [17]- use bold font for the semantic outcomes ("Target not qualified", etc)

Agreed

>
> --------------------------------
> Section 4:
>
> [18]- as a gneral rule, there should be a short introductory sentence
> between the sub-Titles and the
> formal def of the construct they announce:
> E.g. below testAssertionSet and its formal def, add:
> "Test Asserions may be grouped using a testAssertionSet class."
> - Same for testAssertionRef: move up the sentence :
> "A test assertion set may refer to one or more test assertions by their test
> assertion identifiers rather than
> include the test assertions literally within the set."
> - Same for testAssertionResource:
> "testAssertionResource is used when test assertions are contained in another
> external document."

OK

>
> --------------------------------
> Section 5:
>
> [19]- Conformance clause:
> - in (2) and especially (3): should we replace "test assertion parts" with
> "test assertion constructs",
> because in section 4 these are not TA parts per say, but more generally
> higher level constructs as well.

I agree

> - Should we also consider a second kind of conformance target, beside
> "representations":

As per last meeting, I'm not sure the model spec
should give conformance clauses about conformance to
all representation specs. I would leave that to the representation
spec writers. The users of a markup or other representation
might not even have access to this model spec's conformance clause.

I do appreciate, however, that in some cases there may not
be a representation spec but here it would be difficult to claim
conformance anyway to the model (and maybe not important).

I'm open to discussion though - maybe in view of any public
review comments.

> By extension, instances of Test Assertions can be said conforming to this
> model, if:
> - if they use a "TA representation" that is itself conforming.
> - or even if they use a simple informal notation not defined by a formal
> representation, they must use this
> notation in a way that is unambiguously mapped to this model and its class
> system, e.g. respecting
> cardinalities and using unambiguous names.
> (I am thinking of the informal notation we introduce in the Guidelines, for
> examples...)
>