Re: [tag] TA Model

["Dave Pawson" <dave.pawson@gmail.com>]

On 23/09/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote:

> > Why? IMHO a test should be independent. Dependencies on other tests goes
> > against this. Perhaps such a test is at too low a level?
>
> An example I'm aware of where a test depends on the pass outcome of a
> previous test is a Schematron validation test isn't worth performing
> (and therefore can't result in a pass) unless previously there was a
> successful pass outcome to a test of the XML syntax such as with XML
> Schema or RelaxNG.

Test flow issue.
 The Schematron test shouldn't be run, as determined by the flow control.
 Not down to the test. The Schematron test should remain independent.



> There may, however, be reason to include the XML Schema validation in the
> same TA as the Schematron validation (feasible but I'll take it as an
> unproven assumption) but to put the two validation phases into two tests
> in the flow in such a way that the dependency can be expressed formally.

I'd separate those.
I'd expect the latter not to be called.   A pre-condition would be
belt and braces, requiring the accumulation of test results somewhere...
That's getting too close to 'how' for my liking.



> Say the second test step of the flow uses an ID from the first step to
> express the dependency. It could include a reference in the pre-condition
> of step 2 to the first step's result (which requires a unique ID on that
> result).
If the two were separate, then the entire TA (rng validation say) would
be the unit level, and the item with the ID.
I don't think its needed beneath that level, other than internally
to the test. Data hiding ideas.




>
> So to summarize so far, things like pre-conditions might need to reference
> either things like triggers, results/effects (I too prefer 'result' to
> 'effect'), pre- and post-conditions in tests so each need unique IDs
> *or* they may need to reference the final outcome of a test flow as a whole
> so that needs an overall unique ID which could just be the ID of the TA.

Yes.

>
> I could imagine there might be some situations where the pre-condition
> might need to reference something other than the entire TA or the result
> of a particular step/test in a multi-step/test flow.

I'd view that as bad design. Keep each test black box 'ish.


 If there is an
> aspect of the referenced TA or step which isn't actually formally tested
> it could be something which is a post-condition. So the post-conditions
> need unique reference too to facilitate that.

If part of a test doesn't run, then that would need agreement with the
design authority, and presumably would allow the test to pass.
If that's the case, then the black box model still works.




>
> Either way, I see these as all aspects of a spec (the example is part of
> the in-progress OASIS Codelist Methodology and plays a part in UBL's spec)
> and therefore to need TAs and not therefore just for the test designers.
> They live in the space between spec and tests in such cases. (Two-phase
> codelist validation is the example I worked from.)

The tests can't be written well without design authority sign off.


regards

-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk