[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [tag] TA Model
Quoting Dave Pawson <dave.pawson@gmail.com>: > On 22/09/2007, stephen.green@systml.co.uk <stephen.green@systml.co.uk> wrote: >> So how about >> >> 1. introducing steps called 'Test' into the flow with IDs, >> 2. making such 'Test' steps multiple occurrence > .. of what Stephen? > >> 3. adding a unique ID to each part of the step >> 4. allowing preconditions and postconditions to reference parts of another >> step > > Why? IMHO a test should be independent. Dependencies on other tests goes > against this. Perhaps such a test is at too low a level? An example I'm aware of where a test depends on the pass outcome of a previous test is a Schematron validation test isn't worth performing (and therefore can't result in a pass) unless previously there was a successful pass outcome to a test of the XML syntax such as with XML Schema or RelaxNG. If the XML Schema validation test was in a previous TA then there would be a need to make that TA's test result a pre-condition which I propose could either be in the pre-condition description or be by a reference to the TA test's result or to the TA as a whole. Hence the need to have an ID in the TA as a whole but also, I think, in the Test result to allow greater precision. After all the pre-condition might be to a post-condition of the previous test - to something that isn't actually tested formally but is just found to be true (boundary clarity needed here?). There may, however, be reason to include the XML Schema validation in the same TA as the Schematron validation (feasible but I'll take it as an unproven assumption) but to put the two validation phases into two tests in the flow in such a way that the dependency can be expressed formally. Say the second test step of the flow uses an ID from the first step to express the dependency. It could include a reference in the pre-condition of step 2 to the first step's result (which requires a unique ID on that result). So to summarize so far, things like pre-conditions might need to reference either things like triggers, results/effects (I too prefer 'result' to 'effect'), pre- and post-conditions in tests so each need unique IDs *or* they may need to reference the final outcome of a test flow as a whole so that needs an overall unique ID which could just be the ID of the TA. I could imagine there might be some situations where the pre-condition might need to reference something other than the entire TA or the result of a particular step/test in a multi-step/test flow. If there is an aspect of the referenced TA or step which isn't actually formally tested it could be something which is a post-condition. So the post-conditions need unique reference too to facilitate that. Either way, I see these as all aspects of a spec (the example is part of the in-progress OASIS Codelist Methodology and plays a part in UBL's spec) and therefore to need TAs and not therefore just for the test designers. They live in the space between spec and tests in such cases. (Two-phase codelist validation is the example I worked from.) Regards -Steve > > >> 5. adding 'Previous Test' and 'Following Test' pointers to each step > > I find this confusing. Test flow / routing shouldn't be set by the test. > It may be that a test could be re-used in different places? > > >> 6. adding 'Comment' outside the flow >> 7. adding something to state what constitutes overall pass or fail of TA >> 8. adding specification quote > or reference. By whatever means are appropriate. >> 9. being explicit in the name of the part about it being an ID or reference > > I don't understand this one. > > >> >> Is that too complex or just complex enough? Maybe most things are optional >> I suggest where there is a reference such as 'Test Trigger Cross Reference' >> then it could be implemented as a hyperlink if the implementation is HTML >> (so it might not add complexity all that much). >> >> Extended example: >> >> Test Assertion ID: TA-101c >> Specification requirement ID: < req 101> >> Specification requirement quote: "Purchase Order Receipt documents >> must be XML documents that are valid against the XML schema >> po-receipt.xsd, and that must use the approved format for product >> identifiers." >> Item under test: A business endpoint. >> Assertion prose: "Purchase Order Receipt documents must be XML >> documents that are valid against the XML schema po-receipt.xsd, and >> that must use the approved format for product identifiers." >> Comment: Test TST#112 is only required when test TST#111 results in >> a warning >> Test Assertion Pass Criteria: Test Assertion results in 'pass' if no >> test result is 'fail' and all required tests are run. > > Except for the test criteria, a good example. > Pass conditions must be explicit and UUT oriented. This would might be > 'Parser must not report any errors when validating the instance. > > Other than that this seems to be about the right 'level' of abstraction. > Says 'what' to test. Not how. > Is independent of other tests. > (Might add a precondition: An xml instance - PO receipt, has been received > by the end point). > Post condition might be, PO receipt on disk as file identified by > ...PO ident number > with XML extension. > > I'd generalise that the 'right' level of abstraction is at, or > slightly lower than, > the sentence level in the specification. The usual caveats about no > conjunctions > etc. Unsure if its reasonable to go much beyond such a generality. > > > > regards > > > -- > Dave Pawson > XSLT XSL-FO FAQ. > http://www.dpawson.co.uk >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]