Re: [docbook-apps] docbook testing

[DavePawson <davep@dpawson.co.uk>]

DeanNelson@aol.com wrote:

>      > Smoketests for xsltproc and saxon are now part of the release
>     process.
> 
>     I like the phrase! guessing it means 'something' is transformed?
>     Does that mean the entire test suite?
> 
> Dave,
> Smoke test is a phrase dealing with an immediate failure of a piece of 
> electronics causing rising smoke when it is initially powered up. Akin 
> to "Letting the Smoke out".

My electronics background made sense of the term - just that I'd not
heard it used in connection with software! I can imagine Mike Kay
being a little disgruntled when he sees smoke pouring from Saxon :-)

>  
> In this case is should be a very quick indication of failure or 
> catastrophic failure of the build. It doesn't exhaustively test the 
> system, just a quick indication of go/no go.

good point. The sort of testing needed when a change causes a rebuild.

** Test partitioning.

Build tests:
b1.  Smoke tests (seek catastrophic failure)

Pre-delivery tests:
p1.  Full set of test documents run through, expected vs actual comparison.
All formats.

XSLT engine tests:
e1. Some subset of p1 to check for faults down to one XSLT engine.
Issue: Which engines. xsltproc, Saxon1, Xalan. All latest versions only?

Format tests:
f1. html. html only from p1
f2. fo.  Some form of comparison of fo output. Which processors?
f3. PDF? Not automatable? Consider Tonys suggestion?
Issue: which processors? xep, ah, fop?
f4. htmlhelp.
Issue: How to compare output? Any suggestions please?
f5. Man pages?
Issue: How to compare output? Any suggestions please?
Issue: What input material do we have suitable for this?
f6. xhtml. As per html
f7. epub?
f8. .rtf?


What have I missed?
Do the tools need testing?
Website? Slides?

Plenty to chew on.


regards

-- 
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk