Re: [docbook-apps] docbook testing

[Stefan Seefeld <seefeld@sympatico.ca>]

DavePawson wrote:
> Tony's post made me start to think about what form / category(ies?) of 
> testing is needed.
>
> From that thread, clearly testing with different processors is required.
> Does that mean we need to test xsl-fo output and (x)html output, 
> against some expected values? That smells like using some form of 
> XML-diff engine; ndw uses http://www.deltaxml.com/ which is costly and 
> commercial.
>
> I guess it would be possible to script running against multiple
> xslt engines (or some subset of all) followed by *some* form
> of comparison. Schematron?

I think this has to be an incremental process, as even *some* testing is 
better than no testing. In particular, I can see different steps:

1) Run different XSLT processors / configurations on the various 
stylesheet sets ((X)HTML, FO, man, etc.) and make sure they don't fail.

2) validate the output for syntactical correctness (xmllint without 
validation for XML output, say)

3) validate against document types, if applicable (xmllint with 
validation, etc.)

4) validate final output by visually inspecting it.


The last step (4) is obviously hardest, and almost impossible to 
automatize. But even if you only do (1) you already catch those errors 
that made the 1.74.1 release a failure.

And *please* make sure the testing procedure you implement is applicable 
by ordinary people (e.g., doesn't involve commercial tools), as that 
will increase chances that contributors will run it, too.

Thanks,
       Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...