Re: [oiic-formation-discuss] Acid Tests (was: Re: [oiic-formation-discuss] Interoperability versus Conformity)

["Sam Johnston" <samj@samj.net>]

On Thu, Jun 12, 2008 at 4:03 PM, Dave Pawson <dave.pawson@gmail.com> wrote:

2008/6/12  <robert_weir@us.ibm.com>:

> "Peter Dolding" <oiaohm@gmail.com> wrote on 06/12/2008 07:49:30 AM:
>> Its a simple technical reason.   W3C CSS2 test suite is harder to use
>> compared Acid test.   Anyone with even basic skills can point there
>> browser to the Acid Test and see result.  More experience is need to
>> use CSS2 test suite and the like.   Note the W3C test suites are older
>> than the Acid Test.   Acid Test make problems more displayed.

> OK.  So reading between the lines, I'm getting the impression that one of
> the important goals of an Acid-type test would be that it can be run by
> anyone -- end users, journalists, i.e., non-geeks.  They don't need to first
> download and install a JDK, ant, and a dozen XML tools first.   They just
> load the document in their editor and see how it looks.

And (me also reading between the lines), these LSD tests are also automated?
Could be a contradiction there.
I've put a stupidly simple definition of a 'tester' on the google site.
I'd love anyone to run a test suite. It's more likely to be a geek though.
Even if he does have to follow a dumb install document first.

 But we may also
> have another set of tests (I've been calling them 'atomic tests') that are
> targeted to vendors and testing labs.

+1, if an atomic test tests all or part of one para in ODF standard.

Yes, an Acid Test would need to be easily executed, which means opening a doc from a website (perhaps a dense, multi-column beast with embedded charts, sheets, etc., or a number of separate components/pages) and comparing it to an image/PDF reference and/or uploading a doc to a website for online analysis. I like David's definition of an Acid Test as a 'comprehensive test [that concisely?] renders the results visually in a manner that makes failures immediately obvious to the casual observer' by the way.

Traditional test suites also have their place, in testing and certification labs, interop events, implementors, etc. but the bar is too high and the output too boring for consumption by the general public. They also tend to involve more work (2PY for ODF?) and have a hard time keeping up with faster moving standards. There has been some (good?) work done already in this area for ODF which we can reference/consume.

I suggest that the TC have /both/ deliverables; with any luck the Acid Test will build up enough momentum on its own that the TC can focus on a more formal approach, and in the worst case the TC will have some good design notes/ideas/starting points.

Sam