Re: [oiic-formation-discuss] Acid Tests

["Luc Bollen" <luc.bol@gmail.com>]

Dave, I think we are not discussing about the same things.  Obviously, you are considering a comprehensive test suite, made of hundreds or thousands of tests, many of which requiring manual intervention.  The 2 approaches discussed here are exactly to avoid the inconveniences of such "heavy to use" test suite.

- Acid Test: the purpose is not to be a comprehensive test, but to test a few dozen of problematic features, in the same way as existing for browsers (see http://en.wikipedia.org/wiki/Acid2).  Please see by yourself and test your browser at this location: http://www.webstandards.org/files/acid2/test.html

- Validator: again, please make a test yourself by submitting any ODF file at the following location: http://opendocumentfellowship.com/validator

You will see that you get a feedback in a few (dozens of) seconds.  Of course, the feedback is not comprehensive, as the test is not comprehensive, but I think it is well adapted to the needs of a lambda user : simple and fast.

On Fri, Jun 13, 2008 at 10:09 AM, Dave Pawson <dave.pawson@gmail.com> wrote:

2008/6/13 Luc Bollen <luc.bol@gmail.com>:
> On Fri, Jun 13, 2008 at 9:36 AM, Dave Pawson <dave.pawson@gmail.com> wrote:

>> Finance.
>>
>> Let's make a guess. Somewhere between 5-60% of the tests will require
>> manual intervention.
>> Who will host the site, maintain the service. No, not Oasis.
>> Would you use it if you had to pay for it?
>> Who would use it? Vendors ( <6?)
>> Large corporations... Possibly.
>> .gov organisations? Possibly.
>> Others?
>> I don't think it's financially viable in this format.
>
> Dave,
>
> Finance:
> - Approach 1 (Acid Test) have no recurrent costs: once the Acid Test
> document is ready, it can be hosted on OASIS servers like any other report.

If we could write 'right first time' software we would.
We don't. Same with documents.
As ODF evolves so will this. There will be recurrent cost.

> - Approach 2 (Validator) has recurrent costs, but we can collaborate with
> groups already offering such a service, like the OpenDocument Fellowship
> (http://opendocumentfellowship.com/validator)

If you could document how such work would be financed, or where
sufficiently skilled volunteer effort would be sourced, and if how
you could convince such as the EU to accept the output of such a group
of volunteers, then I'd accept it Luc. I think this chain of logic is too slim.

>
> Manual interventions:
> I don't think any manual interventions will be required for both these tests
> (unlike using a large test suite made up of hundreds of atomic tests)

ODF is ~800 pages. I estimate ~ 20 tests per page. Of the ones I looked
at most were manual.
I doubt a fully automatic test suite would be acceptable against
the current ODF standard.

>
> Users:
> - Acid Test: anybody willing to use an ODF consumer tool (an editor, a
> viewer...) could be interested to validate it using the Acid Test (it just
> take a few seconds to run)

2000+ tests? How many profiles? Across the net? A few seconds?
How about a full day for a reasonable test suite that produces a
convincing result? That is conservative. That excludes applications
compliance. That excludes applications interop.

> - Validator: anybody developing an ODF producer could be interested to
> validate the generated ODF files, as well as anybody will to use such a
> tool.  Again, it will take only a few seconds to submit a produced file to
> the validator service and get the report.

I don't think that estimate is realistic Luc.

regards




--
Dave Pawson
XSLT XSL-FO FAQ.
http://www.dpawson.co.uk

---------------------------------------------------------------------
To unsubscribe, e-mail: oiic-formation-discuss-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: oiic-formation-discuss-help@lists.oasis-open.org