some thoughts on test documents

[Bart Hanssens <bart.hanssens@skynet.be>]

Hi,


Some thoughts and questions on getting started with the test documents.

Dave, Rob and others already made several detailed comments about this
topic on the formation mailing list, see also
http://sites.google.com/a/odfiic.org/tc/Home/odf-interoperability-and-conformance

IMHO, we should start with the atomic documents.
Now, the opendocumentfellowship has already created a test suite:
http://develop.opendocumentfellowship.com/testsuite

Maybe we can reuse it (if the fellowship agrees with this) or do
something similar, minus the comments on specific implementations, but
adding remarks on ODF or the testing itself (like "spec not clear")

We could create a Wiki with a page per test document, and an upload
directory on the OASIS website (as mentioned during the TC conf call,
the OASIS Wiki doesn't support file upload) like:

ODF_1_1/
- atomic/
-- 5_Para_Elements/
--- 5_3_1_Note_Element.odt
- complex/
...
ODF_1_2/
- atomic/


And if anyone feels like creating a document for a specific item, he/she
can mention it on the wiki, create a test and upload the file.

Once a week/month/..., a snapshot can be made and archived in a ZIP file
for convenient downloading.


This doesn't necessarily mean going through the whole spec in sequential
order, we can start with areas that are known to have issues (I would
hint: forms, gradients, slide effects, charts...) or easy to test


Questions:
- do we create atomic tests with an office suite, or craft it by hand to
make it really as small as possible ?

- should we do ODF 1.0, now that many (all ?) vendors use 1.1 or 1.2
draft ? Personally, I think we should start with issue-prone parts of
1.1 and 1.2's OpenFormula.

- who will run the tests and report the results, so that the OIC can
create a summary and general recommendations ? Per charter, the OIC
won't be commenting or identifying implementations (at least not in
reports, but isn't a wiki some kind of report ?)


Best regards,

Bart