Suggestion of how to define tests for ODF 1.2 - part 1 - ODF Schema

[Svante Schubert <Svante.Schubert@Sun.COM>]

Currently the OIC TC is already collecting only test documents and we 
all agree the TC will not implement tests.
The test execution is work that might be delegated to the ODFDOM working 
group, I am chairing.

What the OIC TC might work on, is to provide test descriptions for ODF 
feature.
Let's focus for now on part 1 the schema part feature  (e.g. table,
image, hyperlink, etc.).
Previously I had explained that several XML elements (and their
attributes) form such an ODF feature (see
http://lists.oasis-open.org/archives/oic/201001/msg00043.html and 
http://lists.oasis-open.org/archives/oic/201002/msg00005.html ).
To summarize an ODF feature, it is just a high level abstraction of its 
ODF XML model,
providing all its ODF functionality, but hiding all XML
implementation details.

In the following I would like ot show you how  a test is being defined 
for such an ODF
feature.
This creation process can be divided into a sequence of several steps, 
again to break down complexity.
The first step is to identify the XML of an ODF feature. Let's start an 
example with an easy feature, an ODF image.


1) Analyzing the Feature XML
==========================================
The easiest way to get an idea what XML elements are involved is the
empiric way, starting with the most easiest incarnation of the feature.
For an image, it's most likely starting a new text document - in the ODF 
implementation of your choice - and
adding a new image to the document. Afterwards take a look into the XML.

[Note: For the XML analysis I recommend the use of the cross-platform JEdit
OpenSource tool with its Archive and XML plugin:
- with the Archive plug-in you open the files within the zip. Before
opening an ODF document in the open file dialog, choose from the menu
plug-ins Archive - browse Archive, to open explicitly an arbitrary
inner package file.
- with the XML plug-in you are able to (highlight &) indent the XML:
I have a key short cut for this frequent task (CTRL-SHIFT-I).
Nice benefit, when using JEdit, you may edit and save the XML directly
into the ZIP.]

In case you used Open Office 3.2 you might get something like this
<draw:frame draw:style-name="fr1" draw:name="graphics1" 
text:anchor-type="paragraph" svg:width="16.563cm" svg:height="8.546cm" 
draw:z-index="0">
      <draw:image 
xlink:href="Pictures/1000000000000272000001430108E8B5.jpg" 
xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
</draw:frame>

To know what this all means you take a look into the element definitions 
of the part 1 spec:
http://docs.oasis-open.org/office/v1.2/part1/cd04/OpenDocument-v1.2-part1-cd04.html#element-draw_frame
http://docs.oasis-open.org/office/v1.2/part1/cd04/OpenDocument-v1.2-part1-cd04.html#element-draw_image

Each element definition describes all possible parent elements, 
attributes and child elements.
The test should cover all elements & attributes possibly used for the 
feature.
The easiest image would be the one with smallest XML set (therefore 
remove the optional parts from above XML) to:
<draw:frame>
      <draw:image 
xlink:href="Pictures/1000000000000272000001430108E8B5.jpg"/>
</draw:frame>

This is the smallest possible image and still valid ODF 1.2 (test your 
doc with http://tools.odftoolkit.org/odfvalidator/ )


2) Assigning User Scenario to the XML of the Feature
==========================================
Usually in life you come up first with scenarios and do not have the 
model (ODF XML) first.

Still it is worth to add scenarios, when having the XML model already.
The reason is to test if the given XML is capable of the requirements 
from the scenarios and even more to reuse the scenario later to hook a 
user by offering functionality documentation (his problem) to the 
certain XML.

The first (main) scenario would always be to create the minimum feature 
(image XML shown above).
Actually it is already the first set of scenarios, as this minimum image 
have to be added to all places it is allowed for (see possible parent 
elements of draw:frame in spec, for instance it is possible to add an 
image to a table cell).

Possibly the second scenario, as the minimum image has in Open Office 
only the size of a thumbnail would be to change the width and height of 
an image.
This scenarios would be aligned to altering the attributes @svg:width 
and @svg:height.

We might start with two rule of thumb when creating scenarios:
a) Add as much scenarios as possible (making them as small as possible)
b) Do not add redundant scenarios, that are already covering by others 
(do not have 'change height', 'change widht', 'change both') To keep 
your API simple and tidy (Remember —Antoine De Saint-Exupery:
“A designer knows he has achieved perfection not when there is nothing 
left to add, but when there is nothing left to take away.”)

By this we would have (aside of the adding scenarios) two scenarios 
additional. One to change @svg:width and one for @svg:height.


3) Describing a Test for each User Scenario
==========================================
After the feature is crushed to scenarios - - each explaining common 
user functionality -, a test is now described for each scenario. To ease 
test description a pseudo method is being invented for each scenario, 
providing a method signature, which is the method name, method 
parameters (optional) and return value  (optional).

There are three things the test definition requires:
a) a precondition  (Document status - usually via a test document URL)
b) a modification of the test document via the method (plus arguments)
c) a postcondition (Document status - usually via a test document URL 
and/or expected return value)

As usually in testing not everything can be covered by tests (e.g. not 
all possible length of an image or colors of a cell), but different 
status (and their border values) should be covered. For instance in 
terms of a color: no color, black, white, some arbitrary color, 
transparent).


The nice thing of the above approach is that each of these three steps 
is reviewable for itself by others.
And further, as ODFDOM maps those pseudo method of the test description 
to its own Convenient API (see 
http://blogs.sun.com/GullFOSS/entry/odfdom_0_8_the_new - esp. text 
beyond picture), it is more than likely that these test description will 
be readable by a generic test tool, which executes each test description 
triggering a method of that Convenient API, using preconditions (test 
documents) and compares the expected post conditions (test documents).


Imagine if all ODF XML would be covered by such features and their 
scenarios.
The set of all scenarios would define all possible valid modifications 
(or state transitions) of an ODF document.

Does this sound interesting to you?

Kind regards,
Svante