[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [oic] first impression on conformance tests ? - <office:metadata> <dc:creator> element.
Bart, I was taking a quick pass through your first test, and I deep-ended analyzing what little the ODF 1.1 specification has to say about <office:metadata> and its <dc:creator> predefined element. Here are my disorganized notes. I am not sure how to make something simple and sensible out of them. I haven't answered your question. I mostly have questions (4, below) and a rough assessment of the situation analyzing your tests reveals to me (5, below). I don't think the test procedure in metadata-dc-creator.html should be called a conformance test. I don't think there is much of a conformance requirement to be found with respect to the (optional) <dc:creator> element of the (optional) <office:metadata> element, or much else about <office:metadata>. I DO THINK THESE TESTS ARE THE START OF SOMETHING USEFUL. First, they are invaluable in what they reveal to us about what is testable, and secondly they are important for what they provide that others can confirm for themselves. Based on my analysis of the ODF 1.1 specification and operation of the test document, I think the situation is relatively hopeless with regard to much meaningful (and measurable) conformance of <dc:creator> in <office:metadata> for ODF 1.1. (I suspect there is no improvement for ODF 1.2, at least not as of cd01.) [I am also distressed by the liberties taken in redefining something in the Dublin Core namespace, something that I say the ODF TC is not entitled to do.] There might be something to be done with regard to interoperability. And it is useful to have confirmation tests that demonstrate whether and how a feature is supported by providing point cases. In short, I like this kind of test, but I think we need to find out something better to call them. In particular, they should not be called conformance tests if we can't find a specific conformance requirement against which the test provides demonstration. - Dennis THE NOTES 4. WHAT DO WE CALL THESE TEST CASES? I notice we are referring to test scenarios as well as application scenarios. It is clear that the scenarios here are test scenarios or procedures. What kind of tests are they? What should we call them? I am curious because I think of tests like this faster than I can write them down. To me they are some kind of point tests or basic feature point-confirmation tests. They may find some sort of boundary (e.g., a discretionary limit on the length of the <dc:creator> content string, the degree to which language-specific Unicode features are supported, tolerance for format-effectors in the string) testing, but mostly these are establishing single points in the domain of a feature's cases and provide a simple kind of conformation to indicate whether there is some sort of provision for the feature or not. I think there should be test cases like this. I don't know what to call them. I am thinking about confirmation tests or feature confirmation and demonstration tests. These are simple tests, or point tests in that they make simple probes into the features set of potential instances as a way to reveal something simple. I don't know what there is that would distinguish these from something farther toward comprehensiveness, but I think it would be good to think about terms that would work. We shouldn't refer to them as conformance tests unless we can back that up with normative requirements expressed in the specification. 5. WHERE DO THESE TESTS FIT? In the face of the incredibly weak requirement here (along with the overall weak conformance requirement for <office:metadata>), there is very little to test for conformance of. It appears that all we can do is (a) provide assessment procedures for confirming whether an implementation supports the element or not (and what the degree and nature of that support is), and (b) provide assessment procedures for confirming whether, if there is support, it is consistent with the definition in the ODF specification to the extent there is something to be consistent with (c) provide assessment procedures for determining the degree to which the implementations are consistent with each other (i.e., what one produces another consumes in a consistent manner), (d) provide assessment procedures for determining the degree to which disparities in support by an implementation are handled without failures and corruption of documents I don't know which of these are clearly under our brief. I think (a-b) would fit. It would be useful to encourage some sort of arrival at (c), but (d) is probably not our business at all. MORE NOTES 1. MODEST CONCERNS ON THE TEST ZIP 1.1 Connecting the Pages. There need to be some instructions in the Zip about how to unpack it in a way where the instructions work. 1.2 Using a Working Copy of the OIC SVN, the pages come up nicer than directly in the Zip of course (because the CSS file is found), and the links work. However, my browser won't launch the documents/ODF11/metadata-test.odt file from the scenarios/html/metadata-dc-creator.htm page. The browser wants me to authorize an unsigned so_activex.dll and I am not going down that road. (All tests on a Windows XP SP3 Media Center PC configuration.) 1.3 I notice that the OIC SVN has svn:mime-type application/octet-stream for metadata-test.odt. Although the .svn information and SVN properties do not appear in the Zip, I think the svn:mime-type property should be set on the document files in the OIC SVN. I am not sure what sort of guidance there should be about this that flows into the Zip-carried documentation so folks using that version of the material know what to do concerning accessing the documents properly via the HTML pages (or not). [I notice we are here seeing a whole set of point cases around ODF documents represented in packages and ways they might be accessed and used. These tend to be external to the scope of the specification, but I find them rather interesting for establishing contexts for interoperability.] 1.4 I agree there also needs to be more documentation on the making of these tests, but my first-order inspection has to do with ensuring that there is sufficient information to *use* them. 1.5 There needs to be a way to ensure the documents are kept as read-only for testing purposes. I noticed that my test application wanted to update the metadata-test.odt document, although I hadn't knowingly modified it. This might be additional guidance, since Zip implementations won't always transfer and preserve that kind of file-system attribute settings. 1.6 I notice that the hand-crafted meta.xml file has an XML comment about ODF 1.1 errata. What errata are those? 2. FURTHER CONCERNS Looking at the metadata-test.odt file and the cited passage of the specification, I have further concerns. 2.1 The implementation that I am using shows me an empty document. 2.1.1 When I select the File | Properties dialog of that application for that document (typical for a UI based on CUA principles), on the "Properties of metadata-test" dialog window I see three items in which there is some sort of named principal under items "Created:", "Modified:", and "Last Printed:" (and my application is installed with en-US localization). 2.1.2 The Created entry is "12/26/2008, 20:57:01, John Doe (init)" (reflecting <meta:initial-creator> and <meta:creation-date> content and the Modified entry is "01/07/2009, 21:59:12, John Doe (create)" reflecting the <dc:creator> content "John Doe (create)" and the <dc:date> content. I notice that the date and time are not shown in accordance with my system's default format settings, but that is not exactly an ODF implementation problem. 2.2 The problem I have with this is that I am speaking of representative consumer behavior for an ODF Text document. There is no way to know what the connection with a dc:creator element is and what constitutes conformance. Furthermore, there is no ODF conformance requirement on this information being available and on how implementations influence it, either mechanically or as an user agent or both. There is, of course, also nothing specified for ODF on who has control over the dc:creator content and what is allowed for that content. 2.3 I am not sure what needs to be done to navigate this thicket. 3. SERIOUS CONCERN 3.1 I looked at section 3.1.7 Creator, and I find this statement to be abominable: "The name of this element was chosen for compatibility with the Dublin Core, but this definition of "creator" used here differs from Dublin Core." However dc:creator is bound to local name creator of the namespace "http://purl.org/dc/elements/1.1/" and it is inappropriate to misappropriate namespace terms from a normatively-used namespace that is not under the control of OASIS or the ODF specification. (The last time I checked into DC Creator, far too long ago, they also provided some principles for specialization of their elements, but this is not that.) 3.2 Furthermore, section 3.1.7 provides a kind of semantic redefinition ("the last person to modify the document" in contrast with "an entity primarily responsible for making the content of the resource") with no guidance on how that is expressed syntactically in the content of the element. So, in addition to namespace abuse, there is nothing authoritative here that qualifies as a conformant entry for this element. (I note that Dublin Core itself has this problem, but appealing to it normatively does not wash our hands of it.) 3.3 The limited description in 3.1.7 also tends to defeat the statement at the beginning of section 3 with regard to "Metadata elements drawn directly from the Dublin Core work use its namespace prefix." (Reference to the prefix is a mistake. It is the use of the namespace that matters, and the choice of prefix for binding the Dublin Core local names is irrelevant to the strength of this statement [and one could create a test case for that, too]). 3.4 I also notice, in the first paragraph of section 3.1, the conformance language statement that the pre-defined metadata elements SHOULD be processed and updated by applications. Also, each of these elements are optional and may also occur multiple times. I wanted to sort this out exactly and I notice the following: 3.4.1 The <office:meta> element is itself optional in the ODF 1.1 schema. The normative schema simply allows anyElements (plural) as content. 3.4.2 There is more SHOULD in section 2.2.1 with regard to the predefined metadata. It is curious that they should be processed and updated, but there is no specific suggestion that they be created initially or introduced when absent. (This SHOULD is also ISO/IEC JTC1 normative terminology, not any IETF one where failure to follow a SHOULD requires justification.) 3.4.3 The pre-defined metadata elements are all introduced as choices for the office-meta-data Relax NG construct. This construct is introduced in section 2.2 as the <zeroOrMore> occurrence of Relax NG construct office-meta-content-strict. construct office-meta-content-strict is not employed at all in the normative schema. The Appendix A strict Relax NG Schema defines construct office-meta-content to consist of construct office-meta-content-strict. This appears to over-rule the normative schema definition of office-meta-content as anyElements. (NOTE: This is worked out quite differently in ODF 1.2 cd01, so the conformance conditions will differ slightly. As of ODF 1.2 cd01, I don't think there is any substantive difference, nevertheless). 3.4.4 It appears to me that (1) there is no conformance condition stronger than SHOULD on any of these predefined metadata elements, and according to the schema the elements MAY be present and MAY be produced, but of course they might simply not be supported by a processor (in which case preserving them seems like a bad idea). [end of notes] -----Original Message----- From: Hanssens Bart [mailto:Bart.Hanssens@fedict.be] http://lists.oasis-open.org/archives/oic/200903/msg00064.html Sent: Tuesday, March 31, 2009 01:22 To: oic@lists.oasis-open.org Subject: [oic] first impression on conformance tests ? Dear members, I'd like to ask if so far there are major remarks on the conformance tests I've added to SVN ? A new zip is available here: http://www.oasis-open.org/committees/document.php?document_id=31869&wg_abbre v=oic I've added a few more tests on user defined metadata, but that'll be where I draw the line for now wrt metadata. For instance, I have no idea how to write tests for document statistics like syllable count. [ ... ]
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]