[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Subject: Re: Test Case Markup & Cataloging, Iron Man edition
Thank you David for posting this. In error I made comments earlier today to an out-of-date document; I will start afresh with this document. At 01/06/11 13:47 -0400, David_Marston@lotus.com wrote: >Notable features, not necessarily changes: >1. I didn't name the element that encloses all the <test-case> elements. Is ><catalog> acceptable to everyone? Fine with me. >2. The <Source> element doesn't carry a filetype suffix, but <input-file> >and <output-file> may do so, unless someone knows a problem that would >occur. From an automation point of view the answer depends on whether we are using the Source element in the synthesis of other names ... if not, then I think it should be a complete file name. >(1) IDENTIFICATION >Each group of tests is marked with a globally-unique "Title", which string >should also be valid as a directory name in all prominent operating >systems. The title can be suggested by the submitter, but must be approved >by OASIS. Thus, Lotus would submit a test suite called "Lotus" and the >OASIS procedures would load it into a "Lotus" directory (assuming that the >name "Lotus" is acceptable to the OASIS committee). The catalog as >submitted need not have the <Title>Lotus</Title> element in every entry >(thus it's shown as optional below), but it will be necessary in the merged >catalog that results from the "assembly" process. A submitted suite can >have arbitrary directory structure under its top-level directory, captured >in the "Identifier" element for each case, with forward slashes as the >directory delimiters. The actual name of the particular file (and test >case) would be in the "Source" element, which should be a valid file name >in all prominent operating systems. The Identifier contains the Source >string at the end, but not the Title at the beginning. Note that the test >suite may contain directories that have no test cases, only utility or >subsidiary files. Is there a need for multiple titles in a single submission? I don't think there is so I would make this either an attribute of the document element or an unspecified value (to the submitter) that would be incorporated in the final suite by the committee stylesheets. This proposes the Identifier value to be mandatory for each test case ... could it be a separate element used as a branch in a multi-level tree where the identifier specifies (without punctuation) exactly one level of the tree ... the committee stylesheets could then assemble file paths from the hierarchical Identifier values and arbitrary punctuation. <!ELEMENT Catalogue ( Identifier+ )> <!ELEMENT Identifier ( Identifier*, test-case* )> <!ATTLIST Identifier directory CDATA #REQUIRED> Note that if we decide to rearrange files according to categories we can encode in the final output a derived attribute value from the hierarchical components. ><!ELEMENT Source ( #PCDATA ) > ><!-- Identifier uses forward slashes as separators, begins with the name of >a > directory that is directly within the top directory named per Title, and > ends with the name-part in Source. --> ><!ELEMENT Identifier ( #PCDATA ) > My personal convention regarding the attribute/element religion is that #PCDATA is *reserved* for human-readable text and attributes are *reserved* for machine-readable text, and I coerce my models to meet these very strict constraints. The justification is that #PCDATA exists to be available for mixed content which is almost (or completely?) exclusive to human-oriented content. >OASIS may bless a particular hierarchical organization of test cases. If we >do, then an attribute called "category" should be used to track where the >test fits in OASIS' scheme of categories. That way, OASIS categories will >not dictate the directory structure nor the case names. The goal is that no >case should be marked as belonging to more than one category. We may need a >category named "Mixed" if we don't have a clean partitioning. The DTD >excerpt below shows a slight adaptation of Carmelo's proposed categories. > ><!ATTLIST test-case > category ( XSLT-Structure | XSLT-Data-Model | XSLT-Template > | XSLT-Result-Tree | XSLT-Data-Manipulation | XSLT-Extendability > | XSLT-Output | XPath-Location-Path | XPath-Expression > | XPath-Core-Function | XPath-Data-Model | Mixed ) #IMPLIED > Thinking about future utility of the stylesheets we write for manipulating the catalogues, for this and other XSLT/XPath-specific semantics of testing I propose we genericize the value to simple CDATA and have a supplemental expression of what attribute values or value pairs (or other) are allowed in the XSLT/XPath use of the test suite. That way importation (and perhaps a supplemental validator we supply) could check all categories, discretionary items, etc. against an XML instance encoding available semantics and their associated values. It makes the document model easier, but does engage a validation step. Where any aspect of this model is hardwired to semantics of XSLT/XPath testing, we would have a "test suite validation criteria" instance used for validating a given submission against the anticipated values for a particular class of test semantics. >Submitters should be encouraged to use the "Creator" element(s) to name >contributors at the individual-person level. They may also wish to use an >element called "Date" to record, as yyyy-mm-dd, Yet another aspect that can be validated by our semantic validation stylesheet. >the date stamp on the test >case. That will allow the submitter to match cases with their own source >code management systems, and will likely aid in future updates, either due >to submitter enhancements or W3C changes. OASIS reserves the right to >insert this element, containing the date received, if no value was supplied >by the submitter. > ><!-- Dublin Core Creator instead of Author --> ><!ELEMENT Creator ( #PCDATA ) > ><!-- DC/ISO-8601 Date for the date of submission (from creator's POV) --> ><!ELEMENT Date ( #PCDATA ) > Another candidate for an empty element with a required attribute value. ><!-- There must always be an XSLT spec-citation element, and optionally >other spec-citation elements can be added as appropriate --> ><!ELEMENT spec-citation ( place , Version , version-drop? , errata-add? , >errata-drop? ) > I would prefer using attributes above instead of separate elements. ><!ATTLIST spec-citation > Rec ( XSLT | XPath | XML-Stylesheet | XBase ) #REQUIRED > ><!-- Anticipate XML, XLink, Schema, Namespace, Internationalization, XSLFO, >XInclude in the future --> ><!ELEMENT place ( #PCDATA ) ><!-- syntax of content depends on Type --> ><!-- Type is a Dublin Core keyword --> ><!ATTLIST place Type ( section | doc-frag | OASISptr1 ) #REQUIRED > The above are yet more configuration facets of a generic test suite cataloguing application. >We have cataloged discretionary choices available to the processor >developer, and these choices have names. Genericizing this to: <!ELEMENT discretion EMPTY> <!ATTLIST discretion choice CDATA #REQUIRED value CDATA #REQUIRED> and then incorporating all the possible values in the configuration/validation instance would allow for future expansion and use elsewhere without any changes to the document model, nor the validation stylesheet (only the configuration file). Actually, this aspect of the configuration file could be generated from the authored discretion document. >(4) OPERATIONAL PARAMETERS >... ><!ELEMENT scenario ( input-file* , output-file* , param-set? , console ) > ><!ATTLIST scenario > operation ( standard | embedded | external-param ) #REQUIRED > compare ( XML | HTML | Text | message | message-XML | message-HTML | >message-Text | > manual ) #REQUIRED > ><!ELEMENT input-file ( #PCDATA ) > ><!ELEMENT output-file ( #PCDATA ) > Perhaps attributes of empty elements again. >overkill for this usage. The most likely formats are: >(1) (type) name=value [new-line delimits?] >(2) a simple XML element with name and type attributes > ><!-- This needs further design. Assume it designates an input file. --> ><!-- This value is only relevant when operation="external-param" --> ><!ELEMENT param-set ANY > Perhaps <!ELEMENT param EMPTY> <!ATTLIST param name CDATA #REQUIRED value CDATA #REQUIRED> >We also want to be able to test that a message was issued (as in >xsl:message) and that an error was issued. The "console" element will be >used to designate strings that must be present in either the standard >output or standard error stream. Verbatim strings or prose the operator interprets against the string ... I'm guessing the latter since there are no prescriptions for message strings, so being a prose element I agree it can be PCDATA or even HTML as in the description. >Note that the test suite itself is not filtered by scenario values. The >test lab may wish to devise a harness that can be configured to exclude >certain scenarios from some runs, but I think we want to encourage testing >and reporting against the full range of scenarios. > >When a test case is included, it is run according to the values in the ><scenario> element. If inputs are specified, they are marshalled as >necessary. If no inputs are specified, a single file named <Source>.xml is >assumed to be the input and <Source>.xsl is assumed to be the stylesheet. Oh, okay. Now I see the filenames are synthesized. I hope these comments are helpful. ..................... Ken -- G. Ken Holman mailto:gkholman@CraneSoftwrights.com Crane Softwrights Ltd. http://www.CraneSoftwrights.com/s/ Box 266, Kars, Ontario CANADA K0A-2E0 +1(613)489-0999 (Fax:-0995) Web site: XSL/XML/DSSSL/SGML/OmniMark services, training, products. Book: Practical Transformation Using XSLT and XPath ISBN 1-894049-06-3 Article: What is XSLT? http://www.xml.com/pub/2000/08/holman/index.html Next public instructor-led training: 2001-06-18,06-21,08-12,08-13,09-19 Training Blitz: 3-days XSLT/XPath, 2-days XSLFO in Ottawa 2001-06-18/22
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Powered by eList eXpress LLC