RE: [oic] Re: submitting test documents, simplified

["Dennis E. Hamilton" <dennis.hamilton@acm.org>]

Mingfei,

Thank you for the good questions.  Here is what I have been exploring.

We may need parallel structures, not necessarily all under a single tree
organization.  We can provide linking among them.  Fortunately, the SVN
server is also a web server, so linking will work and HTML might work too.
I am going to conduct some experiments to see how we can take advantage of
that.  We can also use the Wiki as a way to provide organized access to
material on the SVN.  This frees us from working under a single giant
hierarchy.  I agree that the SVN is a better place for test materials and
documents than the TC Document repository.

I think the specification analysis should be decoupled to some degree,
especially from the collections of test documents.  I am not clear how that
is done well at this point.  Although I have been providing for discussion
and analysis pages alongside the replicated content from ODF 1.1 sections, I
was thinking that the analysis should link to further details elsewhere
rather than be expanded under the specification analysis material.  For me,
the analysis would be where we record our understanding of the material,
what we see as testable conditions, and where we might need to determine and
rely on what the actual practice is.  But tests, survey information,
reports, and sample documents would be elsewhere in places that can be
linked to.  I do not have a clear picture of this for myself.  I am unclear
where the lines should be drawn.  I think we need some small trials to see
what works.  Also, we may need to distinguish between informal material and
material that must be approved as OIC outputs.

I have a milestone coming up around packaging.  When that milestone is
reached, there will be something more tangible to discuss.

I am also learning much more on how SVN and the wiki can be used together.
 
 - Dennis

PS: I agree that it may be easiest to accept plaintext forms for
submissions, transforming them to XML and HTML using a tool.

-----Original Message-----
From: Ming Fei Jia [mailto:jiamingf@cn.ibm.com] 
http://lists.oasis-open.org/archives/oic/200903/msg00024.html
Sent: Thursday, March 12, 2009 09:08
To: Hanssens Bart
Cc: oic@lists.oasis-open.org; robert_weir@us.ibm.com
Subject: [oic] Re: submitting test documents, simplified

	Hi Bart,


The ideal way of submitting test cases may be through filling a web form,
but it seems the facility is unavailable as you mentioned before. The
current proposed method also sounds a good alternative. Firstly, just want
to confirm the presenting of test metadata will be a html document that is
generated from the XML, which in turn is generated from the template in the
contributor's mail, right? Second, a question: for a test case, we hope its
test metadata(html doc) and test document are put together on the document
repository.OASIS document repository seems not be able to do that. How about
put them together on SVN? For example, the SVN structure is a tree structure
which is mapping to ODF specification sections. The leaf node in SVN is a
folder that is corresponding to the lowest level section in ODF
specification. And under this folder, we put both the test metadata html
file and the test document there. Also I saw Dennis is building a
SpecAnalysis tree in SVN, maybe it is better if the test suite tree can have
a direct relationship with the SpecAnalysis tree. But I do not know how to
do that right now? Add some link somewhere? Or combine the 2 things
together? etc...

"Hanssens Bart" <Bart.Hanssens@fedict.be> wrote on 03/11/2009 11:53:02 PM:
http://lists.oasis-open.org/archives/oic/200903/msg00021.html
> From:
> 
> Subject:
> 
> submitting test documents, simplified
> 
> Rob / Mingfei,
> 
> 
> Thinking through the whole metadata proposal/testscenario once again, it
> seems that creating the "doctest" XML by hand isn't the most user
> friendly way.
> 
> So the suggestion would be to use a simple mail template (like the ODF
> TC uses for gathering the requirements for ODF Next), and create an XML
> out of it, like this:
> 
> 
> 
> * Step 1) Member can non-members can create small or large test
> documents (either using their favorite implementation or hand crafted).
> The most important thing is to make sure that it is clearly specified
> what exactly is tested, and how to do it with the submitted test
> document.
> 
> 
> * Step 2) In addition, they must fill out the following template (can be
> put on the Wiki and/or xml.org and/or OIC TC document repository)
> 
> 
> + CONTACT (mandatory)
> Your name (Your@email.com)
> 
> + APPLICATION (pick one or more, remove what doesn't apply)
> Text / spreadsheet / chart / presentation / drawing / form / template
> 
> + CATEGORY (pick one or more: remove what doesn't apply)
> Data / visualization / behavior
> 
> + DESCRIPTION (mandatory)
> A short, general description of the test scenario
> Multiple lines are allowed
> 
> + STEPS (mandatory)
> - This is the first step, like "open the test document"
> The step can span multiple lines, just start each new step with a "-"
> - The second step, like "go to page 3"
> You can add as many steps you want
> 
> + EXPECTATION (mandatory)
> - What is the expected result ? Like "page 3 must have a blue
> background"
> - Better even: results, there can be more than one 
> And they can span multiple lines of course
> 
> + SPECS (optional, for specialists)
> - ODF 1.1 / 1.1 / Section title goes here
> Optionally, quote the specific paragraph(s) that contains the aspect to
> be tested. Once again, this can span multiple lines  
> - ODF 1.1 / a.bc / Second section title
> You may test multiple parts of the specification
> - ODF 1.2 / x.yz /
> Probably the test can be used for different ODF versions
> 
> 
> 
> (Maybe EXPECTATION and STEPS can be mixed together, reducing the
> template even more)
> 
> 
> 
> * Step 3) Send the filled out template with the test document(s) in
> attachment to the OIC comment list. The subject must start with
> something like "[TestDocument]". Then the mail archive can be processed
> in a (semi-)automatic way.
> 
> 
> 
> I do have some thoughts on the next steps and how to process it as well,
> but perhaps it's easier to first discuss the first three steps listed
> above, and see if that's the way to go (or not)
> 
> 
> Best regards,
> 
> Bart