OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

regrep message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [regrep] 7/29/2003: Conformance Efforts and ebXML IIC


Mike,

Apologies for delayed response. Just catching up with email.

Michael Kass wrote:

>  Please see my comments below.
>
>Thanks,
>Mike
>
>----- Original Message ----- 
>
>  
>
>[MIKE] - I see an ebXML RS conformance test suite specification document,
>which
>contains a list of conformance test requirements ( like the example table
>we've been
>discussing ),a  conformance test suite ( an XML document, with test cases
>that validate to
>the ebXML Test Suite schema as defined in the IIC Test Framework
>Specification document).
>
>Also included in the conformance test suite specification is supporting test
>material ( XML files
>that that will be used in LifeCycleManager and QueryManager transactions,
>like those put together
>by Len Gallagher in the initial ebRS conformance test suite ).
>
.....

Your detailed responses have clarified things very much and I feel we 
are on the same page.

>I would suggest a "staged" approach, in which test assertions are
>compiled for a selected
>"profile" or "level" ( even though the RS specification does not define
>them ).  Spending a year compiling
>assertions would not necessarily serve the ebXML community, whereas building
>upon a "core" set of
>asssertions over time would.
>
+1 on this very pragmatic advice.

>>-Who produces automated test suite for verifying Conformance?
>>
>>    
>>
>
> The IIC is normally tasked with this effort.  Although again, due to our
>own resource limitations,
>we would ask for help from the registry TC in contributing tests to the
>suite.
>
I think that would be very reasonable. Is any one in the RegRep TC 
available to start the effort by taking ownership of marking up the RS 
spec for potential assertions to harvest into an initial skeletal CTS ?

>[MIKE] - Based upon our own experience in compiling about 200 conformance
>test assertions for ebMS, this
>can be a time-consuming task.  However, if you take a "divide and conquer"
>approach to RS testing, you may be
>able to make this easier.  For instance, choosing a single binding (e.g.
>SOAP) to write test requirements, 
>
+1 on choosing SOAP binding as a starting point

>selecting LifeCycleManager
>functionality as a higher priority for test requirements than QueryManager
>functionality...  could get the ball rollling sooner.
>
I think we cannot do only LCM and not QM. Maybe we should consider doing 
More LCM and some QM.

>I started "marking up" the ebRS 2.5 specification, pages 1-40, ( sectoins
>#1 - #6) just to begin to get
>a feel for what it would take to "highlight" the specification and identify
>conformance test requirements.
>I found that portion of the specification to be straightforward regarding
>identifying conformance test requirements.
>
>The LifeCycleManager section (section #7) appears to be straight-forward and
>easy to identify testing requirements, grouped into fundamental
>sub-categores of "parameter", "response",  "exception" and  "audit trail"
>types of testing
>
+1

>
>The QueryManager (section #8) could be a very rigorous exercise in test
>requirement annotation, specifically since each
>"node" in a FilterQuery tree could be a candidate for testing... I would
>suggest a "generalized" approach to FilterQuery or SQLQuery
>testing, in which "duplicate" query tree node testing is eliminated to
>simplify both test requirement writing, as well as actual test generation.
>
I think that tests will require publish using LCM and verifying using QM 
that data written can be read back as expected.

>
>Section #9, Content Management Services, is an optional service, that can be
>tested by creating/testing an IIC defined content
>management service to the registry, then verifying its conformant
>functionality.  This could be done by creation and publishing of a
>"dummy" validation service, I believe.  Because this is an optional feature,
>test requirements for this service could be described.. but
>actual implementation of test cases "could" be deferred if time/resources
>are an issue.
>
We could only do the default XML Cataloging Service initially which is 
the onloy required part of CMS chapter.

>Section #10, Event Notification Service, is also an optional service, that
>is well documented and fairly simple to generate test requirements.
>Acutal test case scenarios should be easy to construct.  Again, because this
>is an optional feature, it is a candidate for test requirement
>generation, but actual test cases COULD be deferred for a later time.
>
I could see defering this initially altogether.

>
>Section #11, Cooperating Registries Support is a complex scenario requiring
>federated registries for conformance testing.  The requirements
>are well stated, but implementation of test cases will be complex and
>time-intensive.  The specification does not eplicitly say whether this is an
>optional feature, although I assume that it is.
>  
>
SOme features are required but most are optionall. I could see defering 
this initially altogether.

>Section #12,  Security an be a large exercise, particularly with all of the
>possible XMLDSIG options.  But this is a "required" feature, and can perhaps
>be described with a small initial set of test assertions and test cases, and
>a limited testing scope.
>
Testing default ACcess Control POlicy is a minimal requirement IMO.

>
>
>So from a deliverables standpoint, if you are doing the entire ebRS
>specification  I would suggest ( working in parallel if resources are
>available ) :
>
Indeed. That is why I ask team again to speak up idf you are bale to help.

>
>1) "Marking up" the ebRS document to identify testing requirements - 4 weeks
>
Maybe we can divide chapters among the peopel available.

>....
>6) To create a "full blown" ebRS Conformance Test Suite 12 months
>7) To create a Conformance Test Suite Specification - (integrating test
>requirements, test cases and supporting documentation) - 1 month
>
>This is my estimate, based upon IIC work with ebXML Messaging Services.
>
Your estimates seem right on  to me and based on first hand experience.

Lets hope that some colleagues are able to volunteer time to begin this 
work soon.

On a personal level I can commit to all review activity. I am reluctant 
to take on any primary role I am over committed between spec work and 
implementation.

Thanks again for laying this out so clearly for us to begin this 
important work.

-- 
Farrukh




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]