regrep message
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]
Subject: Re: [regrep] 7/29/2003: Conformance Efforts and ebXML IIC
- From: "Michael Kass" <michael.kass@nist.gov>
- To: "Farrukh Najmi" <farrukh.najmi@sun.com>
- Date: Thu, 28 Aug 2003 13:46:17 -0400
Farrukh and all,
I've >snipped< portions of our
previous conversation regarding ebRS conformance testing,
for brevity. Below are my additional
comments highlighted in BOLD
----- Original Message -----
[MIKE] - I see an ebXML RS conformance test
suite specification document,
>which contains a list of conformance test
requirements, and a conformance test suite
>Also included is supporting test material
like that put together
>by Len Gallagher in the initial ebRS conformance
test suite
[Farrukh] Your detailed responses have clarified things
very much and I feel we
are on the same page.
[Mike] I would
suggest a "staged" approach, in which test assertions are
>compiled for a
selected "profile" or "level"
[Farrukh] +1 on this very pragmatic
advice.
[Farrukh] Who produces automated test suite for verifying
Conformance?
[MIKE] - The IIC is normally tasked with this
effort. Although again, due to our
>own resource limitations,we
would ask for help from the registry TC in contributing tests.
[Farrukh] I think that would be very
reasonable. Is any one in the RegRep TC
available to start the effort by
taking ownership of marking up the RS
spec for potential assertions to
harvest into an initial skeletal CTS ?
>[MIKE] - If you take a
"divide and conquer" approach to RS testing, you may be
>able to make this
easier. For instance, choosing a single binding (e.g. SOAP) to write test
requirements,
>
[Farrukh] +1 on choosing SOAP binding as a
starting point
[MIKE] selecting LifeCycleManager
>functionality as
a higher priority for test requirements than
QueryManager
>functionality... could get the ball rollling
sooner.
>
[Farrukh] I think we cannot do only LCM and not QM.
Maybe we should consider doing
More LCM and some QM.
[MIKE] - I agree. We must test
both.. but initially focusing on LCM might be best.
[MIKE] - I
started "marking up" the ebRS 2.5 specification, pages 1-40, (
sectoins
>#1 - #6) I found that portion of the specification to be
straightforward regarding
>identifying conformance test
requirements.
>
>The LifeCycleManager section (section #7) appears
to be straight-forward and
>easy to identify testing requirements, grouped
into fundamental
>sub-categores of "parameter", "response",
"exception" and "audit trail"
>types of testing
>
[Farrukh] +1
[MIKE] The
QueryManager (section #8) could be a very rigorous exercise ... I
would
>suggest a "generalized" approach to FilterQuery or
SQLQuery
>testing, in which "duplicate" query tree node testing is
eliminated to
>simplify both test requirement writing, as well as actual
test generation.
>
[Farrukh] I think that tests will require
publish using LCM and verifying using QM
that data written can be read back
as expected.
[MIKE] - Agreed
[MIKE]
Section #9, Content Management Services, is an optional service, that can
be
>tested by creating/testing an IIC defined content
>management
service to the registry, then verifying its
conformant
>functionality. This could be done by creation and
publishing of a
>"dummy" validation service, I believe. Because this
is an optional feature,
>test requirements for this service could be
described.. but
>actual implementation of test cases "could" be deferred
if time/resources
>are an issue.
>
[Farrukh] We could only do the default
XML Cataloging Service initially which is
the onloy required part of CMS
chapter.
[MIKE] - This is a good minimalist approach
initially.
[MIKE] Section #10, Event Notification Service, is also an
optional service, that
>is well documented and fairly simple to generate
test requirements.
>Acutal test case scenarios should be easy to
construct. Again, because this
>is an optional feature, it is a
candidate for test requirement
>generation, but actual test cases COULD be
deferred for a later time.
>
[Farrukh] I could see defering this initially altogether.
[MIKE] - Agreed
[MIKE] Section #11, Cooperating
Registries Support is a complex scenario requiring
>federated registries
for conformance testing. The requirements
>are well stated, but
implementation of test cases will be complex and
>time-intensive.
The specification does not eplicitly say whether this is an
>optional
feature, although I assume that it is.
>
>
[Farrukh] Some features are required but most are optionall. I could
see defering
this initially altogether.
[MIKE] - Agreed
[MIKE] Section #12, Security an be a large exercise, particularly
with all of the
>possible XMLDSIG options. But this is a "required"
feature, and can perhaps
>be described with a small initial set of test
assertions and test cases, and
>a limited testing scope.
>
[Farrukh] Testing default ACcess Control POlicy is a minimal
requirement IMO.
[MIKE] - Agreed
[MIKE] So from a deliverables
standpoint, if you are doing the entire ebRS
>specification I would
suggest ( working in parallel if resources are
>available )
:
>
[Farrukh] Indeed. That is why I ask team again to speak up idf
you are bale to help.
[MIKE] 1) "Marking up" the ebRS document to
identify testing requirements - 4 weeks
>
[Farrukh] Maybe we can divide chapters among the peopel
available.
[MIKE] - I will submit a basic guideline document to the RegRep TC
outlining the
fundamental procedures and principles in compiling a Conformance
Test Suite Specification
document. We are working on that in the IIC. This will
assist your TC in getting traction
on initial requirements and test cases.
[MIKE] 6) To create a "full blown" ebRS Conformance Test Suite
12 months
>7) To create a Conformance Test Suite Specification -
(integrating test
>requirements, test cases and supporting documentation)
- 1 month
>
>This is my estimate, based upon IIC work with ebXML
Messaging Services.
>
[Farrukh] Your estimates seem right on to me and based on
first hand experience.
Lets hope that some colleagues are able to
volunteer time to begin this
work soon.
[MIKE] - I will try to re-arrange my schedule so that I can have
Thursday afternoons free for
regrep conference calls, and we can discuss these issues and
measure progress. I will not be
able to make today's call, if there is one.
[Farrukh] On a personal level I can commit to all review
activity. I am reluctant
to take on any primary role I am over committed
between spec work and
implementation.
[MIKE] - Understood.
Thanks again for laying this out so clearly for us to begin this
important work.
--
Farrukh
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]