OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

ebxml-iic-msg message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Subject: RE: [ebxml-iic-msg] A (rough) draft of the Test Framework


Title: RE: [ebxml-iic-msg] A (rough) draft of the Test Framework

Matt:


>I'm just a bit concerned about the AT-Driver concept, because I think
>that it is all but handled via the "initiate" action of the
>TestService.  All that is needed for an "AT-Driver" is a simple tool to
>kick off a test with a remote MSH, do you agree?

I think it is just a matter of deciding where we support the "driver" functions
like: test case interpretation and control, test step sequencing/execution/correlation,
condition checking, reporting.

These functions are the same for interop and conformance tests. So maybe we could
consider them as part of a unique Test Driver component, used in both tests.
Pushing the modularity further than what I did with the "Test Case Interpreter"
may address your concern on AT. So we could have:

1. in conformance testing:

- Test Driver, combined with a "wire" adapter for:
(1) sending test messages directly on wire (e.g. HTTP),
(2) receiving on wire. The adapter would notify the driver with the
received messages envelope, abstracting the transport.
The Driver would handle test case interpretation, verification logic, reporting.

- Test Service, mostly reacting, and if needed "reporting" to the Test Driver through
dedicated messages (via MSH) (so all decision/reporting still done in Driver).

2. in interoperability testing:

On MSH #1 ("master"):

- same Test Driver, with a "MSH" adapter for sending test messages via MSH API.
(Or, as you suggested, could instead invoke the "Initiator" action of the
local Test Service, through a method call, to get the kick-off done).
For its inputs, the Driver would directly get notifications through its API,
from ALL actions of the local Test Service (not just from Initiator action).
Indeed, we assume that in Interop tests, we don't want to snoop directly on the wire,
but get and do everything via MSHs (am I right, Prakash/Steve ?)

- same Test Service, mostly reacting to received test messages, and "reporting"
to the Test Driver through its API (so all decision/reporting still done in Driver).

On MSH #2 ("slave" or "reflector"):

- Test Service, mostly reacting to received test messages, and if needed "reporting"
to the Test Driver through dedicated messages (so all decision/reporting still done in Driver).

                        ----------------

So with this design, the kick-off job is always controlled by the Test Driver,
which only requires a different adapter depending from where we want to kcik-off
(from wire, in Conformance tests, or from application/service layer in Interop tests).
We don't really need to distinguish anymore "AT" driver and "WT" driver.
Would that be OK?

Jacques

On Thursday, May 30, 2002, at 10:27  AM, Jacques Durand wrote:

> Matt:
>  
> - the "Wire" Test Driver is your Test Driver ,just a renaming
> of Message test Driver (because it craftsand capture
> messages at wire / transport level, as opposed to through the MSH
> application interface)
> - the App test Driver is doing same thing,  except it controls the MSH
> just for sending messages, and
> through its Message Service interface (app interface) - could say its a
> Test Driver with an adapter to MSH app interface.  
> only useful to drive interoperability tests. The idea here is that for
> many (if not all) interop testings,
> we control test cases from the app layer - we don't mess at wire
> level - unless some tests require to.
> (opinion of the Interop team here?)
>  
> I'm open to other suggestions.
> Trying to identify test components needed to support both conformance
> and interoperability.
> Don't get the "creeps" on this creeping AT driver :)
>  
> regards,
>  
> jacques
>  
>
> -----Original Message-----
> From: Matthew MacKenzie [mailto:matt@xmlglobal.com]
> Sent: Wednesday, May 29, 2002 6:21 AM
> To: Jacques Durand
> Cc: ebxml-iic-msg@lists.oasis-open.org
> Subject: Re: [ebxml-iic-msg] A (rough) draft of the Test Framework
>
> Jacques,
>
>
> This looks quite good, but i am a bit disturbed at how the AT-Driver
> thing creeped back in, and what is this "WT-Driver". My understanding
> was that we were going to work with the following two concepts:
>
>
> - TestDriver
>
> - TestService
>
>
> -Matt
>
> On Tuesday, May 28, 2002, at 02:22 PM, Jacques Durand wrote:
>
>
> All:
>
>
>
> here is an initial draft of the ebXML Test Framework, for review.
>
> Clearly, this is just asnapshot of a work still in progress.
>
> (still very incomplete, needs editing in many place).
>
> But this is to get early feedback, for finding out major problems or
> issues as soon as possible.
>
>
>
> I can serve as ultimate editor for this document, unless someone wants
> to.
>
> But I'd prefer CTTF and ITTF folks to focus on their MS Test Suite docs
> (separate from this one)...
>
> (from which this doc will borrow anyway, especially in the "Test Case
> Representation" section)
>
>
>
> - As I mentioned before, this document is ONLY intended - a priori -
> for describing
>
> a test Framework (architecture, components, functions, interfaces,
> mark-up for test cases / test suites, schemas).
>
> It does NOT describe the actual test Suites for MS Conformance and
> Interoperability: these will be
>
> defined in a separate document(s), though they will assume to run and
> be expressed with the Test Framework material.
>
>
>
> - Somehow the draft is still (mostly) focused on MS testing, but the
> ambition of the Framework, as it evolves, is
>
> to be a basis for testing other ebXML specs.
>
>
>
> - Note the renaming / modification of Service/Actions, based on some
> recent discussions. To be improved...
>
>
>
> - Please look at the example sequence of Test Steps given in section
> 3.5 (Executing Test Cases). It is critical to
>
> agree soon enough about what test steps are, can do, what is the
> general protocol we may follow
>
> from one test case to the other, etc.
>
>
>
> Note: The minutes of the last week meeting(s) are still in the work: a
> bit of time shortage right now...
>
>
>
> Regards,
>
>
>
> jacques
>
>
>
>
>
>
>
>
>
>
--
Matthew MacKenzie
XML Global R&D
PGP Key available upon request.



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]


Powered by eList eXpress LLC