minutes, and next call Monday July 28

[Jacques Durand <JDurand@fsw.fujitsu.com>]

Title: minutes, and next call Monday July 28

All:

1. Attached , the minutes of two past meetings...

2. Next meeting:

Time: Monday July 28, 11am PT
Host: Fujitsu
Toll Free - : 1-800-251-6413
Toll - : 1-913-905-1400
Participant code: 598136

Agenda:

1. Status of MS conformance test suite spec. (Mike Kass editor)
- review current draft of MS conf test suite, and specific issues:
. Issue #1: the problem of "app-level" test cases: do we have a list?.
. Issue #2: ordering / partitioning based on config/CPA? (how to avoid too many reconfigs)
. Issue #3: conformance profiles or levels (detailed content?)(please read last minutes, we'll decide)
. Issue #4: various editorial issues (see previous mails).
. Issue #5: message correlation details.
. Issue #6: check / fix parameter tables.

2. BPSS test update (Serm K., Monica M.):
- spreadheet from Serm / Monica, BSI testing definition.
- what cooperation/feedback with/to the BPSS team
(emphasis on testing operations).
- idea of prototype.

3. Implementers corner, and PR:
- Drake Certivo test driver status.
- KwareSoft question on ErrorURLNotify
- presence at AutoTech: who demoes, IIC visibility...
- IIC members demo (Test Framework, test suites) at XML 2003, 7-12 December

<<IIC_July_14_03_Minutes.txt>> <<IIC_July_21_03_Minutes.txt>>

IIC Conference Call Minutes: Monday, July 14, 2003
 
Attendance:

Aaron Gomez (DGI)
Mike Dillon (DGI)
Mike Kass (NIST)
Jacques Durand (Fujitsu)
Monica Martin (Sun)
Pete Wenzel (SeeBeyond)
Steve Yung (Sun)
Serm Kulvatunyou (NIST)

Minutes taker: Jacques Durand

Time: Monday July 14, 11am PT
Host: Fujitsu 
Toll Free - : 1-800-251-6413 
Toll - : 1-913-905-1400 
Participant code: 598136 


Agenda: 

1. MS Interop testing: practice and future reqs 
- Follow-up on global interop challenges (Drummond / Dillon) 
- how can MS conf testing and interop testing be integrated in practice? 

2. Status of MS conformance test suite spec. (Mike Kass / Jacques D.) 
- test req review (M. Dillon) 
- review current draft of MS conf test suite, and specific issues: 
- spec coverage doc (annotated spec?) 
- configuration data (test driver / MSH) 

3. BPSS test update (Serm K., Monica M.): 
- BPSS req document, scope of testing? (BSI def, bus signals) (Serm) 

4. Implementers corner, and PR: 
- Drake Certivo test driver RF implementation, IPR, interface questions. 
- KWARE needs. 
- presence at AutoTech: who demoes, IIC visibility... 
- Demoing Test Framework at XML 2003, Philly from 7-12 December
 
-----

1. MS Interop testing: practice and future reqs 

- Mike Dillon: matrix testing can be done without test bed as router.
- Jacques: test center can be used for facilitating logistics (downloads of
tset suites / harness components), as well as trouble shooting.
Also for conformance testing, as hosting the test driver.
- test center as "centralized intermediary".

2. Status of MS conformance test suite spec. (Mike Kass / Jacques D.) 

- [AI] need clean-up current posting, redundant documents (specs annotated, test reqs doc).
- issue of testability: some test reqs are app level (Mike).
- Jacques: that has been described in Test Framework as "contingent" coverage 
(out of the control of test harness)
- Monica: we should identify patterns of test cases (similar choreography and logic).
- Mike Dillon: MSH implementations are not always built to be easily testable...
e.g. lots of time was spent on configurations in past test rounds.
Test suite should be designed so that config overhead is reduced.
- Mike Kass: multi-hop tests need review: how to configure them?
 
 
3. BPSS test update.

- Serm: target is 1.05. Challenges: state management at bus transaction level, 
timeout, exceptions.
- do we address collaboration semantics. 
- Interop requirements are at two levels.
- Sermwill communicate summary of issues (BPSS spreadsheet) to Monica.


4. Other: 

- Monica: eBES does more than ebMS testing: they have plans for RR, and CPPA 2.0.


IIC Conference Call Minutes: Monday, July 21, 2003
 
Attendance:

Mike Kass (NIST)
Jacques Durand (Fujitsu)
Monica Martin (Sun)
Pete Wenzel (SeeBeyond)
Styeve Yung (Sun)
Serm Kulvatunyou (NIST)
Tim Sakach (Drake Certivo)
Angela Warburton (Drake Certivo)

Minutes taker: Jacques Durand

Time: Monday July 21, 11am PT
Host: Fujitsu 
Toll Free - : 1-800-251-6413 
Toll - : 1-913-905-1400 
Participant code: 598136 

Agenda: 

1. Status of MS conformance test suite spec. (Mike Kass editor) 
- test req additional review / feedback if any (M. Dillon) 
- review current draft of MS conf test suite, and specific issues:
. the problem of "non black-box" test cases, and test cases that would
require a control at app level, that the test service does not provide.
. "patterns" of test cases, 
. ordering / partitioning based on config/CPA (how to avoid too many reconfigs)
. conformance profiles (and do we support multi-hop?).
. intro and references to other related/similar test initiatives.
. use of parameters? (for test driver / mesg elts / MSH config)

2. MS Interop testing: practice and future reqs 
- Follow-up on global interop challenges (Drummond / Dillon) 
- logistics: the role of test centers. 

3. BPSS test update (Serm K., Monica M.): 
- what cooperation/feedback with/to the BPSS spec editing team 
(emphasis on testing operations, not modeling).
- idea of prototype.
- BPSS test req document, scope of testing, levels of interop (Serm) 

4. Implementers corner, and PR: 
- Drake Certivo test driver status. 
- KwareSoft question on ErrorURLNotify 
- presence at AutoTech: who demoes, IIC visibility... 
- Demoing Test Framework (or more) at XML 2003, Philly from 7-12 December

 
 
-----

1. Status of MS conformance test suite spec. (Mike Kass editor) 

General test reqs / test cases:
- Mike K. got detailed feedback from Mike DIllon: overall, very positive.
- Consensus that "other activities" intro section should be diverted. 
We should not have to mention orgs or companies that are not directly related
to the conformance effort at this point.
- [AI] Monica will give a second try to a section that might describe how the use
of this conf suite could fit a test center environment. In any case, that looks like
appendix material, non-normative.
- Mike: we still have some test requirements that seem more relvant to application
than to MSH: even if they still are "black-box", they cannot be properly implemented 
without control of the application, in addition of test driver. 
Maybe enhanced Test Service actions (future version) would help for these?
e.g. req #50 "manifest ref will always use a URI when referencing payload". 
It seems these values are not always under control of the MSH: unclear.
- [AI] Mike to provide a list of such "app-dependent" test reqs.
- Pete: coverage still lacking, need be fixed (new req table)

Conformance profiles:
- Need to put this section at a higher level in doc (header 1 or 2), need be put at the end,
not in intro, associated with "profile docs" as specified in Test Framework.
- 3 levels: we agree that ping / pong / status requests, should not be in level 2,
as they are quite optional, and not aggressively implemented by developers
(though might be very useful). So they should go in Level 3.
- (Jacques) syncReply mode , in level 1, could be required only for "mshOnlySignals",
as there is a large difference between this option and all others, implementation-wise.
(much easier) This split core features / additional features has been required last year, 
and did not meet opposition.

Remaining issues:
- contributor list to expand (+Pete W., Mike Dillon...).
- multi-hop abstract tests still use "concrete" syntax (PutMessage, GetMessage...).

Ordering of Test cases:
- Following Mike Dillon mention that we should avoid / reduce re-configurations,
we will see that test cases are grouped by CPAids - at least, test suite execution should
not require shifting back-and-forth from one CPA to the other.
- Mike: 70% test cases use basic CPA, so the current grouping should not be much
affected. Security tests, RM tests, are also grouped. 
- [AI] Check that the CPA scopes are not fragmented. Tim Sakach says that is very
important: will check.


2. MS Interop testing: practice and future reqs 

- skipped this item, as not the right attendees for this topic.

 
3. BPSS test update.

- Monica: will produce with Serm a spreadsheet about what is testable in BSI.
- Serm: focus is on conformance for now: verify interaction between req/resp bus activities,
BSI-level ack generation (receipts).
- need to separate messaging level from BSI app level.
- [AI] Serm will send a doc on this, for review.
- Monica: will ask BPSS team members who have implemented BSI, for their feedback.
Our testing for now will focus on the operational aspect, not the modeling aspect,
as this facet of conformance is the most critical for interoperability.


4. Implementers corner, and PR: 

- KwareSoft is questioning the use (implementation)of ErrorURLNotify. 
This is out -of scope for this agenda item, but we'll need to answer (Jacques).
- Tim: DCertivo is currently implementing MS conf test suite,
would consider demonstrating Test Framework and test suites at XML 2003, 
Philly from 7-12 December. (as mentioned, each TC has been offered to demo
their specs.)