Assembly TC Test subcommittee meeting, 18th February 2008

[Mike Edwards <mike_edwards@uk.ibm.com>]

Present:

Mike Edwards
Ashok Malhotra
Anish Karmarkar
Dave Booz

Agenda:
1. Opening

Introductions
Roll call
Scribe confirmation
Agenda bashing

2. Sub-committee structure

Chairman/chairmen
Scribe

Liaison with test sub committees of other TCs

3. Subcommittee Outputs

- Identifying conformance statements in the Assembly specification
- Test assertions
- Testcases and testcase artifacts

4. Creating a work plan

- suggest creation of a single testcase from beginning to end
- use as basis for the creation of a plan
- publish to all TCs for comment 

 5. Test Harness
 - do we need to write one and if so, how and
when

Item 2. Subcommittee structure
 a) Chairman
 - leave this until next week (it's a holiday
today in the USA)
 b) Scribe - Mike Edwards will do it for the
present - any one else is welcome

 c) Liaison with other test subcommittees
 Problem is that the TCs are formally separate
in OASIS
 Mike E: Initial idea is to send email to each
other TC containing the minutes of this subcommittee
 - the general idea will be that Assembly produces
artifacts like composites which can be used as the basis for other groups
to build their tests on
 - hope that there is little formal liaison needed
 - hope that there is little formal liaison needed
 3. Subcommittee Outputs
 - Identifying conformance statements in the
Assembly specification
- Test assertions
- Testcases and testcase artifacts
 Anish - Identifying & labelling has been
placed as an action on the Editors in the main TC
 Anish - Observation from other groups: there
are clear statements in the specification (with RFC 2119 keywords) that
need testing, but there are equivalent simple assertions that also need
testing which don't have the keywords.
 Ashok - yes, the fact that there can only be
one default qualifier of an intent is an example of something that can't
be captured in schema
 b) Test assertions
 Anish - Do we really need them
 Anish - Identify conformance statements
 Anish - Identify conformance statements
 Anish - then create tests which work on those
conformance statements
 Anish - we should focus on writing these tests
 Anish - asking Jacques for his input and experience
would be useful
 c) Testcases and testcase artifacts
anish: here is a link to WS-I TAD: http://members.ws-i.org/dman/Document.phx/Private+Folders/Community+Folder/Working+Groups/Testing+Working+Group/TAD/BP/Basic+Profile+1.2-TAD-BdAD?folderId=Private%2BFolders%2FCommunity%2BFolder%2FWorking%2BGroups%2FTesting%2BWorking%2BGroup%2FTAD%2FBP&cmd=download
anish: unfortunately it is member-only, but
as ibm/orcl employees u can see it
anish: need a un/pwd though
anish: here is an example of test assertion
(hope the cut-and-paste works, there is a table in it):
anish: Test Assertion: BP2201
Entry TypeTest TypeEnabledAdditional Entry TypesPrerequisitesProfile Requirements
Message InputWSDL InputTargetPartial-TargetCollateral
definitionsrequired

true
[Not specified]R4003

Context:
For a candidate Web service definition within a WSDL document with a XML
declaration statement.

Assertion Description:
The XML declaration statement uses UTF-8 or UTF-16 for the encoding.

Failure Message:
XML declaration statement within WSDL document does not use expected encoding
(UTF-8 or UTF-16).

Failure Detail Description:
XML declaration statement.

Comments:
checks WSDL encoding.
anish didn't quite paste it correctly
 SCDL artifacts + related stuff like WSDLs, interface
files, implementation files
anish: here is an edited template of ws-i bp
test assertion:
anish: Test Assertion: XXXX

Table that specifies the entry type, test type, msg input, wsdl input,
prerequisites, target, partial targets

Context:

....

Assertion Description:

.....

Failure Message:

.....

Failure Detail Description:

....

Comments:

.....
 Mike E - I like the idea of executable testcases
 ones that can be run and checked almost immediately
 Anish - there will be a problem in testing without
a C&I (implementation)
 Anish - want tests to be usable by "everybody"
 - with minimal changes
 d) Test harness etc
 Ashok - we should write a piece of software
that reads a SCDL and produces output
 Ashok - checking for valid SCDLs
 Dave - I thought that the target software is
the vendor's implementation
 Anish - Expected that it would be limited to
"creating a package" like a zip - and then the vendor tools must
take over
 Anish - not a lot of stuff that we can do, since
there is no format for generated errors
 Dave -- It's pretty loose really, unless we
get into spec'ing error messages
 Ashok - I think we should specify standard error
messages - even error numbers
 Ashok - These would form part of the tests
 COB 

Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.
Co Chair OASIS SCA Assembly TC.
IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.
Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  
Email:  mike_edwards@uk.ibm.com

---

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU