Re: [sca-bindings] Fabric3 declaration of SCA 1.1 Web Services Binding Specification conformance and fulfillment of the TC Exit Criteria requirements

[Simon Holdsworth <simon_holdsworth@uk.ibm.com>]

Anish,

I agree the output is a little hard
to validate.  For many tests there is no obvious indication that the
test was executed successfully.  For others, e.g. 5002, there is clear
evidence that the test was run, because the expected output from the component
is visible in the log as below.

Tests where I see a Tests run: 1 entry
in the log:

2004, 2006, 2008, 2009, 2010, 2012,
2014, 2021, 2023, 
4001, 4002, 4003, 4004, 4005, 4006,
4007, 
5001, 5002, 5003, 5004, 5005, 5006,


These correspond almost exactly to the
set of "positive" tests, the one exception being 5003, but that
is a negative test which is only discovered at runtime.

So it appears that any "negative"
test which did not result in a runtime check will only show "Tests
run: 0" in the log.  There is nothing in the submitted log which
shows the expected failure (and hence success of the test).  In each
case the log does show Fabric3 runtime being started and an attempt to
deploy the contribution.

Excerpt from the submitted log for BWS
5002:
[INFO] Executing tests...

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running ClientTest
>>> Starting external JAXWS
service at: http://localhost:9081/JAXWS/Service5Callback
Service5 client - call address is:
http://localhost:8080/Service1
wsa:To address found: http://localhost:9081/JAXWS/Service5Callback
wsa:Action value found: http://test.sca.oasisopen.org/Service5Callback/callback1Request
Address found: http://www.w3.org/2005/08/addressing/anonymous
Tests run: 1, Failures: 0, Errors:
0, Skipped: 0, Time elapsed: 1.989 sec
<sca-bindings@lists.oasis-open.org> wrote on
20/03/2013 23:23:09:

> From: Anish Karmarkar <Anish.Karmarkar@oracle.com>
> To: sca-bindings@lists.oasis-open.org, 
> Date: 20/03/2013 23:23
> Subject: Re: [sca-bindings] Fabric3 declaration
of SCA 1.1 Web 
> Services Binding Specification conformance and fulfillment of the
TC
> Exit Criteria requirements
> Sent by: <sca-bindings@lists.oasis-open.org>
> 
> Jim,
> 
> I see in the output a whole bunch of:
> "Tests run: 0, Failures: 0, Errors: 0, Skipped: 0"
> 
> It has been a long time since I looked at the tests. What does this
> mean? Why was the test not run?
> 
> Thanks.
> 
> -Anish
> --
> 
> On 03/03/2013 01:37 AM, Jim Marino wrote:
> > All,
> >
> > Attached please find output from the OASIS SCA 1.1 Web Services
Binding
> > Test Suite successfully run against the Fabric3 runtime. This
output
> > demonstrates that Fabric3 conforms to all normative statements
of the
> > SCA 1.1 Web Services Binding Specification, including Section
5, the SCA
> > Web Services callback protocol.
> >
> > The Test Suite adaptation to integrate and verify the Fabric3
runtime
> > can be found here:
> >
> > https://github.com/Metaform/fabric3-conformance-ws
> >
> > Note that the README file contains instructions for building
the runtime
> > and executing the Test Suite, which requires JDK 6 and Maven
3.0.4.
> >
> > I believe these results fulfill the SCA 1.1 Web Services Binding
Exit
> > Criteria requirements as laid out in the TC charter.
> >
> > Regards,
> > Jim
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this mail list, you must leave the OASIS
TC that
> > generates this mail.  Follow this link to all your TCs in
OASIS at:
> > https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS
at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
> 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU