[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: FW: WS-I: Updated Basic Profile Test Report Summary September 15, 2010
See below a past test report on BP1.2 and BP2.0 interop testing, showing that IBM, Oracle and Microsoft have implemented these profiles. (Please do not forward that email outside this list, except internally to your own company if you are among the above testers.) This could be used as a basis for statements of use from these member companies. Thanks, -jacques From: Jacques Durand From: wsi_wsbasic@mp.ws-i.org [mailto:wsi_wsbasic@mp.ws-i.org] On Behalf Of Monica Martin This report includes the interoperability update from IBM and reflects recent TA changes to validation report. 15 September 2010 Test report summary IBM, Oracle and Microsoft successfully achieved interoperability for Basic Profile 1.2 [WGAD link] and 2.0 [WGAD link]. Web service instances and clients of IBM Websphere, Oracle WebLogic, and Microsoft WCF and Windows Web Services API (WWSAPI) participated during the 2-year test period. Using the Interoperability Test Suite, IBM, MS WCF and WWSAPI, and Oracle achieved successful interoperability results across 184 scenarios, [see tables at end of this email] With the new test methodology (herein called the Test Tools Process) and tools, we also verified conformance of the enabled test assertions for the Basic Profile requirements.
In some instances, a client and instance were unable to support or enable specific scenarios (i.e. some instances don’t support non-anonymous responses). The interoperability suite also includes negative scenarios difficult to create (require emitting invalid WSDL or using a special client). These factors and a small number of documented issues remain.[1] Overall, interoperability has significantly improved and the robustness of the test materials increased exponentially (for example, more than 170 interoperability issues have been processed). To meet exit criteria outlined in the Basic Profile WG charter, testing was conducted in 2 phases: interoperability testing, and monitoring and analysis. The Interoperability Test Suite (interoperability message exchange using defined scenarios) [link] and the Testing Tools Package with updates identified here (monitoring, log capture and test analysis, and reporting) [link] were used with new updates to provide: 1. Evidence of interoperability between web services instances and artifacts. 2. An indication of conformance to the Profile of web service instances and artifacts analyzed. Interoperability summary results for BP 1.2 and 2.0 are also available [link]. Detailed test reports are available on the BP WG site. BP 1.2: [folder] See: BP12-TestReports-20100915 BP 2.0: [folder] See: BP20-TestReports-20100915 Wire-validation summary: [folder] See: BP12-20-WireValidation-Results-20100915 All issues in test reporting – test assertions, test participant interoperability and test report bugs or scenario updates - have been processed and re-verified, and/or documented where appropriate. [link] Enhancements to the BdAD test assertions and supporting test materials (scenarios) were used to improve the fidelity of and the assurance for the test results provided here. The most recent testing uncovered gaps in the test assertions that resulted, for example, in false negatives or left test code unexercised in specific situations. In addition, several negative scenarios In Basic Profile trigger test assertion failures, and in specific cases, actually provided evidence the profile requirements were exercised. The overall test results are representative of the actual status of the web services instances under test. Overall, about 30% of the enabled test assertions were improved.[2] This effort was only possible because of the commitment of all Basic Profile WG members including current members – Asynchrony, Fujitsu, IBM, Oracle and Microsoft. Interoperability Test Results
[1] Results are based on partners that supported and have executed the tests. In BP testing, core features are supported and interoperable between participating partners. Basic Profile allows rather than dictates what web services instances could support (anonymous responses only or wsam:Addressing, for example). [2] BP 1.2: 35 of 108 (32.4%); BP 2.0: 34 of 104 (32.7%) [3] Potential tool issue – See Known Issues. |
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]