[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: OASIS Call for Participation: Testing and Monitoring Internet Exchanges (TaMIE) Technical Committee
To: OASIS members & interested parties A new OASIS technical committee is being formed. The OASIS Testing and Monitoring Internet Exchanges (TaMIE) Technical Committee has been proposed by the members of OASIS listed below. The proposal, below, meets the requirements of the OASIS TC Process [a]. The TC name, statement of purpose, scope, list of deliverables, audience, and language specified in the proposal will constitute the TC's official charter. Submissions of technology for consideration by the TC, and the beginning of technical discussions, may occur no sooner than the TC's first meeting. This TC will operate under our 2005 IPR Policy [b]. The eligibility requirements for becoming a participant in the TC at the first meeting (see details below) are that: (a) you must be an employee of an OASIS member organization or an individual member of OASIS; (b) the OASIS member must sign the OASIS membership agreement [c]; (c) you must notify the TC chair of your intent to participate at least 15 days prior to the first meeting, which members may do by using the "Join this TC" button on the TC's public page at [d]; and (d) you must attend the first meeting of the TC, at the time and date fixed below. Of course, participants also may join the TC at a later time. OASIS and the TC welcomes all interested parties. Non-OASIS members who wish to participate may contact us about joining OASIS [c]. In addition, the public may access the information resources maintained for each TC: a mail list archive, document repository and public comments facility, which will be linked from the TC's public home page at [d]. Please feel free to forward this announcement to any other appropriate lists. OASIS is an open standards organization; we encourage your feedback. Regards, Mary --------------------------------------------------- Mary P McRae Manager of TC Administration, OASIS email: email@example.com web: www.oasis-open.org [a] http://www.oasis-open.org/committees/process.php [b] http://www.oasis-open.org/who/intellectualproperty.php [c] See http://www.oasis-open.org/join/ [d] http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tamie CALL FOR PARTICIPATION OASIS TESTING AND MONITORING INTERNET EXCHANGES (TaMIE) TC 1a. Name and abbreviation Testing and Monitoring Internet Exchanges (TaMIE) TC 1b. Purpose Electronic collaborations over the Internet between business partners (e-Business / e-Government) appear to be converging toward well-established types of message exchange patterns that involve both user-defined standards and infrastructure standards. At the same time, the notion of event is increasingly promoted for asynchronous communication and coordination in Event-Driven Architectures (EDA) that are considered as either complementary to or part of SOA systems. In both cases collaboration between partners or between components is achieved by the means of choreographed exchanges of discrete units of data - messages or events - over an Internet-based protocol. The TC will define an event-centric test case scripting markup and execution model for such systems. In e-Business transactions as in EDAs, partners or components must agree on the use of a combination of standards in order to interoperate with each other. Typically, these standards can be classified into three layers. . Messaging infrastructure standards, ranging from transport level to higher-level messaging protocols and quality of service (QoS) including reliability and security, such as those defined as SOAP extensions, or REST (Representational State Transfer). . Multi-message exchange standards as manifested in business processes and choreographies. Included standards may conform to UMM business transaction patterns, ebXML Business Process Specification Schema (BPSS or ebBP), WS-Choreography or WS-BPEL. . Business document standards. These may be business content structure and semantics, taxonomies in use, code lists, semantic rules, or the XML schema modeling style. They are industry-specific (e.g. RosettaNet PIP schemas, AIAG Inventory Visibility and Interoperability schemas), horizontal document standards (e.g. based on UN/CEFACT Core Components, OAGIS Business Object Documents schemas), or regional guidelines (e.g., KIEC's e-document modeling guideline). There have been conformance and interoperability test suites (e.g, ebXML messaging V2 test suites, WS-I test assertions) and testing tools (e.g., ebXML IIC, NIST QOD, NIST Business Document Content testbed, KorBIT ebMS testbed, WS-I test tools) for each above layer individually. But the testing of integrations of standards has been ad-hoc (e.g. RosettaNet Self-Test Kit is tied to RosettaNet protocol), or limited mostly to standards in the messaging infrastructure (WS-I). Although the need for testing some form of integration of standards has been well recognized for infrastructure standards (e.g. WS-I profiles), there has been little support for testing integrations that extend to the use of standards specific to a business - e.g. for documents or choreographies. Such integrations can be construed as user-defined profiles. For example, the level of QoS required for a business transaction may depend on the nature of business data being exchanged, or on some property defined by the related business process. Testing and monitoring these infrastructure layers and their integration also requires that test cases access a combination of contracts - agreements, policies or business transaction patterns - represented by meta-level documents (WS-Policy, WSDL, ebXML CPPA, WS-BPEL abstract processes, ebBP definitions, XML schemas). This compliance objective goes beyond quality assurance for the messaging function: it requires the monitoring of live transactions in production environments, as well as verifying conformance of business endpoints in operation conditions. This calls for a flexible test execution model that can accommodate performance constraints as well as different timeliness constraints - e.g. where tests are either deferred over log data, or executed on live exchanges in a monitoring mode. Consequently, the execution model of such test cases or monitoring cases, must accommodate dynamic conditions requiring real-time or near real-time error detection or measurement, allowing to correct and report on business exchanges as they proceed. The output of a monitoring script also must provide more information than a report of the type pass / fail. Different ways of "passing" or "failing" must be reported on, as well as identifying the types of business transactions. The output must be easy to format and feed to another decision engine, such as a rule engine that will process this input in real-time. For example, a rule may decide to generate an alert if a business transaction lasts too long, depending on the nature of the transaction and on the SLA associated to these business partners. This reporting must be easy to consolidate, for Service Level Agreements (SLA) and Business Activity Monitoring (BAM) purposes. The reporting output may itself be subject to dynamic testing and monitoring. For example, a rule may decide to generate an alert if the current average response time for requests of a specific type, is beyond a threshold. The TC will define a testing and monitoring model, as well as a test script markup, so that test cases or monitoring cases can be fully automated, and portable across test environments. 1c. Scope The scope of activity for this TC must be within the following topics: - Use cases and Requirements: Investigation of use cases that may combine standards - or specifications submitted to an SDO - in all areas concerned by message exchanges: protocols, QoS, choreography, documents and business content. Selection and prioritization of related requirements. - Event Management model: A model for representing, searching, correlating and managing events intended for the purpose described in this charter. In particular, an event management approach such as the one described in the event-driven Test Scripting Language (see Input material) will be considered. This event management must allow for the same test case to execute either in static mode, over a set of events all of them are already logged, or in dynamic mode, over a set of events not all of them have already occurred when the execution starts. The event model must be open for reuse by at least some other scripting and processing engines, such as Event-Condition-Action (ECA) rule-based systems. - Test case scripting and execution model: An XML markup for representing executable test cases will be defined. The execution model will rely on the event management model. The test case scripting may reuse or profile subsets of existing dialects that are already in use for domains that have similar functional requirements (if their IP status is compatible with the TC IPR mode), in which case the markup will act as a coordination or integration language, treating these dialects either as language extensions or as external resources used via adapters. In particular, openness to the reuse of XML processing dialects and tools based on recognized standards - such as XPath, XSLT, XQuery - is a requirement. The scripting and execution model will attempt to support extensions for other scripts, in particular should allow for bindings to rule representations such as SBVR (OMG) or RIF (W3C). The design approach for the core scripting will favor (a) simplicity of use - a small number of constructs that are easy to learn and to implement as opposed to feature-rich dialects; and (b) modular units of scripts that allows for functional reuse and extensibility. Test scripts will process run time materials (i.e., messages, events, signals), as well as configurative artifacts such as policies and agreements, schemas and interface definitions. Test cases or monitoring cases will also provide for more detailed outputs than pass/fail, and will strive to support use cases defined in the requirements for SLA monitoring, BAM (Business Activity Monitoring) and Event-Driven Architectures (EDA). However, the objective will not be to specify a BAM system or an SLA enforcement system, but only the monitoring technology that can support these. - Evaluation and Investigation: of third-party markups and dialects, log formats and existing test practices, for possible reuse or interfacing in the test case scripting language. However no third-party specification or tool can be made essential or necessary to an implementation of the specification to be delivered, unless its licensing is free and unrestricted. - Methodology and Guidelines: In scope of this activity, are all forms of support for users, such as example scripts, best practices, primers and guideline documents, concerning all topics listed above. The following documents are input material to this TC, which will deserve prime attention from the TC assuming their IP status is compatible with the IPR mode of the TC: - The event-driven Test Scripting Language (eTSL) V0.85, http://www.oasis-open.org/committees/document.php?document_id=26036 - Conformance Testing and Certification Framework (OASIS, Conformance TC, June 2001), http://www.oasis-open.org/committees/document.php?document_id=309&wg_abbrev=ioc - QA Framework: Specification Guidelines (W3C, November 2004), http://www.w3.org/TR/2005/REC-qaframe-spec-20050817/ - Test Assertion Documents for WS-I profiles (WS-I, 2003-2005). - OASIS IIC Test Framework 1.1, http://www.oasis-open.org/committees/document.php?document_id=10896 Other documents may be considered by the TC, subject to a TC decision. Explicitly Out-of-scope: - Defining or specifying detailed test case metadata or artifacts that would complement the above scripting - e.g. such as supported in ATML, for representing test environments, complete test suites and/or their result. - Definition of particular test harnesses. - Use cases and requirements that refer to infrastructure specifications that are not submitted to an SDO, or that refer to business documents or content that are not approved by an organization or consortium representative of this business domain. 1d. Deliverables The TC will produce the following deliverables: (a) A requirements document, which may include use cases for Internet exchanges, test assertions for related standards, references to existing test case dialects or existing logging formats or systems. (b) A specification defining an event metadata and an event log management model that supports both the testing and monitoring described in this charter, in a way that does not preclude other test case scripting techniques than the one specified in deliverable (c). (c) A specification defining an event-centric test case scripting and execution model that supports both the testing and monitoring described in this charter, and builds upon the deliverable (b). (d) A set of examples and use cases illustrating the use of above specifications for testing or monitoring, that involve business content standards, some choreography standards, and some specifications in the domain of SOAP messaging or others. (e) An implementation of the above specifications (b) and (c) used for proof of the proposed concept and principle. Timeline: The TC will aim at a Public Review draft of both deliverable (b) and (c) above by first half 2009. 1e. IPR Mode Royalty-Free on Limited Terms. 1f. Audience The TC welcomes any OASIS member with an interest and/or experience in testing and monitoring. 1g. Language English Non-normative Information 2a. Identification of similar or applicable work that is being done in other OASIS TCs or by other organizations, why there is a need for another effort in this area and how this proposed TC will be different, and what level of liaison will be pursued with these other organizations In general the following works fall short of addressing TaMIE charter, in that they: . either target a specific protocol stack, . or a particular domain of business. Their support for testing and monitoring the implementation of a combination of standards is either inexistent or limited, and they do not provide a versatile, extensible scripting of test cases. The requirements that are specific to B2B environments are not well addressed either: none is addressing both run-time monitoring of systems in production, and conformance testing prior to deployment. Only the latter is generally supported by existing work, e.g. neglecting real-time monitoring cases that may lead to dynamic notification or remedial actions, or ignoring real-time constraints about events throughput and recovery after shutdown time. These test environments below - at the exception of WS-I tools - do not support the combined processing of electronic documents representing agreements or policies (metadata) and of run-time logs. Finally, none supports a general-enough notion of event, along with adequate time primitives and event correlation and search capability. (a). The OASIS ebXML IIC test framework (TF v1.1) http://www.oasis-open.org/committees/document.php?document_id=10896 This framework was designed specifically to test the conformance and interoperability of the software implementing the ebXML Messaging Specification. The framework defines components and harnesses (configurations) necessary to simulate and control the environment for conformance and interoperability tests. A shortcoming of the IIC TF is that it has been designed for ebXML messaging 2.0, and provides weak extensibility options, both for external events and for specialized evaluations. The IIC TF does not support other suites of B2B integration standards such as other SOAP profiles integrating several Web Services standards. It also does not support ebMS 3.0 and electronic documents testing. (b). RosettaNet Self-Test Kit (RNSTK) RNSTK (http://serg.telecom.lth.se/education/master_theses/docs/72_Kjellin_slutrapport. pdf) is a test system provided by the RosettaNet consortium. The system allows software solutions to perform self-tests for compliance to business collaboration scenarios and the RosettaNet Integration Framework (RNIF) specification. The RNSTK tests an integrated system implementing particular B2B collaboration scenarios, yet is tightly dependent on RNIF messaging. RNSTK test suites are encoded using the RNIF specification itself. Due to these reasons, the RNSTK cannot be used to test other suites of B2B standards including the Document Type Definition semantics . Another weakness is that its architecture only supports conformance test configurations. (c). Web Services Interoperability (WS-I) Test Tools (for Basic Profile 1.1) These tools do not allow for the scripting of a test suite; instead they apply a battery of predefined tests to any Web service material provided as input. The objective is to verify the conformance of this material to one of the WS-I profiles. The tools only passively analyze message traffic and have no ability to control the systems under test. Nevertheless, this capture and analysis architecture is un-intrusive, simple, and supports deferred testing. The analyzer tool of WS-I cannot be used as a general test engine, being tied to profile definitions; but, it can be leveraged as a specialized module by a more general monitoring capability, when the conformance of message material to WS-I profiles is required. (d). TTCN-3 (Testing and Test Control Notation Version 3, ETSI, June 2001) TTCN-3 (http://www.ttcn-3.org/StandardSuite.htm) is a powerful, programmatically complete procedural language with specialized constructs for defining test procedures, test verdicts, matching mechanisms for evaluation, timer handling and distributed test components. Due to its original focus on telecommunication systems, it also has the ability to specify encoding information, to communicate both synchronously and asynchronously, and to perform passive monitoring. These capabilities are all essential for testing in message-based B2B integration. Relative to previous test frameworks, TTCN-3 is more powerful and flexible. However, TTCN-3 notation can be difficult for a business or domain expert to use. TTCN-3 has not been designed to delegate some functions to external modules and it is weak when it comes to validating business transaction events and XML electronic documents, which are essential for B2B integration testing. (e). ATML (Automatic Test Mark-up Language, IEEE, December 2004) ATML (http://grouper.ieee.org/groups/scc20/tii/ATML/Working%20Groups/Management/ATML% 20Overview.doc) has been designed for exchange of diagnostic information, configuration data, and test instrument definitions between conforming software applications. It was not designed to support test scripting and execution. However, ATML representations could be complementary to executable test case scripts. (f). General Software Test Environments and Scripting JXUnit. JXUnit is an XML-based test-scripting system built on top of the JUnit Java classes. It is a data-driven testing system in which input to the system under test and expected output are specified and then the actual output is evaluated. There is no built-in support for B2B testing, in particular the communication and the event-driven tests. However, as a general, test-scripting platform that relies on a common programming language, JXUnit and JUnit could be used as an implementation platform for essentially any test framework including the one proposed here. 2b. First Meeting: January 28, 2008 at 4pm PT/7pm ET. The first meeting will be a 2h conference call. Fujitsu or KIEC will sponsor the call. 2c. Projected Meeting Schedule: A 90mn meeting every two weeks will be scheduled, unless the TC decides on a different frequency. Assuming the first meeting is a conference call, a face-to-face meeting will be set within 2 months of the first meeting once the work has actually started, to maximize the value of face time. 2d. Co-proposers AIAG (Automotive Industry Action Group): Mohammad Abidi, Mabidi @ aiag.org Axway: Dale Moberg, dmoberg @ axway.com Fujitsu: Jacques Durand, jdurand @ us.fujitsu.com KIEC (Korea Institute for Electronic Commerce): Hyunbo Cho, hcho @ postech.ac.kr NIST (National Institute of Standards and Technology): Nenad Ivezic, nivezic @ nist.gov OAGI (Open Applications Group, Inc.): Dave Connelly, Dconnelly @ openapplications.org 2e. Convener Jacques Durand, Fujitsu 2f. Member Section N/A 2g. List of Contributions The following work, already submitted in October 2006 to OASIS ebXML Implementation, Interoperability and Conformance TC, will be contributed to this TC: - eTSL draft V0.85, (originally derived from the Test Framework 1.1 produced OASIS IIC, 2004-2005) 2h. Draft of F.A.Q. Will be provided later. 2i. Proposed Working Title for deliverables Deliverable #1: eLAM - event Log Access and Management Model. A specification defining a event representation and metadata along with the event store management primitives for supporting event-centric testing and monitoring. Deliverable #2: eTSM - event-centric Test Scripting and Model. A specification defining a test case execution model that supports both the testing and monitoring of message and business data exchanges, and the scripting language for defining and executing these test cases. Deliverable #3: eTSM Use Cases Document - A set of examples and use cases illustrating the use of above specification for testing or monitoring business exchanges based on (a) some business content standards, (b) some choreography standards, and (c) some specifications in the domain of SOAP messaging or others. This document may serve to illustrate requirements that have been addressed. Deliverable #4: An implementation of the above deliverables #1 and #2 used for proof of the proposed concept and principle.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]