Re: Testing Harness Proposal

[Eduardo Gutentag <eduardo.gutentag@eng.sun.com>]

Please accept my humblest apologies if this has already been discussed
and it's a moot point (although it couldn't be moot since there can't have
been a vote...), but it was my impression that testing harness is just outside
of the scope of this TC -- all this without in any way impinging on the
value of IQTest.


Lynda VanVleet wrote:
> 
> To XSLT/XPath Conformance Committee members:
> 
> The following is an overview of the capabilities of IQTest and some
> suggestions on how it could be used for the conformance testing.  I have
> included some information about IQTest below and additional information is
> available on our web site http://www.classiq.com/.
> 
> IQTest Capabilities
> 
> IQTest is a configurable testing harness that is designed to support
> multiple technologies.  In order to support a new technology you simply
> write an adapter that provides IQTest the technology specific information.
> This information includes how the test cases should be organized, what
> structure the data should be in, and how to execute the test.  Once these
> functions are written you have access to all of the standard IQTest
> capabilities listed below.
> 
> IQTest Usage Proposal
> 
> What Class IQ will provide to the XSLT/XPath Conformance Committee?
> 
> The IQTest testing harness adapter for XSLT can be configured in several
> ways and we will let the committee decide which would be of benefit if any.
> We will provide any member of the committee IQTest software to use for
> committee work and we will work with committee members to develop the
> adapter to our harness that would be useful.
> 
> Our software runs on Windows NT or Unix (XWindows) and supports most
> relational databases.
> 
> Some of the possible configurations include:
> 1.      An adapter that will browse through the XSLT Specification and allow the
> user to choose a section to test.  The test case(s) associated with that
> section will be sent to the XSLT processor provided by the user.  The
> results returned can be compared to expected results stored in the software.
> 2.      An adapter that will browse a set of test cases and allow the user to
> choose a test case from the suite.  The test case is sent to the XSLT
> processor being tested.  The results are returned and compared to the
> expected results.
> 3.      Both of these
> 4.      Another idea the committee finds more useful
> 
> The areas that would need to be addressed by the adapter include:
> 1.  A way to specify the processor you want to use.
> 2.  A canonical view comparison
> 3.  The details of how to display and organize the tests.
> 
> The following explains some of the highlights of our testing software and
> briefly explains how part of our EAI/B2B solution works to test applications
> that send XML over HTTP.
> 
> Enterprise Component Testing Environment
> 
> IQTest is an enterprise-testing environment designed to eliminate manual or
> inefficient tasks performed by developers and testers.  There are two major
> types of needs IQTest supports:
> Component Development - Support for automated Unit Testing of code.
> Component Integration (EAI & B2B) - Support for the communication protocols
> applications use to communicate with each other.
> 
> IQTest Environment
> 
> In order to support the needs of component integration and development
> efforts, IQTest provides a common testing interface for all the technologies
> it supports. This interface allows you to quickly adapt to testing new
> technologies.
> This fully featured enterprise-testing environment includes the following
> capabilities for all supported technologies:
> Test Harness - The test harness allows you to browse your code or interfaces
> and select the calls you want to test.  From the test Harness you can create
> new tests or open existing test cases.
> Test Case - The test case allows you to enter the input data for a test,
> monitor the test output and set expected results for a given test.  This
> capability is fully automated; all you need to do is supply the data (No
> test code needs to be written!).
> Test Case Repository - All test data and cases you define are stored in an
> ODBC based test case repository.  The tests optionally can also be sent to a
> version control system.
> Data Generation - In order to ensure the testing process is as efficient as
> possible, IQTest allows you to load data from files, generate data values,
> or use data from previous test cases.
> Setup Conditions - Setup Conditions allow you to setup the applications
> environment before a test is executed or validate the effects of a test on
> the environment.   For example, a validation can insure that the proper data
> is added to the database.  This allows IQTest to seamlessly support custom
> frameworks.
> Test Suites - Test suites allow you to organize your tests to be used for
> regular regression testing.  You can also run these suites unattended in
> batch mode on a regular bases to ensure errors are identified as soon as
> possible.
> Security - This feature allows you to organize your tests and control access
> to modifying them.
> Coverage Reporting - IQTest supplies reports to monitor the progress of your
> testing.
> 
> IQTest for XML
> 
> Forte Fusion applications are enabled to process XML documents sent over
> HTTP, either natively or using an adapter.  In IQTest, the user can define
> the XML document and the application's URL.  IQTest messages the document to
> the URL and shows a comparison of the HTTP response with the expected
> response.  IQTest compares the elements and attributes of the XML response
> with the expected values.
> 
> The client PC file structure is displayed in the Project List window.  The
> user selects a XML document file by browsing through the file structure.
> The XML document is parsed/processed using the Expat parser built into Forte
> Fusion.  The element and attribute nodes are displayed in the New Test
> Window.
> 
> The URL_Name of the application is provided by the user.  When IQTest is
> run, the XML document is sent over HTTP to the URL to be processed and the
> result is loaded into the Request_Result field and compared to the expected
> results.
> 
> As you know, I will be in Holland but I will be checking my email so feel
> free to send me any questions.
> 
> Lynda
> 
> ========================================
> Lynda VanVleet           |  "Before one can
> Testing Engineer         |   understand
> Class I.Q.               |   recursion, one
> lvanvleet@classiq.com    |   must understand
> www.classiq.com          |   recursion."
>                           V.Orehck III
>                           (fictitious)

-- 
Eduardo Gutentag               |         e-mail: eduardo@eng.Sun.COM
XML Technology Center          |         Phone:  (650) 786-5498
Sun Microsystems Inc.          |         fax:    (650) 786-5727