Problems with the test format

["Dawes, Rogan (ZA - Johannesburg)" <rdawes@deloitte.co.za>]

Hi folks,

Leading up to the meeting today, I wanted to list the issues that I have
found with the actual test format so far:

Connection:

The Connection element is unnecessary. The information we put into the
Connection element could just as easily be put into the URI element. The
possible benefit that we could have got from the Connection element was to
specify that a series of requests had to happen in the same TCP connection.
It should be possible to achieve the same end with tests against response
headers, and the schema would be a bit simpler.

ThreadGroup and Thread:

An issue that was raised previously was that it is not possible to describe
a test which has multiple simultaneous requests. To support this
requirement, I suggest two new elements, ThreadGroup and Thread. A
ThreadGroup will enclose 2 or more Thread elements, followed by a
TestCriteria. Each Thread will contain 1 or more Step elements, describing a
Request, getting variables from the Response, and evaluating TestCriteria
*for that thread*. The TestCriteria at ThreadGroup level would then allow
the writer to test the results of all preceding Thread's once all have
completed.


Other problems with the format as it stands:

Request Authentication

Different Authentication Methods (Basic, NTLM, Digest, None, Client Cert)
Optional and Mandatory (MAY, MUST, MUST NOT) authentication

Custom 404

Supporting fairly arbitrary POST Content?

Allowing the engine to set variables for authentication or other headers,
etc.

Describing desired test result, in terms of positive tests vs negative
tests, and when to report on the test result.

Rogan
-- 
"Using encryption on the Internet is the equivalent of arranging an 
armored car to deliver credit card information from someone living 
in a cardboard box to someone living on a park bench."
  - Gene Spafford
-- 
Deloitte & Touche Security Services Group
Tel: +27(11)806-6216     Fax: +27(11)806-5202     Cell: +27(82)784-9498
-- 

Important Notice: This email is subject to important restrictions, qualifications and disclaimers ("the Disclaimer") that must be accessed and read by clicking here or by copying and pasting the following address into your Internet browser's address bar: http://www.Deloitte.co.za/Disc.htm. The Disclaimer is deemed to form part of the content of this email in terms of Section 11 of the Electronic Communications and Transactions Act, 25 of 2002. If you cannot access the Disclaimer, please obtain a copy thereof from us by sending an email to ClientServiceCentre@Deloitte.co.za.