was message

Subject: Re: [was] How is Test (Detect) progress ?

From: Rogan Dawes <rogan@dawes.za.net>
To: David Raphael <draphael@citadel.com>
Date: Mon, 05 Apr 2004 11:49:34 +0200

Hi,

David, please could you give my account "rogan" write access to the WAS 
module?

Attached please find a sketch of an updated Test document. It doesn't 
test for anything specific, but illustrates how one can write fairly 
complex tests (including multiple threads, and "delegated" 404 detection).

I think we may still want to add various attributes at various points to 
allow us to control things like connection reuse, and allowing or 
prohibiting the executing engine to add credentials to the request for 
password protected sites, but I hope that the basic structure is fairly 
sound.

Opinions and feedback always appreciated.

Regards,

Rogan

David Raphael wrote:

> Ok,
> 
> 
> Here is the CVS info:
> 
> 
> cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/owasp login 
>  
> cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/owasp co WAS
> 
> 
> 
> The module name is WAS.
> 
> Let me know if you need write access to CVS.  You will need to set up a
> sourceforge account and send me the info.
> 
> 
> Regards,
> David Raphael
> 
> 
> 
> 
> 
> -----Original Message-----
> From: Mark Curphey [mailto:mark.curphey@foundstone.com] 
> Sent: Wednesday, March 31, 2004 6:41 PM
> To: David Raphael; yuvalben@netvision.net.il
> Cc: was@lists.oasis-open.org
> Subject: RE: [was] How is Test (Detect) progress ?
> 
> Dave
> 
> Lets setup a module in the OWASP Sourceforge CVS ?
> 
> Anyone who wants to add changes let us know and well add you to the CVS
> ACL. 
> 
> 
> -----Original Message-----
> From: David Raphael [mailto:draphael@citadel.com] 
> Sent: Wednesday, March 31, 2004 4:40 PM
> To: Mark Curphey; yuvalben@netvision.net.il
> Cc: was@lists.oasis-open.org
> Subject: RE: [was] How is Test (Detect) progress ?
> 
> I would be happy to merge changes, but this could get out of hand rather
> quickly.  Is there any way we can just use CVS?
> 
> 
> /d
> 
> 
> -----Original Message-----
> From: Mark Curphey [mailto:mark.curphey@foundstone.com]
> Sent: Tuesday, March 30, 2004 2:57 PM
> To: yuvalben@netvision.net.il
> Cc: was@lists.oasis-open.org
> Subject: RE: [was] How is Test (Detect) progress ?
> 
> I believe Dave Raphael and Peter Michalek were updating the current
> schema. I know we also need to change the elements thesaurusEntry to
> match the vulnType etc.
> 
> Dave / Peter ?
> -----Original Message-----
> From: Yuval Ben-Itzhak [mailto:yuvalben@netvision.net.il]
> Sent: Tuesday, March 30, 2004 3:49 PM
> To: Mark Curphey
> Subject: RE: [was] How is Test (Detect) progress ?
> 
> 
> Who is in charge of was-core0.3.xml ?
> In addition to the issue below I found few other Schema issues, shell I
> submit an corrected version or should the person in charge ?
> 
> Example:
> <xs:attribute name="versionRevisionDateHistory" type="xs:dateTime"
> minOccurs="1"/>
> 
> The tag 'attribute'  does not support an attribute of 'minOccurs'. It
> should be remove, associated with the parent 'element' or structure
> differently to represent the "versionRevisionDateHistory" elements - if
> needed.
> 
> More readin regarding the 'attribute' can be found:
> http://msdn.microsoft.com/library/en-us/xmlsdk/htm/xsd_ref_460k.asp
> 
> Yuval.
> 
> -----Original Message-----
> From: Yuval Ben-Itzhak [mailto:yuvalben@netvision.net.il]
> Sent: Tuesday, March 30, 2004 7:46 PM
> To: Mark Curphey; Rogan Dawes
> Cc: was@lists.oasis-open.org
> Subject: RE: [was] How is Test (Detect) progress ?
> 
> 
> 
> Hi,
> 
> I'm trying to sync. was-core0.3.xml (a Schema) to VulnXML-1.4.dtd for
> the Detect project.
> It appears that although was-core0.3.xml is a valid XML document it is
> not a valid Schema.
> 
> Check the following ComplexType:
> 
> <xs:complexType>
> 	<xs:element name="thesaurusEntry" use="required">
> 		<xs:attribute name="thesaurusGroupName"
> type="xs:string"/>
> 		<xs:attribute name="thesaurusSubGroupName"
> type="xs:string"/>
> 		<xs:attribute name="additionalInformation"
> type="xs:string"/>
> 	</xs:element>
> 	<xs:element name="vulnDatabase">
> 		<xs:attribute name="databaseName" type="xs:string"/>
> 		<xs:attribute name="databaseLocation" type="xs:uri"/>
> 		<xs:attribute name="databaseRef" type="xs:string"/>
> 	</xs:element>
> 		<xs:element name="shortDescription" type="xs:string"/>
> 		<xs:element name="longDescription" type="xs:string"/>
> 	</xs:complexType>
> 
> It looks to me that the complextype should include attributes and not
> elements (see other ComplexTypes in the Schema). An element is a parent
> 'element' of a complextype.
> I used an eval of XMLSPY and it alerts on the same issue.
> Here is a reference from MSDN:
> http://msdn.microsoft.com/library/en-us/xmlsdk/htm/xsd_ref_9qpg.asp
> 
> Any comments ?
> 
> 
> Yuval.
> 
> -----Original Message-----
> From: Mark Curphey [mailto:mark.curphey@foundstone.com]
> Sent: Monday, March 29, 2004 4:08 PM
> To: Rogan Dawes; yuvalben@netvision.net.il
> Cc: was@lists.oasis-open.org
> Subject: RE: [was] How is Test (Detect) progress ?
> 
> 
> 
> See comments inline.
> 
> I think we need to move towards putting this schema under version
> control.
> 
> CVS OK with everyone ?
> 
> That said I think you Test folks and Protect folks (Ivan) should develop
> a separate schema and we will integrate it later. This will ensure we
> get less merge conflicts etc. Maybe you can name it WAS-Detect-c.xsd,
> WAS-Protect-x.xsd ? You can upload it to the OASIS site as well for now
> until we get the CVS setup.
> 
> Also I chatted to Ivan about Protect and wanted to facilitate a
> conference call to ensure we don't overlap and reuse as much as is
> possible.
> 
> See other Comments inline
> 
> -----Original Message-----
> From: Rogan Dawes [mailto:rogan@dawes.za.net]
> Sent: Monday, March 29, 2004 6:09 AM
> To: yuvalben@netvision.net.il
> Cc: Mark Curphey; was@lists.oasis-open.org
> Subject: Re: [was] How is Test (Detect) progress ?
> 
> 
> 
> Yuval Ben-Itzhak wrote:
> 
> 
>>As I refresh my memory with all the DTDs and Schemas that are 
>>available, I found that there are many overlaps between the current 
>>VulXML DTD
>>(VulnXML-1.4.dtd) and the Meta schema (oasis-was-version4.xsd). In 
>>addition, the two will need to be in Sync. with the Thesaurus schema 
>>(could not find an available schema) as they all reference each other.
> 
> 
> Yes, there are a number of overlaps. The WAS-Core schema that Mark and
> co have been working on replaces the <TestDescription> element of
> VulnXML completely, I think.
> 
> The section that we need to aim for with Test is the part of VulnXML
> that starts at <Variable>*, and continues with <Connection>+, where *
> indicates optional element, and + indicates one or more Connection
> elements.
> 
> MC > Exactly. This allows Protect and Detect to use the same textual and
> meta-data so its all consistent. The work you Detect folks are doing
> should be on the Test Case only and not the descriptive or signature
> management pieces.
> 
> 
>>1. The initial work is to do some 'clean-up' and 'sync.' between the 
>>documents as well as to have them all with the same format (I prefer 
>>Schema). I remember that in the early days of OWASP, Altova (XMLSpy) 
>>contributed a license/s - is this still available? can I have it for 
>>this task?
> 
> 
> Not sure if it is still available. Mark?
> 
> MC > We all agreed this should be developed in Schema.
> 
> No its not. Well at least not the new versions. If you have VS.NET its
> pretty decent or if you havent used SPY before you can get a 30 day
> trial. Also there are a number of good Java based editors now. I will
> send a mail with the ones I know about.
> 
> 
>>2. After this work, in my opinion, the VulnXML will need to be updated
> 
> 
>>to support all "vulnList" defined in the Meta schema. Currently I 
>>could not see how it will support some of the DoS and Injection types
> 
> defined on the Meta.
> 
> Just a reminder, because I'm not sure that Yuval and I are on the same
> page with regards to how/why VulnXML was designed:
> 
> The intention with VulnXML was not to create "functions" that would
> automatically expand into a test for a SQL injection in one of almost
> infinite permutations. VulnXML was designed to document the exact steps
> that one would go through to replicate a test that has been done before.
> 
> This is typical of tests for static vulnerabilities, much as Whisker and
> Nikto do. This is the problem that VulnXML was intended to solve, and
> the intention of WAS-XML as well.
> 
> MC > Perfect
> 
> WAS-XML is not intended to describe how to automatically compose
> multiple tests for a SQL injection on an arbitrary form (or XSS, or
> whatever). It is intended to facilitate the interchange of known
> vulnerabilities between tools.
> 
> MC > Exactly. WAS 2.0 may though !
> 
> So, there are some aspects of the current VulnXML schema/dtd that would
> need to be refined, but we do not need to code explicit support for each
> of the categories mentioned in Mark's schema. If it can be expressed as
> an HTTP Request (request line, headers and body), and tested for in the
> HTTP Response (status line, headers or response body), we should be able
> to express anything we want to.
> 
> MC > That's true. Remember WAS will have support to describe issues that
> maybe found in code by code scanners (issues that cant be found by black
> box scanners. It is the generic vuln description language that people
> can build business process around as well as taking feeds from
> technology and tools. We shouldn'y insist on a Detect or Protect
> signature for every vuln that would be descibed in Meta IMHO.
> 
> However, as mentioned in previous mails, outstanding items that are
> *NOT* currently possible are multiple simultaneous requests for testing
> locking or synchronisation issues, and reliable/intelligent detection of
> custom 404 responses. These still need to be added to the Test elements
> of WAS-XML.
> 
> 
> 
> 
>>My current goal is # 1
>>
>>Yuval.
>>
> 
> 
> Regards,
> 
> Rogan
> 
> To unsubscribe from this mailing list (and be removed from the roster of
> the OASIS TC), go to
> http://www.oasis-open.org/apps/org/workgroup/was/members/leave_workgroup
> ..php
> ..
> 
> 
> 
> To unsubscribe from this mailing list (and be removed from the roster of
> the OASIS TC), go to
> http://www.oasis-open.org/apps/org/workgroup/was/members/leave_workgroup
> ..php.
> 
> 
> 
> To unsubscribe from this mailing list (and be removed from the roster of
> the OASIS TC), go to
> http://www.oasis-open.org/apps/org/workgroup/was/members/leave_workgroup
> ..php.
> 
> 
> 
> To unsubscribe from this mailing list (and be removed from the roster of the OASIS TC), go to http://www.oasis-open.org/apps/org/workgroup/was/members/leave_workgroup.php.

<?xml version="1.0" encoding="UTF-8"?>

<!--
    Document   : Test.xml
    Created on : March 31, 2004, 5:40 PM
    Author     : knoppix
    Description:
        Purpose of the document follows.
-->

<Test timeout="30000">
    <Step id="0" message="getting cookie">
        <Request reuse="false" delay="1000" mayauth="true">
            <Method>GET</Method>
            <URL>${scheme}://${host}:${port}/index.html</URL>
            <Version>HTTP/1.0</Version>
            <Header name="Host">${host}</Header>
        </Request>
        <Response>
            <SetVar name="status" location="statusline">^(\d\d\d)</SetVar>
            <SetVar name="statusclass" location="statusline">^(\d)\d\d</SetVar>
            <SetVar name="cookie" header="Set-Cookie">JSESSIONID=(\w+)[; ]</SetVar>
        </Response>
        <Decide action="stoptest" log="error">
            <Message>index page not found. Could not get a cookie!</Message>
            <Compare variable="is404" operation="equals">true</Compare>
            <!-- is404 is an implied variable, set by the engine when the page is retrieved -->
            <Compare variable="statusclass" operation="equals">4</Compare>
            <Compare variable="statusclass" operation="equals">5</Compare>
        </Decide>
    </Step>
    <ThreadGroup>
        <Thread>
            <Step id="1" message="logging in">
                <Request reuse="false" delay="1000" mayauth="true">
                    <Method>POST</Method>
                    <URL>${scheme}://${host}:${port}${path}</URL>
                    <Version>HTTP/1.0</Version>
                    <Header name="Host">${host}</Header>
                    <Header name="Cookie">JSESSIONID=${cookie}</Header>
                    <Body length="auto" encoding="text">a=1&b=2</Body>
                </Request>
                <Response>
                    <SetVar name="status" location="statusline">^(\d\d\d)</SetVar>
                    <SetVar name="ctype" header="Content-Type">^(.*)$</SetVar>
                </Response>
                <Decide action="stoptest" log="none">
                    <Compare variable="is404" operation="equals"></Compare>
                    <!-- is404 is an implied variable, set by the engine when the page is retrieved -->
                </Decide>
                <Decide action="stoptest" log="vuln">
                    <Message>Vulnerability detected for URL ${scheme}://${host}:${port}${path}</Message>
                    <Compare variable="status" operation="equal">200</Compare>
                </Decide>
                <Decide action="stopiteration" log="error">
                    <Message>Test failed to execute correctly, due to the server requiring authentication for ${scheme}://${host}:${port}${path}</Message>
                    <Compare variable="status" operation="equals">401</Compare>
                </Decide>
                <Decide action="continue" log="none"></Decide>
            </Step>
        </Thread>
        <Thread>
            <Step id="2" message="a simultaneous request">
                <Request>
                </Request>
                <Response>
                </Response>
                <Decide></Decide>
            </Step>
        </Thread>
        <Decide action="continue" log="none">
        </Decide>
    </ThreadGroup>
    <Step id="3" message="abusing session">
        <Request>
        </Request>
        <Response>
        </Response>
        <Decide></Decide>
    </Step>
</Test>
<!-- 
Actions to perform while running a test:

Stop this iteration, no vuln found -> no report
Stop this Test, no Vuln Found -> no report
Stop this iteration, Vuln found -> report
Stop this test, Vuln found -> report
Stop this iteration, Error in processing -> log/alert
Stop this test, Error in processing -> log/alert
Continue to next step -> implies 1 if no more steps


-->

References:
- RE: [was] How is Test (Detect) progress ?
  - From: "David Raphael" <draphael@citadel.com>