RE: [was] WAS Test

["Mark Curphey" <mark.curphey@foundstone.com>]

Hi Rogan / Yuval,

As you know WAS has 4 main elements (to date).

Meta-Data - used to manage sigs such as version control
Profile - descriptive data including the WAS VulnTypes (thesaurus /
classification) and risk ranking model
Test - Used to describe a HTTP test case
Protect - Used to describe an HTTP based protection case.

I will be sending out detailed minutes of yesterdays excellent meeting
this weekend. 

We essentially have 95% completed Meta-Data and Profile which we have
been referring to as WAS Core. We intend to now document it and release
it as a draft at the end of April.

We think Test should be called Detect and there is interest from a few
people to not only describe ways in which an HTTP transaction can test
for the issue but also how to test for it in code. That is obviously not
in scope for WAS 1.0 so well table it for WAS 2.0 but I think
structurally the collection of elements for Detect will look like the
pseudo XML below

<Detect>
	<HTTP Test />
	<Code Test />
</Detect>

For now we wont even describe the Code Test element to avoid my old
friend Scope Creep. The extended work of VulnXML (the actual HTTP test
parts) would go in the HTTP test elements. Anything that was Meta-Data
related should go in the Meta-Data section. VulnXML only really had
small portions of what we now have in WAS Meta and Profile. This data is
really important as it allows people to build systems and process that
manage the signatures. For instance you could pull data about how many
of a specific type of vulnerability is published and how do you fair
against that vulnerability. 

As I will comment on in the minutes, yesterday we produced more than we
would ever have hoped to have done over the list or by phone. I know its
sometimes a little concerning that there is little comment about Test
and Protect but I would just encourage you to build it and put it out
there. If people aren't commenting it means they like and agree with
what they see !

I really really liked what Ivan started to describe in his Protect
document, i.e. a specification for a WAS Protect engine. Maybe we can
take the same approach here with Detect, i.e. describe the engine spec
that is needed to parse and run a WAS signature ?


-----Original Message-----
From: Rogan Dawes [mailto:rogan@dawes.za.net] 
Sent: Wednesday, March 24, 2004 4:00 AM
To: was@lists.oasis-open.org
Subject: Re: [was] WAS Test

Hi,

To the best of my knowledge, there is an overlap between the meta data
section in VulnXML, and the was-core schema. WAS core did not attempt to
cater for the actual execution of the test, rather it is concerned with
the description of the vulnerability, the effects, the recommendations,
etc.

There is no "reference" between the two as such. The intention is to
define the necessary schema for testing the actual vulnerability, in
terms of the actual HTTP request to execute, using the existing VulnXML
elements as a reference, but extending it to cater for the limitations
that have alread been identified, such as 404 detection and concurrency.

As far as WAS-XML goes (as a standard), I don't think that we should
specify when the 404 detection should happen. That would be up to an
individual implementation to decide. A naive implementation that does
not implement a spider as well as the WAS engine would check for one or
two obviously incorrect URLs to base its decision on. A more
sophisticated implementation might spider the site, testing for
obviously non-existent URL's in each directory that it discovers.

That should not be the concern of the WAS-XML standard. Our concern is
to define the callback to the implementation to actually peform the
check, and return a true or false value.

In my opinion, that is ;-)

This is one way on which the implementations can distinguish themselves,
and offer additional value to their users. The better the engine, the
fewer false positives you will get. And an improvement in the 404
detection will not necessitate redoing the entire WAS-XML test base,
too.

Regards,

Rogan

Yuval Ben-Itzhak wrote:

> Let me summarize where we stand on the Test project, as I see it.
> We have Rogan's VulnXML schema + WAS was-core0.3.xml schema.
> There is no reference between the two.
> 
> Let me know if I missed something.
> 
> As for the 404 detection process suggested below - from my experience,

> you should have it at the beginning and at the end of the Crawl phase 
> as during the crawl you might find new directories that might use 
> additional 404 pages. Your suggested XML element is good but to 
> describe the 404 patterns we will need an additional schema.
> 
> Yuval.
> 
> 
> -----Original Message-----
> From: Rogan Dawes [mailto:rogan@dawes.za.net]
> Sent: Monday, March 22, 2004 3:28 PM
> To: Mark Curphey
> Cc: yuvalben@netvision.net.il; was@lists.oasis-open.org
> Subject: Re: [was] WAS Test
> 
> 
> There have not been any significant changes to the last published 
> VulnXML schema, if that is what you are asking. At this point, I have 
> been trying to get feedback from list members on a couple of proposals

> to extend the existing format to support concurrent access (via 
> ThreadGroup and Thread constructs), as well as engine specific 404 
> detection.
> 
> I have no idea what the sqlInjection complex type refers to. It 
> definitely does NOT refer to either the original VulnXML, or the 
> proposed updates.
> 
> Your analysis of the current state of Test is pretty accurate.
> 
> In order to move forward, the most important thing is that people 
> RESPOND. I've made some suggestions, and had NO response, whether 
> mailling folks directly, or via the list. I don't want to do this in a

> vacuum, and don't want to do it by myself.
> 
> I am happy that we move to using schemas to describe WAS-XML.
> 
> To reiterate past discussion, the weak areas currently under 
> "discussion" are concurrency and 404 detection.
> 
> Concurrency is important to allow for tests that require two or more 
> simultaneous requests to reveal a problem, such as failure to lock 
> data structures (see the Webgoat simultaneous logon as an example).
> 
> My suggestion was to create a ThreadGroup element, which contains 2 or

> more Thread elements, followed by a Test element. Each Thread element 
> would contain 1 or more Request-Response-Test elements.
> 
> By extending the spec to allow for specifying delays before each step 
> is executed, we should be able to describe concurrent requests 
> reasonably well. The Test after the Thread elements is intended to 
> test the results of all previous threads together, once all have
completed.
> 
> w.r.t 404 detection, I have come to the conclusion that it is better 
> for the detection of custom 404 responses to be performed by the 
> executing engine, then trying to code all the possibilities into the 
> XML. Firstly, the XML would become out of date really quickly, and 
> actually, I believe that it is impossible to describe all the ways in 
> which web servers react to non-existent pages in such a format.
> 
> It makes sense that the executing engine could probe the server under 
> test prior to the start, request e.g. an index page that DOES exist, 
> request a couple of obviously bogus non-existent URLs, and determine 
> how the particular server responds, and applies similar logic to pages

> retrieved later. It could be as simple as a "<Test URLExists="true">" 
> or whatever. This would result in a callback to the engine which would

> perform some analysis on the Response that it got to the Request, 
> react appropriately.
> 
> suggestions for appropriate attribute tags or tag names are 
> appreciated ;-)
> 
> Regards,
> 
> Rogan
> 
> 
> 
> 
> 
> 
> 
> 
> Mark Curphey wrote:
> 
> 
>>Guys
>>
>>Yuval Ben-Itzak (now an individual member) is going to help act as 
>>custodian to get the Test element complete. I have sent him the last 
>>few weeks emails as I fear they may not have got to him. He has this 
>>question that I thought I would forward.
>>
>><snip>
>>Is Rogan's Schema to describe a vulnerability still valid ? or do we 
>>use another one.
>>As I did not see a definition for the ComplexType element
"sqlInjection"
>>in the file you sent me I thought it probably reference Rogan's schema

>>- am I correct ?
>></snip>
>>
>>This is where I think the Test element development is.
>>
>>Initial VulnXML DTD defined
>>Some weaknesses identified and styles / approaches to moving forward 
>>(calling reusable functions etc) Initial Java execution engine built 
>>into WebScarab for POC Plans for a C# engine to show interoperability 
>>of signatures
>>
>>Things that I know need to happen before WAS 1.0 spec release are;
>>
>>Confirm, explore and document the weaknesses in VulnXML Convert that 
>>existing work to Schema and use schema moving forward Develop test 
>>cases to explore weaknesses and make improvements to schema design 
>>until we are happy with WAS Test 1.0 Develop the Java and C# reference

>>implementations Document
>>
>>
>>
>>
>>
>>
>>
>>To unsubscribe from this mailing list (and be removed from the roster 
>>of
> 
> the OASIS TC), go to
> http://www.oasis-open.org/apps/org/workgroup/was/members/leave_workgro
> up.php
> ..
> 
> 


To unsubscribe from this mailing list (and be removed from the roster of
the OASIS TC), go to
http://www.oasis-open.org/apps/org/workgroup/was/members/leave_workgroup
.php.