RE: [was] Agenda for Thursday 25th

["Dawes, Rogan (ZA - Johannesburg)" <rdawes@deloitte.co.za>]

If someone is prepared to show me how best to parse XML into an object, or
object tree, I will write the executor in Java, since it is required for
WebScarab.

For what it is worth, here are my thoughts on how to actually implement a
systematic processing of WAS-XML tests in WebScarab. Places where we have
user interaction could always be replaced by programmatic logic, if desired.

This is very roughly done, mostly notes to myself, so if anything is
unclear, or too rambling, please let me know.

Rogan




The Attack generator maintains a "master AttackQueue" ( an ArrayList of
Attacks ?), which can be stopped, started, moved up and down (prioritized),
etc. The Attack keeps the URL that is being targeted, and a list of
applicable WAS-XML attacks.

As we see a new URL, consult a list of WAS-XML tests, for host:port,
directories, or files. 

For example, as we see a new host, we have a number of applicable WAS-XML
tests. Those tests would be composed into an "Attack against the URL in
question", and passed to the master queue for execution. The user would have
control over the actual execution of attacks, at the top level, at the very
least. 

(Tests are only queued when we know that the host or directory or file
actually exists. e.g. when we get a successful page response (2xx) or 3xx
for a directory. Tests are not queued for implied links or directories
before we know if the directory exists or not. This then depends on the
spider to force fetching of those implied links.)

So:

We get a new host. We determine that there are 15 WAS-XML tests that we
should execute that would apply to that host (some CGI-tests, fetching
robots.txt, whatever). The WAS-XML tests are packaged into an Attack along
with the URL that they apply to, and queued. (We may want to have some
method of selecting which WAS-XML tests to include automatically, similarly
to Nessus) (We may also want to incorporate functionality from webserver
fingerprint to more accurately identify the web server that we are testing)

The user reviews the WAS-XML queue (toplevel, by URL and Attack description)
and may review the Attack details to disable specific tests if desired).
Then the user chooses to execute the attack (from wherever it is in the list
of possible Attacks). The attack then moves to the WAS-XML attack executor,
where it is executed. Attacks that reach the executor are executed
automatically (if the executor is so configured), but the executor can be
stopped/paused/restarted by the user.

For each test in the attack, the WAS-XML is expanded using the cross product
approach suggested by Ingo, to arrive at a final number of tests to execute.
Each row is then assigned a reference number, so that the executor can track
which have been executed, and which have timedout, or whatever.

The executor is then passed a single instance of a test, with all additional
variables filled in. the executor then performs the test, and passes back
the reference number and the test status. If the status indicates a result,
it is logged for reporting. If the test indicates nothing interesting, it is
marked as completed by the master executor. When all the tests in a
particular WAS-XML are completed, the WAS-XML is marked as completed (and
hidden in the display?). When all WAS-XML for an Attack are completed, that
Attack is removed from the queue.

All pages that return a non-4xx response should be added to the model, for
storage. Should we store all of them, maybe?

Implementation would look something like:

Attack
    String URL;
    WAS-XML[] tests; // This may actually be the entire list of attacks
    boolean[] enabled; // We enable by default those that "apply" due to
matching web server, app server, platform, etc characteristics, and appear
in the "enabled" tests list.
    boolean[] completed;

AttackQueue
    ArrayList attacks;

AttackExploder
    WAS-XML test;
    Map[] crossproducts; // contains the various permutations of variables,
based on the provided lists in the test.
    // calls a StepsExecutor with the Steps to execute, and the Variables
that it should execute with
    ArrayList results;

StepsExecutor
    Steps steps; // the sequence of steps to execute and the checks to
perform against them.
    Map variables; // Any variables that may be required for the execution
    // returns the result of a single test.

So a single Attack consists of an URL, and multiple WAS-XML descriptions,
which may or may not be enabled.

The AttackExploder takes each Attack, and (threadedly) distributes the URL
and a single WAS-XML to an AttackExploder, and tracks the results.

the AttackExploder takes an URL and a single WAS-XML description, takes the
cross-product of the variables, and iteratively (or threadedly) passes a Map
of variables, and the necessary Steps to perform to a StepsExecutor, and
tracks the results. Once all of the cross-products have completed, it
returns to the AttackQueue.

Ultimately, we should be able to track the state of any tests that are in
the queue.

Select the URL, we get a list of WAS-XML.

Select the WAS-XML, we should get a list of cross-products, and we should be
able to see at which point in the particular test we are.




> -----Original Message-----
> From: Ingo Struck [mailto:ingo@ingostruck.de] 
> Sent: 25 September 2003 10:37 AM
> To: Dawes, Rogan (ZA - Johannesburg); 'mark@curphey.com'; 
> was@lists.oasis-open.org
> Subject: Re: [was] Agenda for Thursday 25th
> 
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Folks, 
> 
> I will try to participate today - let's see if it works overseas. ;o)
> Sorry for my silence during the last weeks, but we are currently
> setting up a very gainful project right now, that will last 
> till December.
> 
> Some annotations regarding the "draft"
> - - naming: please remove all non-alpha chars from the names.
>   names containing blanks or other special characters are 
> always problematic
>   during data processing (normalization etc. pp.)
>   "VulnDB's" or "Risk Ranking" are not acceptable
> - - naming: please hold on to a strict naming convention, lets say
>   all lowercase or java convention (starting with a lowercase char),
>   e.g. "Risk Ranking" -> "riskRanking"
> - - Remedy group: I don't think that a "Patch" is sufficient 
> here. Most often
>   the remedy does not consist of a simple patch, but of an abstract
>   instruction. Thus the remedy should contain a textual 
> description too.
> - - ApplicableTo left out: I guess this is *the* criterion 
> one would like to
>   search for. The default scenario for me is: "I have got app 
> server x and
>   web server y on platform z, so what issues are known for that?"
>   Everything else is only a refinement (e.g. "only those of 
> the last month",
>   "only the GPLd ones", etc.)
>   So the applicableTo thing is a central point for retrieval.
>   BTW the ApplicableTo as found in the current VulnXML DTD is one of
>   the most over-worked things there: the cardinality and 
> structure of the
>   parts should be exactly what we need, so we could just 
> adopt that part.
> - - data entry stuff: I still dont understand why we should 
> write yet another
>   "skunkwork" editor to perform data entry based upon xml:schema while
>   having a completely functional DTD based editor online that could be
>   easily adapted.
> 
> As for the extension of the VulnXML execution logic:
> I think it would be better to write a working executor based upon
> what we have now as a proof-of-concept (the python based stuff
> is rather outdated and I dont know, if someone is willing to adapt it)
> before thinking of extensions. 
> 
> Let's discuss the the minutiae later on. :o)
> 
> Kind regards
> 
> Ingo
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.0 (GNU/Linux)
> 
> iD8DBQE/cqkkhQivkhmqPSQRAnKEAKDMk0h8XCWwL3CKr/C9HZPd/yRFwACgpcs8
> /gaQ2BP2Su54u+3yIjZmI68=
> =wxw4
> -----END PGP SIGNATURE-----
> 

Important Notice: This email is subject to important restrictions, qualifications and disclaimers ("the Disclaimer") that must be accessed and read by clicking here or by copying and pasting the following address into your Internet browser's address bar: http://www.Deloitte.co.za/Disc.htm. The Disclaimer is deemed to form part of the content of this email in terms of Section 11 of the Electronic Communications and Transactions Act, 25 of 2002. If you cannot access the Disclaimer, please obtain a copy thereof from us by sending an email to ClientServiceCentre@Deloitte.co.za.