[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Mail from Ingo Struck re VulnXML and WAS
Forwarded from Ingo struck, a new member whose account is not yet active and one of the original VulnXML folks....welcome to WAS Ingo. -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Mark / WAS-list, I saw that you currently submitted the VulnXML dtd together with the old sample test descriptions. However, the VulnXML dtd has changed in many points since. It underwent some normalization (nodes to attributes whereever possible / appropriate, removal of "dummy" nodes) and some repeatability flags have changed. See attached the current VulnXML-1.4.dtd in two versions: raw and (heavily) commented. They reflect the current status of the VulnXML db application that I developed during the last year. I totally agree with the analysis of the work phases and the WAS phases outlined in the last (anonymous) message found on the WAS list: a) problem classification / textual description (WAS classification scheme) b) problem grading (WAS risk ranking) c) detailed technical description how to detect the problem (VulnXML / WAS vulnerability description) These phases clearly outline a priority list too: - - create (and agree upon) a classification scheme - - describe a risk ranking methodology - - integrate a technical description Based upon some experience with thesauri within another scope I would like to add that the central point in creating a thesaurus is not the thesaurus itself, but a *well-defined process how to build it up*. Thesauri / classification schemes tend to be highly dynamic and they are "review-intensive". What we need here is a clear description of how a single thesaurus entry should look like, how revision management should be accomplished and what review processes are involved in building up a sustainable classification scheme. Adding the concrete entries should be a peer-based review and approval process. The definition of the classification scheme should be independent of XML (best accompanied by a simple BNF notation), with XML as a "sample" serialization format. The thesaurus should then contain some basic hints for the risk ranking methodology, e.g. a score value linked to each specific classification entry which could then be comprised in calculating an overall risk score. The risk methodology should take into account all information gathered during the classification and give clear rules how to derive a risk rating from that. Referring to Jeff's last mail / basic characteristics breakdown, I would tend to say that the central point of the classification scheme is to define / look up the "security" characteristics of a concrete vulnerability and to provide appropriate "remedy" characteristics, while the "basic" and "test" characteristics are part of the last (technical) stage. Thus I would reorder and assign Jeff's characteristics as follows: - - classification: "security", "remedy" - - risk ranking: - - - technical description: "basic", "test" While the "exploit" part is clearly out of the scope of a schematical description due to its algorithmic nature (like Jeff already stated) and should be provided as "additional info" only. If you have a look at the current VulnXML dtd, you will see that it already contains a rather detailed description for the "technical" part (Jeff's, "basic" and "test") while being quite sketchy for the classification part. That said, the VulnXML could only be a startover for the "last" stage of the overall WAS deliverable suite, while the "interesting" part of classification / description still needs to be worked out from scratch or based upon other sources (like the paper Jeff mentioned). Kind regards Ingo Struck -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (GNU/Linux) iD8DBQE/VG0GhQivkhmqPSQRAmONAJ9qiGM0Mh6Ki/gyM1k9NW8ilQ8hewCfQUio VaeIromCjvFzu3gJlPBUfOQ= =t/xW -----END PGP SIGNATURE-----
<!ELEMENT WebApplicationTest (TestDescription, Variable*, Connection+)> <!ELEMENT TestDescription (Reference*, Copyright, Description, ApplicableTo*, TriggerOn, Impact, Recommendation)> <!ATTLIST TestDescription name CDATA #REQUIRED version CDATA #REQUIRED released CDATA #REQUIRED updated CDATA #IMPLIED protocol (HTTP | SOAP | WebDAV) #REQUIRED mayproxy (true | false) "false" affects (server | site | path | file) #REQUIRED severity (low | medium | high) #REQUIRED alert (success | failure) #REQUIRED type (Validation | Canonicalisation | ParameterManipulation | AuthenticationSessionManagement | Overflows | Informational | Cryptographic | Configuration) #REQUIRED > <!ELEMENT Reference EMPTY> <!ATTLIST Reference database CDATA #REQUIRED URL CDATA #REQUIRED > <!ELEMENT Copyright (#PCDATA)> <!ELEMENT Description (#PCDATA)> <!ELEMENT ApplicableTo (Platform, WebServer+, ApplicationServer*)> <!ELEMENT Platform (OS, Arch+)> <!ELEMENT OS (#PCDATA)> <!ELEMENT Arch (#PCDATA)> <!ELEMENT WebServer (#PCDATA)> <!ELEMENT ApplicationServer (#PCDATA)> <!ELEMENT TriggerOn (Match+)> <!ATTLIST TriggerOn event (authority | path | file | manual) #REQUIRED > <!ELEMENT Match (#PCDATA)> <!ATTLIST Match type (exact | regex) "exact" > <!ELEMENT Impact (#PCDATA)> <!ELEMENT Recommendation (#PCDATA)> <!ELEMENT Variable (Description, Item+)> <!ATTLIST Variable name CDATA #REQUIRED type CDATA #REQUIRED > <!ELEMENT Item (#PCDATA)> <!ATTLIST Item encoding (text | base64) "text" > <!ELEMENT Connection (Step+)> <!ATTLIST Connection scheme CDATA #REQUIRED host CDATA #REQUIRED port CDATA #REQUIRED > <!ELEMENT Step (Request?, Response, TestCriteria+)> <!ATTLIST Step name CDATA #REQUIRED > <!ELEMENT Request (MessageHeader?, MessageBody?)> <!ELEMENT MessageHeader (Method, URI, Version, Header*)> <!ELEMENT Method (#PCDATA)> <!ATTLIST Method encoding (text | base64) "text" > <!ELEMENT URI (#PCDATA)> <!ATTLIST URI encoding (text | base64) "text" > <!ELEMENT Version (#PCDATA)> <!ATTLIST Version encoding (text | base64) "text" > <!ELEMENT Header (Name, Value)> <!ELEMENT Name (#PCDATA)> <!ATTLIST Name encoding (text | base64) "text"> <!ELEMENT Value (#PCDATA)> <!ATTLIST Value encoding (text | base64) "text"> <!ELEMENT MessageBody (Separator, Item+)> <!ELEMENT Separator (#PCDATA)> <!ATTLIST Separator encoding (text | base64) "text" > <!ELEMENT Response (SetVariable+)> <!ELEMENT SetVariable (Description, Source)> <!ATTLIST SetVariable name CDATA #REQUIRED type (string | int | binary) #REQUIRED > <!ELEMENT Source (#PCDATA)> <!ATTLIST Source source (status | body | header) #REQUIRED > <!ELEMENT TestCriteria (ErrorMessage?, Compare+)> <!ATTLIST TestCriteria type (success | failure) #REQUIRED > <!ELEMENT ErrorMessage (#PCDATA)> <!ELEMENT Compare (Value, Compare*)> <!ATTLIST Compare variable CDATA #REQUIRED test (eq | neq | cont | ncont | gt | lt) #REQUIRED >
<!-- WebApplicationTest describes an automated test performed to discover a specific vulnerability of a web application. --> <!ELEMENT WebApplicationTest (TestDescription, Variable*, Connection+)> <!-- TestDescription is used to describe and to classify a test and the appendent vulnerability. Attributes: name: a unique name that can be used to index tests, or refer to a specific test version: the version number of the test. It should be incremented with each public release. released: the date on which the test was made available. The date MUST be in RFC822 format. updated: the date on which the test was last modified. This is an optional field, but its use is recommended if tests are modified. The date MUST be in RFC822 format. protocol: the protocol used to exploit the vulnerability. Different protocols may have different handling characteristics. Might be one of HTTP, SOAP or WebDAV mayproxy: specifies whether the test may be executed through a proxy, or whether the test requires direct access to the server in question. Note that even if no proxy is specified explicitly in the attack engine's configuration, there may still be transparent proxies between the attack engine and the target server. This could lead to false positives or negatives, depending on the test. DEFAULT: "false" affects: describes the element(s) of the target web application that are affected by this vulnerability. This tells the examiner whether to keep trying the test repeatedly, or if a single instance of the vulnerability is sufficient, i.e. whether the test affects the entire server, or is just specific to a particular virtual host, path or HTML file. (e.g. you need to know an existing .asp file exists before you can test for the "Chunked-Encoding" overflow, but if you find it once, there is no point reporting it for every single .asp file found) may be one of: server, site, path, file severity: a rough severity classification of the vulnerability alert: flag whether the description is positive (i.e. the vulnerability is present if the checks succeed) or negative (i.e. the vulnerability is present if the checks fail) type: categorises the nature of the vulnerability that the test is for. The class attribute refers the vulnerability classes defined by the OWASP asac project, and the URL should point to a web page with more detail about the vulnerability class. This would typically be the owasp.org website, but does not have to be --> <!ELEMENT TestDescription (Reference*, Copyright, Description, ApplicableTo+, TriggerOn, Impact, Recommendation)> <!ATTLIST TestDescription name CDATA #REQUIRED version CDATA #REQUIRED released CDATA #REQUIRED updated CDATA #IMPLIED protocol (HTTP | SOAP | WebDAV) #REQUIRED mayproxy (true | false) "false" affects (server | site | path | file) #REQUIRED severity (low | medium | high) #REQUIRED alert (success | failure) #REQUIRED type (Validation | Canonicalisation | ParameterManipulation | AuthenticationSessionManagement | Overflows | Informational | Cryptographic | Configuration) #REQUIRED > <!-- Reference is intended to provide resources such as BugTraq bid's, CERT announcements, or CVE reference numbers. The URL should point directly to the discussion of the particular vulnerability, wherever possible. The database attribute is freeform text, but would typically be something like bugtraq, CERT, CVE, although it could also point to the discoverer's website. Attributes: database: the plaintext, official full name of the database referred to URL: a valid URL where the referenced database or a specific entry of the referenced database could be found --> <!ELEMENT Reference EMPTY> <!ATTLIST Reference database CDATA #REQUIRED URL CDATA #REQUIRED > <!-- Copyright is used to store a copyright notice from the creator of the test description. We should also allow for an attribute, classifying the nature of the license, such as GPL, or proprietary, if vendor's want to restrict distribution, or use outside of their own product? See Linux Modules tainting discussions. --> <!ELEMENT Copyright (#PCDATA)> <!-- Description is a plain text description of the vulnerability, that could be included in a report or report summary. Could be taken from the CVE description, or formulated on similar lines.--> <!ELEMENT Description (#PCDATA)> <!-- ApplicableTo should be used to reduce the number of tests that need to be run against a particular server. If the tool running can identify the server, in the following ways, they can either eliminate or select tests to run. This section may be repeated as many times as necessary to describe the vulnerable platforms.--> <!ELEMENT ApplicableTo (Platform?, WebServer+, ApplicationServer*)> <!-- Platform can be repeated as many times as required, if it is a cross platform vulnerability. E.g. Apache on Windows and Unix, but not others (e.g. Netware, S390, etc) If it affects ALL platforms, then this can be left blank. It is preferable to be too loose in defining these sections, and test too much, rather than too little.--> <!ELEMENT Platform (OS, Arch+)> <!-- OS should be a string match against the uname or other official product string. Alternatively, it could be a match against the nmap OS id that can be tested against by the vuln application. --> <!ELEMENT Arch (#PCDATA)> <!-- Arch should be the official architecture string. i386, ia64, ppc, ppc64, etc., as per the GNU compiler suite (GCC) --> <!ELEMENT OS (#PCDATA)> <!-- WebServer is a substring match against the Server header returned by the server. This is an exact match and should include a detailed version of the affected server. There can be more than one <WebServer> tag if required. The test should be executed if ANY tags match.--> <!ELEMENT WebServer (#PCDATA)> <!-- ApplicationServer identifies if the test is specific to a particular application server, e.g. WebSphere, ColdFusion, etc. In some cases it could be difficult to determine the application server, especially if it is not identical to the WebServer (e.g. tomcat behind httpd through mod_proxy), so this restriction should be considered only if the application server is well-known. --> <!ELEMENT ApplicationServer (#PCDATA)> <!-- TriggerOn defines three different events currently: authority, path, and file. The test should be triggered when a new instance of the item is discovered. If a URL has multiple levels of path, each level should trigger a separate event. An authority event MUST be triggered for each new matching combination, (e.g. *://host.com:* should generate one event for each of http://host.com:80 ftp://host.com http://host.com:4711), default ports for protocols should not be replicated (e.g. http://host.com:80 http://host.com should only trigger one event). A path event MUST be triggered for each matching path segment (e.g. ba* should trigger two events for the file /bar/foo/baz/basic). A file event MUST be triggered for each matching file (e.g. ba* triggers no event for the path /bar/foo/baz/ and one event for /bar/foo/baz/basic). A manual event cannot be determined automatically and MUST be presented to the user for interactive processing. Attributes: event: describes the URL part that is used to trigger a checking event may be one of: authority, path, file, manual --> <!ELEMENT TriggerOn (Match+)> <!ATTLIST TriggerOn event (authority | path | file | manual) #REQUIRED > <!-- Match defines a matching criterion by means of a regular expression or exact matching, e.g. https://.*:4443/ would only trigger when secure servers running on port 4443 are found, if the event was authority. Attributes: type: flag whether the presented pattern should be treated literally or as a regular expression --> <!ELEMENT Match (#PCDATA)> <!ATTLIST Match type (exact | regex) "exact" > <!-- Impact is a text description of the impact of this vulnerability. It could be as verbose as desired, but should probably be kept to a single paragraph. It would be included in a report, if the vulnerability were found.--> <!ELEMENT Impact (#PCDATA)> <!-- Recommendation contains a text description of what should be done to address the vulnerability. Aimed at the system admin, web admin, or developer, depending on the nature of the test. --> <!ELEMENT Recommendation (#PCDATA)> <!-- A Variable is dereferenced by prefixing it with a dollar sign $, and surrounding it with curly braces { and }. E.g: ${variable} refers to the contents of the variable named variable. Variables will often be defined within data fields, which can have different encoding types. If a field has an encoding type of base64, no variable substitution should be performed on the decoded contents. If a field has an encoding type of text, variable substitution should be performed, replacing any variable references in the above form with their values. There are a number of variables that are predefined, and can be used within a test. These are supplied by the execution engine, on receipt of a new URL to be tested. Examples in the definition will be based on the following URL: http://www.example.com:81/myhost/cgi-bin/formmail.pl They are: ${protocol} This is the protocol component of the URL. It will typically be http, or https. The execution engine is only expected to understand http and https protocols, at least initially. For the example URL, the value would be http. ${host} This is the host component of the URL. This can be dotted-quad, or DNS name. For the example URL, the value would be www.example.com. ${port} This is the port component of the URL. If it is not specified in the actual URL, it is set to the default port value for the protocol. (e.g. http == port 80, https == port 443). For the example URL, the value would be 81. ${path} This is the path component of the URL, excluding the trailing /. If the URL is referencing the default homepage of a site, or any file in the root directory, it would be set to the empty string. Any use of this variable in a path context should be explicitly followed by a trailing slash, if one is appending a file component to it. For the example URL, the value would be /myhost/cgi-bin. ${file} This is the file component of the URL. If no file is specified, for instance when using index pages, this variable will be empty, i.e. set to the empty string. For the example URL, the value would be formmail.pl Attributes: name: variable name, note that this MUST NOT be one of the implicit variable names mentioned above and must follow standard C variable naming conventions, i.e. it may contain characters, digits and an underscore character, but may not begin with a digit. type: defines the variable type. Currently the only type used is string, but it may be useful to define other types such as integer at a later stage. --> <!ELEMENT Variable (Description, Item+)> <!ATTLIST Variable name CDATA #REQUIRED type CDATA #REQUIRED > <!-- Item contains one or more predefined variable values. One could think of it as an array of values. One would expect the test to be executed once for each value in the array. If there is more than one variable defined with more than one value, one would expect the test to be executed the product of the quantity of each variable values. E.g. 4 backup extensions in one variable, and 3 prefixes in another mean the test is executed 12 times per trigger event. In general, the cross product of all variable value vectors creates the room of variable driven events. Attributes: encoding: defines plain text or base64 variable encoding. Possible encoding values are text, which implies no conversion required, and base64, which implies that the value should be base64 decoded when it is read in. base64 encoding would typically be used for binary data, or to include carriage returns in variables, for instance. --> <!ELEMENT Item (#PCDATA)> <!ATTLIST Item encoding (text | base64) "text" > <!-- Connection defines a set of connections made to the target server to check for the vulnerability. The included Step elements must be carried out within a single TCP connection. This is required to support things like Transfer-Encoding: chunked, or NTLM auth, where multiple steps must happen in a single connection. The TCP connection is made to the host and port provided in the original URL sent to the execution engine. In the event of an HTTPS URL, the SSL connection is negotiated on the port provided, and maintained for the duration of the <Connection> element. If more than one Step is specified within a Connection, appropriate attention should be paid to "Connection: keepalive" headers, or protocol version (HTTP/1.1), when writing the test. In the event of a TCP connection being dropped between steps (e.g. if the server closes the connection before a following step can be executed), the test is considered to have failed. Attributes: scheme: the scheme to be used to set up a connection host: the target host to connect to port: the port used for connection By default all these values are filled from the implicit variables that are set by a TriggerOn/authority event, but they can be overwritten to set up a connection somewhere else. --> <!ELEMENT Connection (Step+)> <!ATTLIST Connection scheme CDATA #REQUIRED host CDATA #REQUIRED port CDATA #REQUIRED > <!ELEMENT Step (Request, Response, TestCriteria+)> <!ATTLIST Step name CDATA #REQUIRED > <!-- Request describes the request string that is sent to the web server. One may want to have no header if one is continuing a "Transfer-Encoding: chunked" message, for instance, and simply want to transfer data. One may have no MessageBody when doing a HEAD or GET request, for instance.--> <!ELEMENT Request (MessageHeader?, MessageBody?)> <!-- MessageHeader defines the header of a message. This follows the HTTP protocol specification. It needs to be checked whether this is appropriate for different protocols too. If a MessageHeader element is defined, it should be followed by a blank line, to separate the header from any following data. Note that the data section may imply setting of headers as well, before the blank line is included. If no MessageHeader element is defined, no blank line should be printed, and no headers should be created during processing of the MessageBody element. --> <!ELEMENT MessageHeader (Method, URI, Version, Header*)> <!-- Method contains the HTTP method used to perform the test. This is typically a string like GET, HEAD, POST, etc, but could well be an arbitrary string. This is simply set as the first field on the first header line, followed by a space character. If desired, base64 encoding can also be used to specify arbitrary data to be used as the method. --> <!ELEMENT Method (#PCDATA)> <!ATTLIST Method encoding (text | base64) "text" > <!-- URI contains the request that should be made of the target server. For example, when connecting directly to the target server, on a file event trigger, it would typically be something like: ${path}/${file} It may also include a scheme and host component, for example, if testing for existence of proxy capability. If desired, base64 encoding may be used to supply arbitrary data. This data is simply appended to the Method field, separated by a space.--> <!ELEMENT URI (#PCDATA)> <!ATTLIST URI encoding (text | base64) "text" > <!-- Version specifies the protocol version being used, typically HTTP/1.0, or HTTP/1.1, but may be arbitrary data, or empty. If it is empty, the request line should simply consist of the Method and URI elements.--> <!ELEMENT Version (#PCDATA)> <!ATTLIST Version encoding (text | base64) "text" > <!-- Header specifies a header key/value pair. This is modelled after HTTP and should be checked if it is appropriate for other protocols too. --> <!ELEMENT Header (Name, Value)> <!-- Name is the header name, and should be followed by a colon (:), and then the header value. Arbitrary data can be specified using the base64 encoding type. If a header is specified with an empty value, that header should NOT be set by any other routines, for example, if one specifically requires no UserAgent, or Authentication headers. (The attack engine may be configured to add an Authentication header when testing a password protected site, for example.)--> <!ELEMENT Name (#PCDATA)> <!ATTLIST Name encoding (text | base64) "text"> <!-- Value defines the data appended to a given header key. It could contain arbitrary data, either as plain text or binary using base64 encoding. --> <!ELEMENT Value (#PCDATA)> <!ATTLIST Value encoding (text | base64) "text"> <!ELEMENT MessageBody (Separator, Item+)> <!-- Separator allows the test writer to provide multiple Item elements, and have them separated by the specified separator. This could be an ampersand for a typical POST message, or could be an entire MIME separator string, for multipart forms. The separator should be placed between each decoded Item. If the separator is not set, then each Item should follow on directly from each other. In combination with the Item element, this allows the test writer to specify logically separated items that should be included in the message body. One may choose to separate each variable POSTed to a server for readability, for instance, or may choose to supply the entire POST string in one Item element. Multiple Items should simply be concatenated in order, separated by the Separator if defined.--> <!ELEMENT Separator (#PCDATA)> <!ATTLIST Separator encoding (text | base64) "text" > <!-- Response contains information as to how the response received from the server should be treated. Typically, this involves setting certain variables, based on the response, from the Status Code, through to specific text pattern matches.--> <!ELEMENT Response (SetVariable+)> <!-- SetVariable is similar to the Variable element, except it makes use of a regular expression match to set the variable value, rather than specifying Items. --> <!ELEMENT SetVariable (Description, Source)> <!ATTLIST SetVariable name CDATA #REQUIRED type (string) #REQUIRED > <!-- The Source attribute may be one of status, body or header and describes which string to compare the regular expression to. The element value takes the form of a Perl regular expression, and should include a single referenced pattern that will become the value of the variable (i.e. ${variable} = $1 ). The pattern match should be done as a multi-line pattern match, and can include begin and end of line matches, to e.g. match Header strings.--> <!ELEMENT Source (#PCDATA)> <!ATTLIST Source source (status | body | header) #REQUIRED > <!-- TestCriteria are executed sequentially, until one successfully evaluates a Compare "leaf" node. At that point, if the test type was "success", the next step is executed, if it was "failure", the test is complete. If all TestCriteria elements are processed, and none successfully evaluates a Compare "leaf" node, then there is an implicit "failure". Attributes: type: specifies whether successful evaluation of the tests results in failure or success of the complete test. This is also related to the TestDescription/AlertOn element. If the result of the TestCriteria for the last step results in the a value equal to that specified in AlertOn, it becomes a report item.--> <!ELEMENT TestCriteria (ErrorMessage?, Compare+)> <!ATTLIST TestCriteria type (success | failure) #REQUIRED > <!-- ErrorMessage allows for a test to return a notification to the operator, in the event of an unexpected failure, e.g. the test is looking for the presence of a vulnerable CGI program. One would expect a 404 or 302 response if it is not there, and a 200 response if it is there. The first would indicate failure, in which event no message is required, the second would indicate success, in which event the TestDescription message should be displayed to the operator. However, if the server returns a 500 message, because the server is currently overloaded, the test is inconclusive, and should possibly be rerun, after the operator has investigated and corrected the problem. This element provides for those kinds of messages. Any variables defined using the ${variablename} format should be dereferenced when presenting the message to the operator.--> <!ELEMENT ErrorMessage (#PCDATA)> <!-- Using nested Compare elements it is possible to build complex logic. Compare elements are evaluated depth first, starting at the top, and pruning the tree if a comparison evaluates to false. The TestCriteria is satisfied if a Compare leaf is reached, and evaluates to true, i.e. nested Compare elements form an "and" condition while sibling Compare elements at the same depth form an "or" condition. Attributes: variable: the variable to be used for comparison, it should be worked out what to do on undefined variables test: comparison operator: eq(ual), n(ot) eq(ual), cont(ains), n(ot) cont(ains), g(reater) t(han), l(ess) t(han) All comparisons are string comparisons (for the moment, until other variable types are required) --> <!ELEMENT Compare (Value, Compare*)> <!ATTLIST Compare variable CDATA #REQUIRED test (eq | neq | cont | ncont | gt | lt) #REQUIRED >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]