OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

sarif message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [sarif] First Draft Statement of Relationship to Similar Work


Larry,

I agree, it is a reasonable statement. Here I consolidated the edits:

"SARIF represents a different strategy for common representation of the results of static analysis.  The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space that is integrated with the OMG's software assurance suite.  TOIF normalizes and integrates the output of static analysis tools and other artifacts as evidence for software assurance.

"TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse formats into the lowest common denominator representation without having to modify the original tools.  
By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively.

"Both SARIF and TOIF solve an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings, including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool.â


Best regards,
Nick





On Sep 6, 2019, at 6:48 PM, Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com> wrote:

Nick,
 
Thanks for your explanation of the broad set of information that TOIF represents. That helps me understand the value of SARIF and TOIF working together.
 
To answer your specific question: ââ when the âconvertâ approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not?â:
 
If I understand your question correctly, the answer is ânoâ.
 
A SARIF converter typically consumes as much of the toolâs output as SARIF can represent, including any âadvancedâ concepts that are present in the toolâs output. The converter does not restrict itself to those concepts that are common among most or all tools. If the toolâs output format includes concepts that SARIF does _not_ represent, a converter might (but is not required to) store such information in a âproperty bagâ (which all SARIF objects have). For example, the SARIF âresultâ object does not define a âconfidenceâ property, so if a tool like Fortify produces a confidence value for each issue that it detects, the converter can store it in theResult.properties["confidence"].
 
Is that what you were asking?
 
I agree that the statement we have is a good one.
 
Larry
 
From: Nick Mansourov <nick@kdmanalytics.com> 
Sent: Friday, September 6, 2019 3:37 PM
To: Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>
Cc: David Keaton <dmk@dmk.com>; sarif@lists.oasis-open.org
Subject: Re: [sarif] First Draft Statement of Relationship to Similar Work
 
Larry, 
 
Since SARIF  is  centric to the SCA space and TOIF is not,  when it comes to a focused comparison restricted to the SCA space, the two distinctions  âlowest common denominator/advanced toolsâ and âconvert / modifyâ  capture the key differences quite well. 
 
The distinction of âconvert / modifyâ may seem a minute technical detail, but it is not.
 
TOIF is in a broader space of software assurance, where SCA tool produces but one kind of evidence. Other kinds of evidence are produced by dynamic analysis tools, vulnerability scanning tools, OVAL tools, manual testing, etc.
 
Within the SCA space, TOIF normalizes the output of SCA tools so that their findings can be used as evidence for software assurance. There are other kinds of evidence that are either reported by tools other than SCA, or are not produced by tools whatsoever, so there is no tool to modify (but there is output to convert). Therefore the TOIF ecosystem involves converters. 
 
I think that removing the second distinction  âconvert/modifyâ may introduce an unwanted bias into the comparison by skipping an important aspect related to how the ecosystem of each standard is evolving.
 
To your point, when the âconvertâ approach is used in SARIF, it will address a common denominator between participating tools, not advanced information, would it not?  When the âmodify existing toolâ or even âbuild a new toolâ approach is used in TOIF, this leads to no contradictions to its objectives.
 
The good news is that both SARIF and TOIF can be working together to address both the SCA space  and the larger software assurance space.
 
I think we have a balanced statement (see modified statement below) claiming that both SARIF and TOIF address the software assurance space, where TOIF âs  normalizes and integrates output of static analysis tools and other artifacts as evidence for software assurance, plus the original two points âadvanced / low denominatorâ and âmodify / convertâ.
 
A minor edit : remove the word âinputâ in the following:
 
"TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverse input formats into the lowest common denominator representation without having to modify the original tools.  
 
(in is actually âoutputâ rather than input)
 
What do you think ?
 
Best regards,
Nick
 
 
On Sep 6, 2019, at 4:14 PM, Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com> wrote:
 
My only comment is about this, referring to SARIF: "... which generally requires modifying the tools to produce SARIF output natively".

The spec describes "converters" as well as "direct producers" -- that is, converters are definitely a "thing" in SARIF -- so I suggest: "... which generally requires either modifying the tools to produce SARIF output natively, or writing a converter from the tools's output format to SARIF."

But once you say that -- isn't the same true of TOIF? If you want TOIF, you either have to modify your tool to produce it, or (as TOIF apparently prefers) write a converter.

Larry


-----Original Message-----
From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of David Keaton
Sent: Friday, September 6, 2019 12:10 PM
To: Nick Mansourov <nick@kdmanalytics.com>
Cc: sarif@lists.oasis-open.org
Subject: Re: [sarif] First Draft Statement of Relationship to Similar Work

Nick,

     Thanks.  How about one small change to keep the two strategies together so that the "By contrast . . ." still makes the most sense.

"SARIF represents a different strategy for common representation of the results of static analysis.  The Object Management Group's Tool Output Integration Format (TOIF) is an existing standard in this space that is integrated with the OMG's software assurance suite.  
 
TOIF normalizes the output of static analysis tools so that it can be used as evidence for digital certification of software.
TOIF normalizes and integrates the output of static analysis tools and other artifacts as evidence for software assurance.


"TOIF's strategy involves creating adapters from various tools to the reporting format, and as such, it is focused on integrating the diverseinput formats into the lowest common denominator representation without having to modify the original tools.  
 


By contrast, SARIF aims to support the full capabilities of advanced tools, which generally requires modifying the tools to produce SARIF output natively.

"Both SARIF and TOIF solve an important problem for the organizations performing software assurance by providing a uniform and vendor-neutral way of deploying and running multiple tools on the same code base, disseminating and interpreting the combined findings, including the reduction in the costs of training developers in how to use multiple tools and, especially, how to interpret the results from each tool.â
 


David

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at:
https://nam06.safelinks.protection.outlook.com/?url="">




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]