Re: [sarif] Interoperability testing

[David Keaton <dmk@dmk.com>]

     That's right.  The idea is to toss different vendors' tools 
together and see if they work.  For example, one vendor's static 
analysis tool might be paired with another vendor's visualization tool 
to see if the latter can consume the SARIF emitted by the former.  It 
might also be worthwhile to try combining the SARIF output of two or 
three vendors' static analysis tools on the same code, and see if the 
resulting SARIF makes sense (either by hand or by feeding it into a 
visualization tool).

     Each vendor that participates gets to advertise that fact, and 
often the process helps people shake out the bugs in their implementations.

     When security protocols are involved, sometimes people get points 
for crashing someone else's code, but I don't think we need to go that 
far. :-)

					David

On 10/11/2018 03:52 PM, Larry Golding (Myriad Consulting Inc) wrote:
> Thanks David! Could you tell us more about how a bake-off works? Do a set of tool vendors implement SARIF support in their tools, and we evaluate the results (perhaps by examining the files by hand, perhaps by opening them in the Visual Studio viewer)? Is there some sort of incentive to participate ("SARIF mug to the best implementation!" ð)?
>
> Larry
>
> -----Original Message-----
> From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of David Keaton
> Sent: Thursday, October 11, 2018 8:04 AM
> To: sarif@lists.oasis-open.org
> Subject: [sarif] Interoperability testing
>
>        OASIS arranged a talk about SARIF with WhiteSource this morning, because they are thinking of joining the TC.  I attended so I could answer some questions for them.  The WhiteSource people had an excellent idea which I thought I would pass along.
>
>        After the SARIF standard is published, they suggested a bake-off to demonstrate interoperability between tools supporting SARIF.  This sort of thing is especially common for IETF standards where many vendors are expected to interoperate with each other, and it would be a good fit for SARIF.
>
>        There is no need to devote resources to this before our document is published, but it's a good thing to keep in mind going forward.
>
> 					David
>
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail.  Follow this link to all your TCs in OASIS at:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.oasis-open.org%2Fapps%2Forg%2Fworkgroup%2Fportal%2Fmy_workgroups.php&amp;data=02%7C01%7Cv-lgold%40microsoft.com%7C55b75bc14ac449974d4208d62f8abee7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636748670298135664&amp;sdata=Pv4qwByKWaL4o168Ae15oeST6%2B3%2Fr83gBy0qHbvVwnM%3D&amp;reserved=0