Re: [sarif] Interoperability testing

[Chet Ensign <chet.ensign@oasis-open.org>]

That could work too. I was just throwing out the idea that there might be a way for the Open Repo to help encourage and feed into the work.Â

Thinking about it, if the OR was a locus where you could house uniform sample data and expected results, it could help to demonstrate interoperability. "We ran our product on the sample data and here are the results we got."Â

I wasn't pushing for anything by the way - I was just throwing out some ideas.Â

On Tue, Oct 16, 2018 at 8:00 PM David Keaton <dmk@dmk.com> wrote:

Chet,

   I'm not sure I understand. Most of the implementations are
commercial, and would not be able to contribute their source code.

   However, an open repo could be used for the sample code to be
analyzed, and for the results. We could put it in a subdirectory of the
repo we have now.

https://github.com/oasis-tcs/sarif-spec

   Maybe I'm misunderstanding, though.

                    David

On 2018-10-16 12:34, Chet Ensign wrote:
> That's the idea. I thought that the bake-off could become the initial
> code and then it could take on a life of its own - sort of an ongoing
> bake-off.
>
> On Tue, Oct 16, 2018 at 12:29 PM Larry Golding (Myriad Consulting Inc)
> <v-lgold@microsoft.com <mailto:v-lgold@microsoft.com>> wrote:
>
>Â Â ÂChet, are you proposing a repo to which each bake-off participant
>Â Â Âwould contribute their implementation?____
>
>Â Â Â__ __
>
>Â Â Â*From:* sarif@lists.oasis-open.org
>Â Â Â<mailto:sarif@lists.oasis-open.org> <sarif@lists.oasis-open.org
>Â Â Â<mailto:sarif@lists.oasis-open.org>> *On Behalf Of *Chet Ensign
>Â Â Â*Sent:* Monday, October 15, 2018 7:24 AM
>Â Â Â*To:* David Keaton <dmk@dmk.com <mailto:dmk@dmk.com>>
>Â Â Â*Cc:* OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org
>Â Â Â<mailto:sarif@lists.oasis-open.org>>
>Â Â Â*Subject:* Re: [sarif] Interoperability testing____
>
>Â Â Â__ __
>
>Â Â ÂLarry, David, would this be a good candidate for an Open Repo?
>Â Â Âhttps://www.oasis-open.org/policies-guidelines/open-repositories
>Â Â Â<https://na01.safelinks.protection.outlook.com/?url="">>____
>
>Â Â Â__ __
>
>Â Â ÂYou could use it to launch the bakeoff and then it could continue
>Â Â Âonwards. ____
>
>Â Â Â__ __
>
>Â Â Â/chet____
>
>Â Â Â__ __
>
>Â Â ÂOn Fri, Oct 12, 2018 at 8:40 PM David Keaton <dmk@dmk.com
>Â Â Â<mailto:dmk@dmk.com>> wrote:____
>
>        That's right. The idea is to toss different vendors' tools
>    Âtogether and see if they work. For example, one vendor's static
>Â Â Â Â Âanalysis tool might be paired with another vendor's
>Â Â Â Â Âvisualization tool
>Â Â Â Â Âto see if the latter can consume the SARIF emitted by the
>    Âformer. It
>Â Â Â Â Âmight also be worthwhile to try combining the SARIF output of
>Â Â Â Â Âtwo or
>Â Â Â Â Âthree vendors' static analysis tools on the same code, and see
>Â Â Â Â Âif the
>Â Â Â Â Âresulting SARIF makes sense (either by hand or by feeding it into a
>Â Â Â Â Âvisualization tool).
>
>Â Â Â Â Â Â Â Â Each vendor that participates gets to advertise that
>Â Â Â Â Âfact, and
>Â Â Â Â Âoften the process helps people shake out the bugs in their
>Â Â Â Â Âimplementations.
>
>Â Â Â Â Â Â Â Â When security protocols are involved, sometimes people
>Â Â Â Â Âget points
>Â Â Â Â Âfor crashing someone else's code, but I don't think we need to
>Â Â Â Â Âgo that
>Â Â Â Â Âfar. :-)
>
>Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â David
>
>Â Â Â Â ÂOn 10/11/2018 03:52 PM, Larry Golding (Myriad Consulting Inc) wrote:
>Â Â Â Â Â > Thanks David! Could you tell us more about how a bake-off
>Â Â Â Â Âworks? Do a set of tool vendors implement SARIF support in their
>Â Â Â Â Âtools, and we evaluate the results (perhaps by examining the
>Â Â Â Â Âfiles by hand, perhaps by opening them in the Visual Studio
>Â Â Â Â Âviewer)? Is there some sort of incentive to participate ("SARIF
>Â Â Â Â Âmug to the best implementation!" ð)?
>Â Â Â Â Â >
>Â Â Â Â Â > Larry
>Â Â Â Â Â >
>Â Â Â Â Â > -----Original Message-----
>Â Â Â Â Â > From: sarif@lists.oasis-open.org
>Â Â Â Â Â<mailto:sarif@lists.oasis-open.org> <sarif@lists.oasis-open.org
>Â Â Â Â Â<mailto:sarif@lists.oasis-open.org>> On Behalf Of David Keaton
>Â Â Â Â Â > Sent: Thursday, October 11, 2018 8:04 AM
>Â Â Â Â Â > To: sarif@lists.oasis-open.org
>Â Â Â Â Â<mailto:sarif@lists.oasis-open.org>
>Â Â Â Â Â > Subject: [sarif] Interoperability testing
>Â Â Â Â Â >
>Â Â Â Â Â >Â Â Â Â OASIS arranged a talk about SARIF with WhiteSource
>    Âthis morning, because they are thinking of joining the TC. I
>    Âattended so I could answer some questions for them. The
>Â Â Â Â ÂWhiteSource people had an excellent idea which I thought I would
>Â Â Â Â Âpass along.
>Â Â Â Â Â >
>Â Â Â Â Â >Â Â Â Â After the SARIF standard is published, they suggested
>Â Â Â Â Âa bake-off to demonstrate interoperability between tools
>    Âsupporting SARIF. This sort of thing is especially common for
>Â Â Â Â ÂIETF standards where many vendors are expected to interoperate
>Â Â Â Â Âwith each other, and it would be a good fit for SARIF.
>Â Â Â Â Â >
>Â Â Â Â Â >Â Â Â Â There is no need to devote resources to this before
>Â Â Â Â Âour document is published, but it's a good thing to keep in mind
>Â Â Â Â Âgoing forward.
>Â Â Â Â Â >
>Â Â Â Â Â >Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂDavid
>Â Â Â Â Â >
>Â Â Â Â Â >
>Â Â Â Â Â---------------------------------------------------------------------
>Â Â Â Â Â > To unsubscribe from this mail list, you must leave the OASIS
>    ÂTC that generates this mail. Follow this link to all your TCs
>Â Â Â Â Âin OASIS at:
>Â Â Â Â Â >
>Â Â Â Â Âhttps://na01.safelinks.protection.outlook.com/?url="">
>Â Â Â Â Â<https://na01.safelinks.protection.outlook.com/?url="">>
>Â Â Â Â Â >
>
>
>Â Â Â Â Â---------------------------------------------------------------------
>Â Â Â Â ÂTo unsubscribe from this mail list, you must leave the OASIS TC
>Â Â Â Â Âthat
>    Âgenerates this mail. Follow this link to all your TCs in OASIS at:
>Â Â Â Â Âhttps://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
>Â Â Â Â Â<https://na01.safelinks.protection.outlook.com/?url="">>
>Â Â Â Â Â____
>
>
>Â Â Â____
>
>Â Â Â__ __
>
>Â Â Â-- ____
>
>
>Â Â Â/chet
>Â Â Â----------------____
>
>Â Â ÂChet Ensign____
>
>Â Â ÂChief Technical Community Steward
>Â Â ÂOASIS: Advancing open standards for the information society
>Â Â Âhttp://www.oasis-open.org
>Â Â Â<https://na01.safelinks.protection.outlook.com/?url="">>
>
>Â Â ÂPrimary: +1 973-996-2298
>Â Â ÂMobile: +1 201-341-1393 ____
>
>
>
> --
>
> /chet
> ----------------
> Chet Ensign
> Chief Technical Community Steward
> OASIS: Advancing open standards for the information society
> http://www.oasis-open.org
>
> Primary: +1 973-996-2298
> Mobile: +1 201-341-1393

--

/chetÂ
----------------

Chet Ensign

Chief Technical Community Steward
OASIS: Advancing open standards for the information society
http://www.oasis-open.org

Primary: +1 973-996-2298
Mobile: +1 201-341-1393Â