Re: [sarif] RE: Change draft for #248: Associate file location with repo

[James Kupsch <kupsch@cs.wisc.edu>]

I still prefer my design.  I do not think SARIF should couple the 
mechanisms for resolving relative paths and version control provenance.

Currently, nothing in SARIF requires the use of UriBaseIds, and nothing 
imposes any restrictions on UriBaseIds.  If we use Michael's design, it 
comes with the requirement to use UriBaseIds.  If you want to make 
general restrictions on UrIBaseIds so they work without surprises then 
at least the following restrictions must be enforced:  1) no two 
UriBaseIds can refer to the same path, 2) if the physical location of 
UriBaseId A is under the physical location of UriBaseId B, but no 
others, then the UriBaseId must use B as its uriBaseId, and 3) all 
fileLocations need to generated using the closest ancestor UriBaseId.

Most tools I've seen output either an absolute path or a paths relative 
to a single location.  Even without submodules, there may still be 
multiple directories checked out independent version control systems. 
Now there needs to at one UriBaseId for each, and tools (and engineering 
systems) will need to use these consistently and can't just us a 
BaseUriId to refer to the SOURCE or PACKAGE directory.

With Michael's design a tool or engineering system adding version 
control details, needs to make add these UriBaseIds and possibly modify 
every location to conform to the restrictions above changing every 
baseUriIds and file location in the SARIF file.  With my proposal, the 
versionControlDetails can be easily added as a mostly static piece of 
text without affecting any other fileLocation or BaseUriIds.

Jim



On 12/10/18 4:24 PM, Larry Golding (Myriad Consulting Inc) wrote:
> Iâd like to hear from Jim before reverting to Michaelâs design. Jim, 
> please take a look at the change draft, and consider the arguments below.
> 
> Thanks,
> 
> Larry
> 
> *From:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>
> *Sent:* Monday, December 10, 2018 9:53 AM
> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>; 
> Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC 
> Discussion List <sarif@lists.oasis-open.org>
> *Subject:* RE: Change draft for #248: Associate file location with repo
> 
> Oh, and Iâm sorry â I keep saying âmy designâ, but actually the idea of 
> linking a versionControlDetails to a fileLocation /via/ a uriBaseId was 
> Michaelâs.
> 
> *From:* sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org> 
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On 
> Behalf Of *Larry Golding (Myriad Consulting Inc)
> *Sent:* Monday, December 10, 2018 9:51 AM
> *To:* Michael Fanning <Michael.Fanning@microsoft.com 
> <mailto:Michael.Fanning@microsoft.com>>; OASIS SARIF TC Discussion List 
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>>
> *Subject:* [sarif] RE: Change draft for #248: Associate file location 
> with repo
> 
> Both my design and Jimâs accomplish that. The difference is that in my 
> design, the link from a versionControlsDetails object to a fileLocation 
> is purely a uriBaseId, whereas in Jimâs design, the link is a 
> full-fledged fileLocation object (versionControlDetails.mappedTo). Jimâs 
> point (as I understood it; see my inline commentary on his design in the 
> issue) was that his design simplified life for the SARIF producer 
> because the producer didnât need a separate uriBaseId for each repo.
> 
> Having said that, and having taken the time to write the change draft, 
> Iâm now coming around to your point of view: that weâve traded producer 
> convenience for consumer complexity. Each consumer has to implement that 
> algorithm to get the right mapping.
> 
> The counter-argument is: âThere are lots more producers (tools) than 
> consumers (viewers and result management systems).â So we should favor 
> convenience for consumers. This is actually a design principle weâve 
> followed in the past.
> 
> The counter-counter-argument is: âThis really only helps producers in 
> the case where there are submodules. In other cases, you would generally 
> have one uriBaseId per repo. Why are we complicating life for consumers, 
> just for the sake of consumer convenience in one scenario, and not the 
> most common scenario at that?â
> 
> One more thing: even though only consumers need to /implement/ Jimâs 
> algorithm, both producers and consumers need to /read and understand 
> it/, because producers need to understand how to populate 
> originalUriBaseIds and versionControlDetails.mappedTo in Jimâs design.
> 
> TL;DR: Iâm coming back around to preferring the simplicity of my design.
> 
> Larry
> 
> *From:* Michael Fanning <Michael.Fanning@microsoft.com 
> <mailto:Michael.Fanning@microsoft.com>>
> *Sent:* Monday, December 10, 2018 9:36 AM
> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com 
> <mailto:v-lgold@microsoft.com>>; Larry Golding (Myriad Consulting Inc) 
> <v-lgold@microsoft.com <mailto:v-lgold@microsoft.com>>; OASIS SARIF TC 
> Discussion List <sarif@lists.oasis-open.org 
> <mailto:sarif@lists.oasis-open.org>>
> *Subject:* RE: Change draft for #248: Associate file location with repo
> 
> I am concerned at the difficult in understanding this proposal. I also 
> am wondering whether we have clearly identified the goals of this design.
> 
> Hereâs the scenario that originally drove this request
> 
>  1. I have a SARIF log file and would like to recreate the enlistment
>     locally in order to get source files that match the analysis
>  2. I see an originalUriBase that points to the non-deterministic root
>     of the enlistment
>  3. Iâd like to find the version control details for this root and to
>     sync to it locally.
> 
> Michael
> 
> *From:* sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org> 
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On 
> Behalf Of *Larry Golding (Myriad Consulting Inc)
> *Sent:* Sunday, December 9, 2018 5:04 PM
> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com 
> <mailto:v-lgold@microsoft.com>>; OASIS SARIF TC Discussion List 
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>>
> *Subject:* [sarif] RE: Change draft for #248: Associate file location 
> with repo
> 
> That probably read snarkier than I intended ð Jimâs description is 
> accurate and much more compact than it would have been if heâd expressed 
> it without the symbols. Itâs just that readers of the spec will need 
> some help to understand it.
> 
> *From:* sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org> 
> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On 
> Behalf Of *Larry Golding (Myriad Consulting Inc)
> *Sent:* Sunday, December 9, 2018 5:01 PM
> *To:* OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org 
> <mailto:sarif@lists.oasis-open.org>>
> *Subject:* [sarif] Change draft for #248: Associate file location with repo
> 
> I pushed a change draft for Issue #248 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foasis-tcs%2Fsarif-spec%2Fissues%2F248&data=02%7C01%7Cv-lgold%40microsoft.com%7C538bb4ec21274f013e0008d65ec84d79%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636800611716554327&sdata=18EsQo85bKMbOCBmBPqOvuj7PQRnroPl%2B1CfNhS2DvY%3D&reserved=0>: 
> âVersion control details not strongly associated with resultsâ:
> 
> Documents/ChangeDrafts/Active/sarif-v2.0-issue-248-versionControlProvenance-file-mapping.docx 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foasis-tcs%2Fsarif-spec%2Fblob%2Fmaster%2FDocuments%2FChangeDrafts%2FActive%2Fsarif-v2.0-issue-248-versionControlProvenance-file-mapping.docx&data=02%7C01%7Cv-lgold%40microsoft.com%7C538bb4ec21274f013e0008d65ec84d79%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636800611716564335&sdata=5URBYh3J654dRERJ1Qs3FHzozkj4AmgzF%2Bf6BKXFBtk%3D&reserved=0>
> 
> We will move its adoption at TC 29 on Wednesday December 12^th .
> 
> We had previously proposed a design for this, but Jim objected and 
> proposed what, after a careful reading, I think is a better one. The new 
> change draft follows Jimâs design.
> 
> Jim, academic that he is ð, expressed his mapping algorithm rather 
> abstractly. Fortunately, I was also an academic in a former life, so I 
> interspersed some comments into Jimâs description of his design 
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Foasis-tcs%2Fsarif-spec%2Fissues%2F248%23issuecomment-437942306&data=02%7C01%7Cv-lgold%40microsoft.com%7C538bb4ec21274f013e0008d65ec84d79%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636800611716574344&sdata=N5dgtx1BV8nJUaBDh%2Fihg1lFD2XBcGZ8q9e9dMxPa7M%3D&reserved=0>, 
> and extensively annotated his example in the text of the change draft. 
> *Please follow the example closely* to make sure you know what Jim is 
> proposing.
> 
> Thanks,
> 
> Larry
>