Subject: RE: Does making resources.rules an array present a problem?
Rule id collisions occur because some tools use the same rule id to refer to a collection of logically related but distinct diagnostics. The spec gives an example.
I think we should probably take a closer look at the resources object in general. Resources now has a root level dictionary (messages) and an array (rules), an inconsistency worth considering. We have discussed in the TC whether we should retain the resources probing mechanism now that we have developed the external files mechanism. As mentioned in the last TC, my intuition is that we’re getting a bit out of our wheelhouse with the resources mechanism, specifying significant behaviors (related to locale/region, probing for files, etc.) that are outside the static analysis domain.
That’s a general comment. My comment on your specific issue: I agree that our current design raises a concern, we’ve lost some useful coupling between a localized resources file and a log (i.e., a stable resource name used as a key into a table) but this issue can be worked around with some discipline. That requirement for discipline, though, underscores a design weakness we’ve introduced.
Can you or someone else recall for me why we believe rule id collisions are likely to occur? File name and logical name collisions are clear because a single tool + analysis run may operate against targets that share these constructs. Presumably a tool ships a single, consistently named body of rules that apply to results generated. I can’t recall the scenario we had in mind for a collision in rule id (but trust that it turned up). Jim, do you recall?
TL;DR: After consideration, I don’t think there is actually a problem here. But please follow my argument to see if you agree.
Consider a SARIF consumer that has in hand a result object that specifies ruleId. If the consumer needs any rule metadata other than its id – for example, if it needs to display the message – then it needs result.ruleIndex to locate the associated rule object. If resources.rules does not appear in the log file, the consumer must probe for an external localized resource file, which might, for example, reside on a web site.
This means that the external localized resource file can never alter the order of the rule objects it contains. Newer versions of the file can’t reorder elements in the array or remove elements from the array. This wasn’t a problem when resources.rules was a dictionary.
Note that this is not a problem if the rules reside in a “external property file”, because such files are logically part of the root file. It’s only a problem if the consumer has to probe for an external localized resource file. It is also not a problem for run.files or run.logicalLocations.
On further consideration, I don’t think this is a real problem because external localized resource files are always associated with a particular version of the tool, so you’ll have something like this:
So the SARIF consumer will always find the same version of the external localized resource file, and ruleIndex will always be good.
Do you agree?