In praise of stable, opaque rule identifiers

Introduction

There has been a lot of discussion on the topic of stable, opaque rule identifiers, some of it instigated by me. It was only recently that a conversation with Michael convinced me that stable, opaque identifiers really provide value to engineering teams. And although the spec no longer absolutely requires rule identifiers to be opaque (Issue #364), still I think it’s worth explaining why they should be. Much of what I write here I will incorporate into the spec at editorial discretion, either before or after the public comment period for CSD.2 begins.

Why stable?

Engineering teams use static analysis tools over years or even decades. The “handles” by which these rules are known become part of the engineer’s vocabulary; security engineers at Microsoft just know that CA5351 means “Do not use broken cryptographic algorithms”. They search log files for “5351”; the type it into their browser bar to find help for it; the use it as a shorthand in conversation (“Did you scrub for 5351?”).

This is why, when Microsoft replaced FxCop with a set of compiler diagnostics based on the Roslyn compiler technology, they kept the same rule ids, even though they were now enforced by a completely different tool.

Why opaque?

Part of the justification for opacity overlaps the justification for stability: the ease of saying it and typing it and searching for it.

But beyond that, opaque identifiers are human-language-neutral. How would you like to understand or remember the rule identifier NeHasználjonTörötKriptográfiaiAlgoritmusokat? That’s what the rule DoNotUseBrokenCryptographicAlgorithms looks like to a Hungarian speaker.

Why not “deprecated”?

Using deprecatedIds helps result management systems to match results produced by a tool that changed its rule ids from one version to the next. But they don’t help engineers who now have to remember that both of those ids are the same, or decide which name to search with.

When Michael proposed deprecatedIds, it wasn’t with the intent of making it easy for tool authors to change their rule ids. It was to help tool authors who decided they’d made a mistake: that they’d authored a rule that was too broad and needed to be split in two, or that two distinct rules really belonged together in one.

Peroration

To be clear, the spec doesn’t explain any of this, and it’s my fault because when I wrote it I didn’t understand the real justification for stable, opaque identifiers. Now that I do, I hope I’ve been able to convey it here.

Thanks for listening!

Larry

sarif message