comment on saml-loa-authncontext-profile: remove 800-63 schemas

["RL 'Bob' Morgan" <rlmorgan@washington.edu>]

Sections 3 and 4 of saml-loa-authncontext-profile-02 define a profile for 
expressing LoA based on NIST 800-63.  I think I understand the motivation 
for having this material in the document, but I think it's a mistake and 
should be removed.  I argue that if assurance concepts are going to be 
useful to the community, assurance profiles have to be defined and 
supported by organizational programs designed for that purpose, not just 
technical specs.

NIST 800-63 was originally written to support the US E-Authentication 
program by specifying technical requirements based on OMB-04-04, which 
defined the infamous 4 levels in business terms.  800-63 is a great 
document, but it was explicitly just a part of an assurance program. 
E-Auth incorporated the technical material in 800-63 into a framework that 
included:

  * taking some elements where 800-63 says "you can do it this way or that
    way" or "methods might be developed to control this factor" and
    providing the necessary additional definition;

  * an auditor-friendly checklist of assessment criteria;

  * an assessment process, including people to answer questions about
    interpretation of the elements, and procedures resulting in assignment
    of level values;

  * assessment processes for relying parties to help them determine their
    LoA requirements;

  * operational specs supporting LoA, including defining how to express the
    levels as values of a SAML attribute.

E-Auth is no more.  But this approach has been carried on by the Liberty 
IAEG, and by the InCommon Federation Identity Assurance program 
(http://incommonfederation.org/assurance).  In the InCommon program we 
have followed the E-Auth model, including a framework with processes for 
assessment, auditable checklists, and URIs that are assigned representing 
assurance profiles (which may be used as values in this SAML authncontext 
profile at some point).

NIST continues to work on revising 800-63, but it is not operating an 
assurance program.  If I have a question about interpreting whether some 
practice in my IdP meets a provision of 800-63, as far as I know there's 
no way to ask NIST that question.  You can't hand 800-63 to an auditor and 
ask them to audit against it.  NIST doesn't review organizations' claims 
to meet 800-63 provisions.  Nor has NIST defined representations of the 4 
levels in protocols such as SAML.

I don't think the SSTC intends, via the publication of this profile with 
these defined 800-63-based schemas, to create an assurance program.  As 
one example, when the next version of 800-63 comes out, presumably the TC 
would be obliged to create a new set of URIs referring to that version. 
That's not hard, but it would raise questions about transition procedures, 
equivalence between the two versions, etc.  The TC could leave these 
questions for someone else to answer, but then have we really done the 
world a favor by defining these schemas in the first place?

It might be argued: these schemas are no different than a profile like 
urn:oasis:names:tc:SAML:2.0:ac:classes:Kerberos from saml-authn-context, 
which is just out there, anyone can use it, it creates no questions, needs 
no program.  And I'd respond: exactly; it has none of those features, and 
it has essentially *no* value in helping sites implement policies that 
meet the assurance needs of inter-organizational federation.  We can treat 
assurance the same way, ie as just a reference to a technical spec, but I 
think this would be profoundly misleading, and would set a very bad 
precedent.

If we agreed to take out the NIST 800-63 material, then the question is
whether to replace it with something that would be more suitable, or just
not include an example in this doc.  I think it would be OK to have the
doc not include an example.  Alternatively a dummy example could be
included, eg with a http://www.example.com/assurance/foo URI.  If we want
to include a real example, I offer two possibilities.  Probably the more
appropriate would be schema for the Liberty IAF 1.0.  I can't tell from
the LIAF material whether there are URIs assigned for the various levels.
Maybe someone knows.  The other possibility would be the InCommon IAF.
InCommon unfortunately hasn't started using SAML 2.0 yet, so this would be
a little speculative, but if people thought it was the best choice we
could push on it.

(Just to mention:  the OpenID PAPE spec has more or less the same feature,
ie a way of sending NIST 800-63 values.  This is also a mistake, in my
view.)

  - RL "Bob"