Simplifying Verification (was: Re: [dss] Is a single <Schema> elementadequate?)

[Trevor Perrin <trevp@trevp.net>]

Tommy Lindberg wrote:
> Hi Trevor - 
> 
> 
>>I think the possible solutions are:
>>1) go back to using <DTD>
>>2) add multiple <Schema>
>>3) not support server resolving <ds:Reference>s
>>
>>I'd vote for 3.
> 
> 
> Given the above options, I'd support 2.  The update would be minimal,
> involving a multiplicity
> change, the only downside being somewhat bloated payloads.


True, (2) is a smaller spec change.  However, the more I look at this 
feature, the more I think we should remove it.

Below is an explanation of how verification works in different "modes". 
  I argue that some of these modes don't belong in the DSS core, and we 
could simplify the core by shifting them to profiles.

(This is re-opening settled issues.  I'd like to summarize them again, 
since my understanding has grown, and since implementation experience is 
causing us to revisit them).


Sorry for the length!


Separated and colocated modes
------------------------------
To verify a signature, the server needs to match the <ds:Reference> 
elements inside the signature with the XML elements they point to.  The 
simplest way this happens is when the signatures and what they point to 
are "separated":
  - The client sends a request containing:
    - <ds:Signature> inside <dss:SignatureObject>
    - Input Documents inside <dss:InputDocuments>
  - The server extracts the <ds:Reference>s from the signature
  - For each reference, the server looks for a matching Input Document

However, we also allow the signatures and what they point to to be 
"colocated" in one of two ways:
  1) The pointed-to XML elements can be enveloped inside the 
<ds:Signature> (itself inside the <dss:SignatureObject>)
  2) The <ds:Signature> can be enveloped inside an Input Document 
(itself inside the <dss:InputDocuments>).  In this case, a 
<dss:SignaturePtr> is placed inside <dss:SignatureObject>.

In the colocated scenarios, we mandate that the server must be able to 
evaluate <ds:Reference> URIs containing null URIs and barename XPointers 
against the enveloping element.

Why we have different modes
----------------------------
Why do we have both "separated" and "colocated" ways of doing things? 
The separated approach is capable of handling any signature.  By 
treating the <ds:Reference> URI opaquely, it avoids issues of the server 
searching for signatures, or resolving URIs by evaluating XPath/XPointer 
expressions.  However, it requires the client to do the work of locating 
signatures within documents, resolving <ds:Reference>s, and extracting 
the signature / Input Document elements so they can be passed separately.

The "colocated" approach allows simpler client-side handling of XML 
documents containing enveloped signatures or consisting of enveloping 
signatures - the client just sends the entire XML document.  The server 
searches for signatures to verify (if necessary), then resolves the 
<ds:Reference>s.  However, for the server to resolve the <ds:Reference> 
URIs containing null URIs and barename XPointers, in the general case, 
the client must indicate to the server which elements in the enveloping 
element contain ID attributes.

So with the "colocated" approach, the client has to transmit (or 
pre-arrange knowledge of) schemas, which the server must be able to 
evaluate on the fly (in addition to locating signatures, and supporting 
these different modes in general).  Thus the colocated modes add a lot 
of complexity to the protocol and to server processing.  Also, in many 
scenarios the client knows where the signature is located, and will have 
to resolve the <ds:Reference> URIs anyways to know what the signature 
covers.  In these scenarios, the "colocated" approach doesn't save the 
client much work.

Colocated mode as "document validation"
----------------------------------------
The colocated approach is nice for clients who are using DSS not so much 
as a "signature validation service", but as a "document validation 
service".  These clients can use a profile where they send an entire XML 
document to a server, get back a good/bad response, and then process the 
document contents.  If the server knows the document type beforehand, 
then much of the complexity of colocation goes away (the server knows 
exactly where to look for a signature and how to resolve <ds:Reference>s).

So "document validation" has value.  But it sits uneasily with the 
"signature validation" functionality of DSS:
   - Trying to generalize "document validation" to work without 
pre-arranged knowledge of the document type results in complicated and 
unnecessary server processing.  (In other words: if the server searches 
for signatures, and then searches for whatever <ds:Reference>s are 
matched, and then says "good", this answer means nothing to the client 
unless the client re-creates the steps the server followed to find the 
signature and references.  And if the client was to do all that, he 
could have just used "separated" processing).
  - It makes the verification protocol asymmetric with the signing 
protocol.  The signing protocol is not a "document creation" protocol 
that takes any XML document on input and produces a somehow-signed 
version on output.  The signing protocol takes specific instructions 
about what exactly to sign and produces a signature.  Similarly, you 
would expect the verification protocol to take instructions on what 
exactly to verify.

Removing document validation from core
---------------------------------------
So without questioning the usefulness of document validation, I question 
whether it should be part of the DSS core.  We could eliminate it and 
make Verify processing much simpler:
  1) All pointed-to XML elements must be in <dss:InputDocuments>
  2) The <ds:Signature> must be in <dss:SignatureObject>

Then we would have no server searching for signatures, handling multiple 
signatures at once, resolving references, or evaluating schemas.

Adding document validation as a profile
----------------------------------------
Where would this leave document validation?  Perhaps we should have a 
document-processing profile of DSS that operates on full XML documents, 
instead of signatures.  This profile would assume the server already has 
knowledge of the document type and the steps needed to sign/verify it.

This profile would have to substantially abuse core:  Signing would 
return a full document, not a signature (and might add multiple 
signatures to the document, or transform it in different ways). 
Verifying would take a full document, not a signature (and might verify 
multiple signatures and different aspects of the document).  However, 
the profile could re-use the optional inputs and outputs.  Anyways, I'd 
rather have a profile abuse core then jamming all this into core itself.


Well, this is lot to digest.  If we want to get more implementation 
experience before making big decisions like this, that's fine with me...


Trevor