[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [pkcs11] Points about AEAD and Counter mode APIs - recap from today
On 3/12/2014 3:25 PM, Wan-Teh Chang wrote:
Hi Mike, Thank you very much for your comments and proposal. I did not have time to read your reply until the day before the F2F meeting, but I spent an entire afternoon studying your proposal carefully and adopted the majority of the ideas in your problem statement and proposed solution. Unfortunately, before the F2F meeting I was only able to prepare the slides but not a proper written response. Last night I uploaded a document titled "PKCS #11 Message-Based Encryption and Decryption" to the OASIS site, which should address most of your comments. I'd like to highlight a few points in this email.
I'll take a look this weekend. I'm just back from a two week trip.
1. My proposal allows each AEAD mechanism to specify whether any plaintext may be released during decryption if the ciphertext or associated data is not authentic. However, I do recommend AEAD mechanisms not be used with multiple-part decryption functions.
The problem with that recommendation is that although our major use for such functions is for protocol message traffic (e.g. TLS, IPSEC), I expect to see these uses in other places - say bulk data encryption such as disk drives. If you take a look at the NIST CCM document for example, it describes the use of the mechanism on VERY big data inputs (e.g. north of 2^24 bytes). Of course that's frustrated by the "can't release the data until verified" note, but that's actually a memory management problem rather than a mechanism problem per se.
I wouldn't make ANY such recommendation in the API. I *would* reflect any per-mechanism recommendation made in the underlying mechanism standard (e.g. SP800-38C for CCM I think) in our definition of the mechanism and nowhere else.
2. I am afraid that you misunderstood how the message authentication tag T in draft-mcgrew-aead-aes-cbc-hmac-sha2 can be implemented:
Answered in separate thread.
3. I adopted your idea of allowing existing encryption mechanisms, such as CKM_AES_CBC, to be used with the new message-based encryption functions. This seems like a good idea. I agree CKM_AES_CBC can be easily made to work in message-based style, but I am worried that this isn't the case for all existing encryption mechanisms.
It isn't - the example is sort of AES-CFB - but it could probably be made to work as the per-message state is exactly equivalent to what you have to do for a C_EncryptUpdate using the existing API.
In any event, we'll need to provide guidance on existing mechanisms, or at least on the class (message vs stream) of mechanisms and how they're handled if the mechanism doesn't say anything. We'll also have to provide an error code that indicates the mechanism is not compatible with message encryption modes.
4. IV generators can't be used during decryption because the decryption side already has the IV.
You lack imagination. :-) Imagine a mode where an per-message IV generation nonce is the first thing output as cipher text. It's included as the cipher text, is not actually the IV, but something used to generate the IV for this message, and has to be combined with the other mixins (the session nonce for example) for the IV generator to get the actual IV for the message.
And no, there is no current mode that does this. But I will point out that AEAD modes are only about 10 years old as a stand-alone cryptographic construct. And it turns out that this is pretty similar to the "non predicable IV generator" now recommended for CBC.
Unless there is a good reason to prevent the use of an IV generator on decryption, let's leave whether or not the generator is used as part of the mechanism specification, and not the API spec.
5. I think I am convinced that message-based signature generation and verification is also useful. I just thought signature generation and verification should be easy to add once we nail down how to do encryption and decryption. I should have noted this in my proposal.
Ok. But I think it makes sense to keep the definitions as symmetric as possible. And no great reason to defer them until later. These changes are pretty large and somewhat complex, and i'd rather not have to try and go through this again.
Wan-Teh On Sat, Feb 15, 2014 at 11:30 AM, Michael StJohns <msj@nthpermutation.com> wrote:Please excuse the top posting. I'd actually spent a bit of time commenting on your comments and realized that we were just moving the deck chairs around. Suffice it to say that I don't think the current proposal is correct enough and I don't think it's easy to morph to something that is. One of the reasons for that is that I realized we actually had three issues, not two: The third issue I didn't call out before is the need for a per-message encrypt/decrypt or sign/verify. This is different, but related to the per-message IV generator stuff and has a lot of the same considerations, but it is ultimately driven by the proliferation of network based HSMs. Performance for systems based on these devices is going to be constrained by the number of round trips necessary to do an encryption. The classic Init/Update/Encrypt on a per message basis is expensive in those terms. I grok this is actually more of a concern than the IV generation. So (without considering IV generation or AEAD at this point): What do we need to encrypt/decrypt each message that is common across all messages? a) The key b) The mechanism What do we need to encrypt/decrypt each message that is specific to the message?: a) An IV b) Data to be encrypted/decrypted So to make this work, we need a call to set up persistent state consisting of the key and the mechanism, and that state either has to be implicit to the session, or explicit to an object passed in to the per-message call(s). Wan-teh proposes this be part of the implicit session state, I'd prefer it to be an explicit state, but I can live with his approach. So this gets me to: C_NewAssociation (CK_SESSION_HANDLE hSession, CK_MECHANISM_PTR pMechanism, CK_OBJECT_HANDLE hKey); which sets up implicit session state containing the key and mechanism. It also tells the session the types of calls it will be getting (so the proper error messages are returned if there are attempts to use intermediate C_EncryptUpdate calls for example). And C_EncryptMessage (CK_SESSION_HANDLE hSession, CK_VOID_PTR pIV, CK_BOOLEAN bLastSegment, CK_BYTE_PTR pData, CK_ULONG ulDataLen, CK_BYTE_PTR pEncryptedData, CK_ULONG_PTR pulEncryptedDataLen); C_DecryptMessage (CK_SESSION_HANDLE hSession, CK_VOID_PTR pIV, CK_BOOLEAN bLastSegment, CK_BYTE_PTR pEncryptedData, CK_ULONG ulEncryptedData, CK_BYTE_PTR pData, CK_ULONG_PTR pulDataLen); C_SignMessage (CK_SESSION_HANDLE hSession, CK_BOOLEAN pFirstSegment, CK_BOOLEAN bLastSegment, CK_BYTE_PTR pData, CK_ULONG ulDataLen, CK_BYTE_PTR pIntegrityTag, CK_ULONG_PTR pulIntegrityTagLen); C_VerifyMessage(CK_SESSION_HANDLE hSession, CK_BOOLEAN pFirstSegment, CK_BOOLEAN bLastSegment, CK_BYTE_PTR pData, CK_ULONG ulDataLen, CK_BYTE_PTR pIntegrityTag, CK_ULONG ulIntegrityTagLen); where pIV is the pointer to the iv, format of this defined by the mechanism (doing it this way to allow for IV generator later). If iv is non-null, then the current segment is the first segment of the message bLastSegment is a boolean which indicates this is the last segment of message and that padding etc should be done prior to emitting the last of the cipher text. if pIV is null (or pFirstSegment is false) and bLastSegment is false, this is equivalent to an update operation. if pIV is non-null (or pFirstSegment is true) and bLastSegment is true, this segment is the complete message. All of the above act pretty much as macros for a C_*Init/C_*Update/C_*Final set of calls. C_NewAssociation uses the same calling conventions as C_EncryptInit. C_CloseAssociation (CK_SESSION_HANDLE hSession) is used to close and discard persistent key and mechanism state. Second issue - AEAD ciphers. Two sub cases - generic (Mods to C_EncryptInit/C_EncryptUpdate/C_EncryptFinal calls) and message (Mods to C_EncryptMessage). The first case is one not addressed by Wan Teh - how to fit AEAD ciphers into the current non-association model. AEAD is complex because of the wide range of possible approaches (sign over ciphertext - i.e. after encryption, sign over plain text, AAD before encrypted data, AAD after encrypted data, AAD mixed in with encrypted data) and side issues like McGrew's API proposals. Also, there are a pair of functions that already exist that are somewhat applicable C_SignEncryptUpdate and C_DecryptVerifyUpdate. For below I've left sign before encrypt vs encrypt before sign as part of the mechanism definition - so there's no indication of the order in the following calls. Again leaving the IV generation issues to later and dealing first with the current API model we create two new initialization functions and two new final functions: C_SignEncryptInit (CK_SESSION_HANDLE hSession, CK_MECHANISM_PTR pMechanism, CK_OBJECT_HANDLE hKey, CK_VOID_PTR pIV, CK_BYTE_PTR aadPrefixData, CK_ULONG aadPrefixDataLen); C_DecryptVerifyInit (CK_SESSION_HANDLE hSession, CK_MECHANISM_PTR pMechanism, CK_OBJECT_HANDLE hKey, CK_VOID_PTR pIV, CK_BYTE_PTR aadPrefixData, CK_ULONG aadPrefixDataLen); In the above, I've called out the IV as an explicit argument. Doing it the way I did above means that we can probably define mechanisms that work both with an IV generator and with a supplied IV. The existing C_SignEncryptUpdate and C_DecryptVerifyUpdate can be used to process intermediate data. The final calls are used to pad, encrypt, output and to emit the integrity tag (or the reverse order for decrypt/verify). C_SignEncryptFinal (CK_SESSION_HANDLE hSession, CK_BYTE_PTR aadSuffixData, CK_ULONG aadSuffixDataLen, CK_BYTE_PTR pLastEncryptedPart, CK_ULONG_PTR pulLastEncryptedPartLen, CK_BYTE_PTR pIntegrityTag, CK_ULONG_PTR pulIntegrityTagLen); C_DecryptVerifyFinal (CK_SESSION_HANDLE hSession, CK_BYTE_PTR aadSuffixData, CK_ULONG aadSuffixDataLen, CK_BYTE_PTR pLastDecryptedPart, CK_ULONG_PTR pulLastDecryptedPartLen, CK_BYTE_PTR pIntegrityTag, CK_ULONG ulIntegrityTagLen); The AAD can preceed and/or follow the data to be encrypted. That means that a generic API can handle McGrew's weird AAD | ciphertext | L(AAD) construct as an extension to the generic CCM and GCM models. The mechanism MAY constrain this, but I wouldn't define any that does. Continuing on with AEAD, for the message models we get new message functions (same C_NewAssociation, C_CloseAssociation functions plus): C_EncryptSignMessage (CK_SESSION_HANDLE hSession, CK_VOID_PTR iv, CK_BOOLEAN bLastSegment, CK_BYTE_PTR pPrefixAADData, CK_ULONG ulPrefixAADDataLen, CK_BYTE_PTR pData, CK_ULONG ulDataLen, CK_BYTE_PTR pEncryptedData, CK_ULONG_PTR pulEncryptedDataLen CK_BYTE_PTR pSuffixAADData, CK_ULONG ulSuffixAADDataLen, CK_BYTE_PTR pIntegrityTag, CK_ULONG_PTR pulIntegrityTagLen); C_DecryptVerifyMessage (CK_SESSION_HANDLE hSession, CK_VOID_PTR iv, CK_BOOLEAN bLastSegment, CK_BYTE_PTR pPrefixAADData, CK_ULONG ulPrefixAADDataLen, CK_BYTE_PTR pEncryptedData, CK_ULONG ulEncryptedData, CK_BYTE_PTR pData, CK_ULONG_PTR pulDataLen CK_BYTE_PTR pSuffixAADData, CK_ULONG ulSuffixAADDataLen, CK_BYTE_PTR pIntegrityTag, CK_ULONG_PTR pulIntegrityTagLen); where pIV is the pointer to the iv, format of this defined by the mechanism (doing it this way to allow for IV generator later). If iv is non-null, then the current segment is the first segment of the message bLastSegment is a boolean which indicates this is the last segment of message and that padding etc should be done prior to emitting the last of the cipher text. if pIV is null and bLastSegment is false, this is equivalent to an update operation. if pIV is non-null and bLastSegment is true, this segment is the complete message. AAD can be both prefixed and suffixed to the encrypted message. Finally, we get to actually the least interesting of all of these - IV generation. One of the things I'm concerned with is that the IV generation model differs from protocol to protocol, rather than from mode to mode. So defining one IV generator as part of the cryptographic mode model seems to short sighted. And it also doesn't work all that well if you consider that the decrypt operation can't always generate the IV, but is getting it as supplied in the message (or derived from protocol elements from all over the message). This is where my specification of "CK_VOID_PTR pIV" above comes in: 1) Existing mechanisms that specify the IV as part of the mechanism, continue to take their IVs from the mechanism structure for the non-message functions. For the existing mechanisms using the message functions, the length of the IV is derived from the mechanism or parameter data and the per-message IV is specified as if bExplicitIV = CK_TRUE were specified (see 2 below). 2) New mechanisms that want to use an IV generator will include these two fields in their mechanism params: a) CK_BOOLEAN bExplicitIV b) CK_ULONG ivLength If bExplicitIV is false, then pIV on the init call is a pointer to handle for an IV generator object. if bExplicitIV is true, then pIV on the init call is a pointer to a CK_BYTE - an array of IV bytes of length ivLength. IV generator objects are session storage objects. They have a specific generator mechanism and specific set of attributes that are set at creation (and can be reset for new associations). They have a way to read the next generated and current IVs (for inclusion in the protocol message for example). New types of generator objects can be defined as new protocols are written - but for the start we will define the pseudo-standard CCM (Appendix A counter format function), GCM and CTR generic models. This approach a) allows for any length of message or encrypted stream (e.g. while the mechanism may restrict this, the API doesn't) b) makes no assumption about whether or not decrypted data may be released before the integrity tag is verified. (again, mechanism or mode of HSM may restrict this). c) has no requirement for pre-call marshalling and formatting of the crypto stream (e.g. no assumptions about exactly where the integrity tag is and its length nor where the AAD is and its length - no memory to memory copies should be required prior to the call). d) can deal with AAD preceeding and following the ciphertext/data to be encrypted (future proofing) e) handles message based encryption as a separate and backwards compatible item from AAD and IV generation. (e.g. can actually do per-message CBC stuff within an association using new calls). f) divorces IV generation specification from mechanism specification - same mechanism can use multiple IV generation mechanisms (future proofing). g) mirrors, extends or combines existing functions so easy transitions for future programming. h) possible to use IV generator for decrypt, but not required. i) includes functions to do message verification and signature My general desire here is that we don't want to have to re-do these things because someone has come up with yet another weird AEAD approach and our APIs won't support it. I want to be able to break things down to composable building blocks. I also want to stay as close to the original spec (e.g. for CCM and GCM the NIST documents) as possible rather than possibly niche documents like the McGrew API. Or to put it another way - the difference between an engineer and a bureaucrat is that an engineer takes large insolvable problems and breaks them down into small solvable pieces, where a bureaucrat takes a bunch of small solvable pieces and combines them into one large insolvable mass. I'm trying to keep the pieces useful. Mike
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]