What to do about encryption?

[David LeBlanc <dleblanc@exchange.microsoft.com>]

We also need to do something with encryption, as well - the existing approach has a number of problems.

From a mail I sent Malte earlier this morning - 

1) You could replace an encrypted file with one that is not (interesting to see this in the blog post - I came on this Friday independently)
2) There's a known plain-text problem, where the encrypted data would be predictable in many files.
3) The integrity check covers only 1024 bytes.
3a) Giving a hash of the unencrypted data externally to the encrypted area may not be a fatal flaw, but it does give out information that isn't required.
4) We tell people the size of the plain text
5) Information leaks WRT file names - I suspect embedded files may leak interesting info here.
6) Signatures and encryption do not play well together.
7) No provision for an independent password verifier and integrity check
8) Low default spin count on the KDF
9) Telling people to reduce randomness by initializing on time - shouldn't be in a spec - just use as good a random number as you can get - this differs by OS and version.
10) There is no intermediate key, so no possibility of doing an escrow key, or allowing public key encryption on more than one key.

How to fix it:

Use an outer archive to contain the inner archive. The outer archive is itself a valid ODF file, and contains:

1) Encryption information (you could put this in the metadata as you do now, or could make a separate file)
2) Benign metadata (optional)
3) Dummy files (optional)

Integrity check -
1) Generate random salt, at least 1 block size, default = 16 bytes
2) Pre-pend the salt to the inner archive to be encrypted. This removes known plain-text attack when in any chaining mode other than ECB.
3) Encrypt
4) Calculate HMAC over crypt-text

Attacker cannot re-create HMAC without decryption.

Creating a key from a password - 
1) Generate random salt, configurable amount, 16 bytes = default
2) use RFC2898 approach as you do now
3) Configurable spin count, as in current spec, but I'd increase default to at least 50,000. [NOTE: Current Microsoft Office uses 100,000, Office 2007 does 50,000, Elcomsoft considers it a worthy challenge]

Password verification - 

Ideally should be independent of integrity check. The way that MS Office has done this seems to be a good way, but I am sure other approaches are valid:

1) Generate 16 bytes random data.
2) Take hash of random data
3) Encrypt and store both of these

Intermediate key -
Password encryptor is used only to encrypt intermediate key, intermediate key is used for decrypting the inner archive.

The above approach solves all of the issues I listed, and there is no reason one could not add a signature to the outer archive if that was desired. I still think this isn't a mainstream scenario, but it is there if wanted.

Some nuances to consider - 

When defining an algorithm, we need to not just define key length and chaining mode, but also padding mode. I would tend to do each one with individual XML elements, and not combine them as in the current approach.

I'd also suggest that since this approach is very different than the previous approach, that it would also be a good time to make a break from requiring Blowfish, which doesn't meet any government criteria I'm aware of. I'd suggest going with AES-128 as a minimum.

Let's discuss - if this seems like a good overall approach, I can try to turn it into a more detailed proposal.