RE: [office] RE: Directories in Zip packages

["Dennis E. Hamilton" <dennis.hamilton@acm.org>]

OK, I have attached a file that includes a dump analysis of the
helloworld-signature.odt file.  With luck, it won't be merged with this
text.

In all of the entries I examined, the external file attribute is always 00
00 00 00.  There are no special internal attributes or special flags of any
kind that I could see.

The only consistent indication I have is that the directory entries always
have names ending in "/" and they always have compressed and uncompressed
sizes of 0 and there is no compression.  Those are the only discriminating
features.  I am going to see what happens if I actually Zip a directory that
has empty directories to see what happens.

But for now, all I can say is WTF (Waiting to Fail!) and continue to affirm
that directories, however represented in the Zip should not be signed and if
one shows up as a file with data later, the META-INF/documentsignatures.xml
verifier should probably complain because it was not included in the
signature, just as it should complain if some other kind of file shows up.

 - Dennis

PS: When I installed PKZIP, it silently stole the file association for Zip,
so I don't have the built-in Windows XP association any longer.  Crap!!

I am uninstalling it to see if things get back to normal.

-----Original Message-----
From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org] 
Sent: Sunday, September 26, 2010 16:20
To: 'Hanssens Bart'; 'David LeBlanc'; 'office@lists.oasis-open.org'
Cc: 'Cornelis Frank'
Subject: RE: [office] RE: Directories in Zip packages

My additional analysis follows in a separate note.  

Here is what I have:

1. Yes, directories/folders can be represented by file entries.  I was
completely off base about that.  I have confirmed that in APPNOTE 6.2.0.
But I don't think it happens the way you think in this particular file.  I
will check the central directory records I reviewed more closely. 

2. There is no reason that ODF 1.2 ever needs such things in a package,
since the manifest is independent of that stuff.

3. It is not possible to sign a directory/folder entry in the Zip because
there is nothing to sign.  There is no data in such an entry and there
should be no way to insert data in such an entry without it being a file
instead of a folder.  We'll have to see if that is actually a hole in Zip.
(Oddly enough, the 0-length file Configurations2\accelerator\current.xml
*has* data, and it is compressed, but the uncompressed size is 0.  Fancy
that.)

 - Dennis

-----Original Message-----
From: Hanssens Bart [mailto:Bart.Hanssens@fedict.be] 
Sent: Sunday, September 26, 2010 14:13
To: dennis.hamilton@acm.org; 'David LeBlanc'; office@lists.oasis-open.org
Cc: Cornelis Frank
Subject: [office] RE: Directories in Zip packages - was RE: XAdES support in
ODF

Dennis, 

the ZIP appnote  (any version, at least since 4.50) says

" external file attributes: (4 bytes)

          The mapping of the external attributes is
          host-system dependent (see 'version made by').  For
          MS-DOS, the low order byte is the MS-DOS directory
          attribute byte.  If input came from standard input, this
          field is set to zero.
...
file name: (Variable)

          The name of the file, with optional relative path.
          The path stored should not contain a drive or
          device letter, or a leading slash.  All slashes
          should be forward slashes '/' as opposed to
          backwards slashes '\' for compatibility with Amiga
          and UNIX file systems etc.
..."

(Winzip probably 'translates' this for viewing purposes)


But the real questions are:

a) is adding (empty) directories, for whatever reason, allowed per ODF 1.2
spec
(since it requires a somewhat special attribute)

b) if so, should it be signed (I would say yes)

c) in that case, should the ODF spec text "sign every file" be changed in
"every entry" 
(or similar)


Best regards

Bart

________________________________________
From: Dennis E. Hamilton [dennis.hamilton@acm.org]
Sent: Sunday, September 26, 2010 10:48 PM
To: Hanssens Bart; 'David LeBlanc'; office@lists.oasis-open.org
Cc: Cornelis Frank
Subject: Directories in Zip packages - was RE: XAdES support in ODF

For further amusement, the attachment is what PKZip for Windows version
12.50.0013 says about helloworld-signed.odt.

It seems that my (1-4) below have more evidence to deal with.

Re (1), PKWare prefers to show "/" as the segment separator.

Re (2), I have no idea at this point whether this is covered in the APPNOTE
or an extension

Re (3), the observation about no directory entries for Configurations2/,
Thumbnails/, etc., also applies to Configuration2//images as well as
Configurations2/accelerator.

Re (4), the "\" business may be answered.  I know where to find a reasonable
hex editor. Back soon.

 - Dennis


-----Original Message-----
From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org]
Sent: Sunday, September 26, 2010 13:39
To: 'Hanssens Bart'; 'David LeBlanc'; 'office@lists.oasis-open.org'
Cc: 'Cornelis Frank'
Subject: RE: [office] RE: XAdES support in ODF

I knew I had a command-line Zipper somewhere.  Here is what 7zip has to say
about that document:

12:57
7-Zip (A) 4.42  Copyright (c) 1999-2006 Igor Pavlov  2006-05-14
12:07:19.34 C:\MyProjects\java\ODMdev> 7za l helloworld-signed.odt
Listing archive: helloworld-signed.odt

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------
2010-09-24 16:49:42 .....           39           39  mimetype
2010-09-24 16:49:42 .....         3105          813  content.xml
2010-09-24 16:49:42 .....          532          243  manifest.rdf
2010-09-24 16:49:42 .....        10864         1993  styles.xml
2010-09-24 16:49:42 .....         1247         1247  meta.xml
2010-09-24 16:49:42 .....         1018          434
Thumbnails\thumbnail.png
2010-09-24 16:49:42 .....            0            2
Configurations2\accelerator\current.xml
2010-09-24 16:49:42 D....            0            0
Configurations2\progressbar
2010-09-24 16:49:42 D....            0            0  Configurations2\floater
2010-09-24 16:49:42 D....            0            0
Configurations2\popupmenu
2010-09-24 16:49:42 D....            0            0  Configurations2\menubar
2010-09-24 16:49:42 D....            0            0  Configurations2\toolbar
2010-09-24 16:49:42 D....            0            0
Configurations2\images\Bitmaps
2010-09-24 16:49:42 D....            0            0
Configurations2\statusbar
2010-09-24 16:49:42 .....         8773         1365  settings.xml
2010-09-24 16:49:42 .....         1989          344  META-INF\manifest.xml
2010-09-24 18:52:26 .....        30891        13945
META-INF\documentsignatures.xml
------------------- ----- ------------ ------------  ------------
                                 58458        20425  17 files

Hmm, attributes.

So, two things to figure out here.

1. Is it really "\" and I must remember to use that all of the time (which
says something about the resolution of relative IRIs inside the package)?

2. What is this attribute business and whose extension of the APPNOTE is
that or is it really provided in the APPNOTE.

3. Finally, notice that there is *no* such entry for Thumbnails and META-INF
nor for Configurations2 nor Configurations2\accelerator either so one
wonders what the point is that there are Zip entries of any form for those
Configurations2\... goodies (and their having bogus manifest:media-type
values in the manifest.xml markup).

4. Finally still, I have no idea how much of the way these names are
expressed is being done by the utility versus what is in the file (e.g.,
WinZip showed "\" on the ends, 7-zip shows "D" in some sort of attribute
display).  I'm still looking for a decent hex editor.

I think, without fear of contradiction, however, that there is no way to
reflect those D-attribute thingies in the DSig, and we should not want there
to be.  An external signature on the entire package would include whatever
that is, but we're not going there, it seems to me.

And finally, it seems like OIC Appnote time, aye?

 - Dennis

-----Original Message-----
From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org]
Sent: Sunday, September 26, 2010 11:30
To: 'Hanssens Bart'; 'David LeBlanc'; office@lists.oasis-open.org
Cc: 'Cornelis Frank'
Subject: RE: [office] RE: XAdES support in ODF

So, please capture the output of the unzip -l helloword-signed.odt and send
me the text file, since I can't duplicate that on my equipment.

Oh wait, I got it.  There are [bogus?] entries in the global directory at
the end of the file that have no files in the Zip package itself.  They all
happen to have names ending in "\" according to WinZip, which I find passing
strange since I thought the separator in Zip is "/" so I am not sure how
much is literally the case or I have been using the wrong character all of
this time.

I've included a PNG of what WinZip says is in the file, in the order in
which the content appears in the file.

If I perform the Winzip integrity test on the file, here is the report I
get:

No errors detected in compressed data of
\\Whs\Users\orcmid\docs\associazione\standards\OASIS\ODF\Development\hellowo
rld-signed.odt.
Testing ...
testing: mimetype                 OK
testing: content.xml              OK
testing: manifest.rdf             OK
testing: styles.xml               OK
testing: meta.xml                 OK
testing: Thumbnails\thumbnail.png   OK
testing: Configurations2\accelerator\current.xml   OK
testing: Configurations2\progressbar\   OK
testing: Configurations2\floater\   OK
testing: Configurations2\popupmenu\   OK
testing: Configurations2\menubar\   OK
testing: Configurations2\toolbar\   OK
testing: Configurations2\images\Bitmaps\   OK
testing: Configurations2\statusbar\   OK
testing: settings.xml             OK
testing: META-INF\manifest.xml    OK
testing: META-INF\documentsignatures.xml   OK

I need to dig up a decent hex editor and see what is really going on in this
file.  I'll follow-up after I have done that.

-----Original Message-----
From: Hanssens Bart [mailto:Bart.Hanssens@fedict.be]
Sent: Sunday, September 26, 2010 02:30
To: dennis.hamilton@acm.org; 'David LeBlanc'; office@lists.oasis-open.org
Cc: Cornelis Frank
Subject: RE: [office] RE: XAdES support in ODF

Dennis,

I use command line unzip 6.0 (ubuntu) which is based upon info-zip's code.
(unzip -l hello.. just lists the entries in the packages, without unzipping
to the file system)

Or, use hexdump and look at the end of the zip :-)

Which makes sense, because one can also zip (outside ODF context)
empty directories, and they get stored as well (probably with some flag that
says "this might be a directory", then we must check if it is allowed to do
so
within ODF, because the ODF packaging is more restrictive about this)

Bart

[ ... ]

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 
Here is the first file block in the signed package.  This is the mimetype item.

helloworld-signed.odt               │ F1 Help │    Col 0    Line 1          0%
0000 0000 50 4b 03 04 0a 00 00 00  00 00 35 86 38 3d 5e c6   PK♥♦......5å8=^╞
0000 0010 32 0c 27 00 00 00 27 00  00 00 08 00 00 00 6d 69   2.'...'...◘...mi
0000 0020 6d 65 74 79 70 65 61 70  70 6c 69 63 61 74 69 6f   metypeapplicatio
0000 0030 6e 2f 76 6e 64 2e 6f 61  73 69 73 2e 6f 70 65 6e   n/vnd.oasis.open
0000 0040 64 6f 63 75 6d 65 6e 74  2e 74 65 78 74 50 4b 03   document.textPK♥

The central directory record for this file is

0000 5380                                            50 4b                 PK
0000 5390 01 02 0a 00 0a 00 00 00  00 00 35 86 38 3d 5e c6   ☺☻........5å8=^╞
0000 53a0 32 0c 27 00 00 00 27 00  00 00 08 00 00 00 00 00   2.'...'...◘.....
0000 53b0 00 00 00 00 00 00 00 00  00 00 00 00 6d 69 6d 65   ............mime
0000 53c0 74 79 70 65                                        type

Which says it is made by a version 1.0 product that is MS-DOS/OS-2 compatible and requires
a version 1.0 to extract and provides nothing new of interest.  There are no internal or
external file attributes flagged.

As I scroll through the file, the successive file items are

content.xml
manifest.rdf
styles.xml
meta.xml (hmm, not compressed - interesting)
Thumbnails/thumbnail.png (yes, it is "/"!)

And then there is this: <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Configurations2/accelerator/current.xml

And that has this Zip entry:

0000 13c0                                            50 4b                 PK
0000 13d0 03 04 14 00 00 00 08 00  35 86 38 3d 00 00 00 00   ♥♦¶...◘.5å8=....
0000 13e0 02 00 00 00 00 00 00 00  27 00 00 00 43 6f 6e 66   ☻.......'...Conf
0000 13f0 69 67 75 72 61 74 69 6f  6e 73 32 2f 61 63 63 65   igurations2/acce
0000 1400 6c 65 72 61 74 6f 72 2f  63 75 72 72 65 6e 74 2e   lerator/current.
0000 1410 78 6d 6c 03 00                                     xml♥.

Here's what is interesting about this one:

13d3 14 00       (20: version 2.0 needed to extract, because Deflate is used)
                   but, uh, it's not)
13d7 08 00       (compression method 8 = Deflated)
13e0 02 00 00 00 (compressed size is 2)
13e4 00 00 00 00 (compressed size is 0)
1413 03 00       (the bytes of the compressed data)

The central directory entry for this

0000 54e0                                   50 4b 01 02 14              PK☺☻¶
0000 54f0 00 14 00 00 00 08 00 35  86 38 3d 00 00 00 00 02   .¶...◘.5å8=....☻
0000 5500 00 00 00 00 00 00 00 27  00 00 00 00 00 00 00 00   .......'........
0000 5510 00 00 00 00 00 ce 13 00  00 43 6f 6e 66 69 67 75   .....╬‼..Configu
0000 5520 72 61 74 69 6f 6e 73 32  2f 61 63 63 65 6c 65 72   rations2/acceler
0000 5530 61 74 6f 72 2f 63 75 72  72 65 6e 74 2e 78 6d 6c   ator/current.xml

which has that version 2.0 MS-DOS/OS-2 compatible application was used to create this as
well as required to extract.  There are no internal or external file attributes set (i.e.,
the MS-DOS attribute byte is 0).


Configurations2/progressbar/ (with no data) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Here's the Zip file entry for that:

0000 1410 78 6d 6c 03 00 50 4b 03  04 0a 00 00 00 00 00 35        PK♥♦......5
0000 1420 86 38 3d 00 00 00 00 00  00 00 00 00 00 00 00 1c   å8=............∟
0000 1430 00 00 00 43 6f 6e 66 69  67 75 72 61 74 69 6f 6e   ...Configuration
0000 1440 73 32 2f 70 72 6f 67 72  65 73 73 62 61 72 2f      s2/progressbar/

Here's the break down of the "local file header" and the file data (there is none)
starting at offset 0000 1415:

 1415 50 4b 03 04 (ASCII "PK" followed by short 0x0403)
 1419 0a 00       (10: version 1.0 needed to extract - default)
 141B 00 00       (general purpose bit flags)
 141D 00 00       (compression method 0)
 141F 35 86       (last mod file time)
 1421 38 3d       (last mod file date)
 1423 00 00 00 00 (crc-32)
 1427 00 00 00 00 (compressed size)
 142B 00 00 00 00 (uncompressed size)
 142F 1c 00       (file name length 28)
 1431 00 00       (extra field length)
 1433 "Configurations2/progressbar/"  (28-byte file name)
 144F start of the next item, since there is no data following the file name.

The central directory entry for this file is as follows:

0000 5540 50 4b 01 02 0a 00 0a 00  00 00 00 00 35 86 38 3d   PK☺☻........5å8=
0000 5550 00 00 00 00 00 00 00 00  00 00 00 00 1c 00 00 00   ............∟...
0000 5560 00 00 00 00 00 00 00 00  00 00 15 14 00 00 43 6f   ..........§¶..Co
0000 5570 6e 66 69 67 75 72 61 74  69 6f 6e 73 32 2f 70 72   nfigurations2/pr
0000 5580 6f 67 72 65 73 73 62 61  72 2f                     ogressbar/

Again, this is allegedly written in a version 1.0 MS-DOS/OS-2 compatible way and
extraction (to a file system) requires compatible treatment.  There are no internal or
external attribute flags set.

IF THIS IS RECOGNIZED AS A DIRECTORY, IT IS NOT BECAUSE OF THE ANY ATTRIBUTE SETTINGS.
IT MIGHT BE BECAUSE OF THE "/" ENDING THE NAME, IT MAY BE BECAUSE IT IS 0-uncompressed
and compressed, it might be some other provision not explained in APPNOTE 6.2.0.