OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

office-formula message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: BITAND, etc. move to 48 bits


I intend to require that implementations must support at least 48 bits
to claim that they properly implement BITAND, BITOR, etc.  Which means
that Gnumeric won't comply with this at first, and this is one of its functions!
But at least one Gnumeric developer agrees that this is what SHOULD happen...
we seem to have stumbled across an unintended semantic.

Please let me know if this is a problem. Otherwise, the next posted version
of BITAND will do this... and next week BITAND will be used as a
template, and be copied to the rest of the bit operation functions.

-- Details --

These functions were originally defined in Gnumeric.   Gnumeric does not document
a maximum, but the process of generating test cases quickly found out
that it didn't realy support more than 2^31-1, and only 32 bits total.
(Test cases found this easily, because our guidance says we should test
min/max, and the initial test case results did NOT produce the expected values).
I haven't looked at their test cases, but they probably just convert the value to a
long or int, and this has the unfortunate side-effect of limiting its results to 32 bits
on 32-bit systems, and an (ugh!) signed value at that.  The signedness is
particularly annoying, because it makes it impractical to have applications
that can do even better.

But this seems to have been completely accidental.  There's no reason to
limit these to 32-bit signed values.  Most apps use at least 64-bit IEEE 754
representation, with 52 bits in the mantissa.  There seems little call for 52 bits,
but 48 bits _IS_ a very useful size, and using slighlty less than 52 bits
means that certain common mistakes won't cause harm.

Gnumeric users of BITAND apparantly normally use smaller numbers.
Gnumeric can continue to have its own 32-bit signed function if it wishes, naming it
ORG.GNUMERIC.BITAND() instead, but it's my hope that they will just
completely switch over to this much more general semantic.

Once again, the process of standardizing, spec'ing things in detail, and
creating test cases has winnowed out a potential misunderstanding.
As a result, we can be VERY precise in our spec, much more precise than
someone who just wrote some English text but didn't include and generate
test cases.

One of the tests that I will add will be:
=BITAND(281474976710649 ; 281474976710645)
with the expected result of:
281474976710641
This is a test for 48 bits.  281474976710649  = 2^48-2-4-1 = 1111...11001 and
281474976710645 = 2^48-2-8-1 = 1111...10101 (with 48 binary digits).
This ANDs two values with a 1 on the top and bottom, and all possible
combinations of 1 and 0 internally.  I'm using the decimal values, so that
if there are problems with other functions, we won't get a false result - in
general, we want each test to test ONLY that function (to the extent we can).
I used Python to compute the answer, b/c it has "infinitely" long precision operations.

--- David A. Wheeler 


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]