[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: BITAND, etc. move to 48 bits
I intend to require that implementations must support at least 48 bits to claim that they properly implement BITAND, BITOR, etc. Which means that Gnumeric won't comply with this at first, and this is one of its functions! But at least one Gnumeric developer agrees that this is what SHOULD happen... we seem to have stumbled across an unintended semantic. Please let me know if this is a problem. Otherwise, the next posted version of BITAND will do this... and next week BITAND will be used as a template, and be copied to the rest of the bit operation functions. -- Details -- These functions were originally defined in Gnumeric. Gnumeric does not document a maximum, but the process of generating test cases quickly found out that it didn't realy support more than 2^31-1, and only 32 bits total. (Test cases found this easily, because our guidance says we should test min/max, and the initial test case results did NOT produce the expected values). I haven't looked at their test cases, but they probably just convert the value to a long or int, and this has the unfortunate side-effect of limiting its results to 32 bits on 32-bit systems, and an (ugh!) signed value at that. The signedness is particularly annoying, because it makes it impractical to have applications that can do even better. But this seems to have been completely accidental. There's no reason to limit these to 32-bit signed values. Most apps use at least 64-bit IEEE 754 representation, with 52 bits in the mantissa. There seems little call for 52 bits, but 48 bits _IS_ a very useful size, and using slighlty less than 52 bits means that certain common mistakes won't cause harm. Gnumeric users of BITAND apparantly normally use smaller numbers. Gnumeric can continue to have its own 32-bit signed function if it wishes, naming it ORG.GNUMERIC.BITAND() instead, but it's my hope that they will just completely switch over to this much more general semantic. Once again, the process of standardizing, spec'ing things in detail, and creating test cases has winnowed out a potential misunderstanding. As a result, we can be VERY precise in our spec, much more precise than someone who just wrote some English text but didn't include and generate test cases. One of the tests that I will add will be: =BITAND(281474976710649 ; 281474976710645) with the expected result of: 281474976710641 This is a test for 48 bits. 281474976710649 = 2^48-2-4-1 = 1111...11001 and 281474976710645 = 2^48-2-8-1 = 1111...10101 (with 48 binary digits). This ANDs two values with a 1 on the top and bottom, and all possible combinations of 1 and 0 internally. I'm using the decimal values, so that if there are problems with other functions, we won't get a false result - in general, we want each test to test ONLY that function (to the extent we can). I used Python to compute the answer, b/c it has "infinitely" long precision operations. --- David A. Wheeler
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]