[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

*Subject*: **GCD and LCM non-integer arguments**

*From*:**Eike Rathke <erack@sun.com>***To*: OASIS ODFF SC <office-formula@lists.oasis-open.org>*Date*: Thu, 18 Jan 2007 17:52:20 +0100

Hi, Section 6.14.33 GCD currently says: | TODO: Applications differ on what happens when non-integers are | provided. Excel 2003 always performs an INT() on each parameter first. | OpenOffice.org 2.0.3 considers the full number, it computes GCD(1.1;2.2) | as 1.1, and GCD(18.1;24.1) as 0. What if 0 returns? OOO has a GCD and | a GCD_ADD. OOo currently tries to calculate GCD and LCM with non-integer arguments, more or less (more less) successfully. Btw, GCD(18.1;24.1) actually returns 7.105427357601E-015 ... Even if there may be applications to calculate a LCM of non-integers, e.g. recurring period of overlapping wave forms, finding a corresponding LCM/GCD pair that suffices the relation |a*b| = GCD(a,b) * LCM(a,b) with GCD really being the _greatest_ common divisor would probably go beyond what we want to define here. I therefor propose to define that arguments are to be converted to INT for both functions. As for negative integer arguments: definitions of GCD say it is the largest integer of the factorization and usually do not define that the two numbers have to be positive integers or natural numbers, but speak of integers. So negative arguments would be permittable, and as there are always two possible solutions a and b for each GCD with a = -b, it would also be correct to return a negative result if one of the arguments is negative, as OOo currently does. However, most GCD definitions say to ignore negative factorization, but it isn't required. Anyway, negative arguments are perfectly valid, just that other applications don't allow them. We could now define a) If one of the arguments is negative, the function results in an error. b) If exactly one of the arguments is negative the result is negative. If both arguments are negative the result is positive. c) The result is always positive. Opinions? Btw, GCD(a;0) results in a. An additional definition allows GCD(0;0)==0, which is useful and also is what Excel and Kspread do. Eike -- Automatic string conversions considered dangerous. They are the GOTO statements of spreadsheets. --Robert Weir on the OpenDocument formula subcommittee's list.

**Follow-Ups**:**Re: [office-formula] GCD and LCM non-integer arguments***From:*"David A. Wheeler" <dwheeler@dwheeler.com>

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]