[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

*Subject*: **RSQ - Unstable Algorithm**

*From*:**Leonard Mada <discoleo@gmx.net>***To*: office-comment@lists.oasis-open.org*Date*: Thu, 07 Jun 2007 23:04:10 +0300

Hi, section 6.16.63 of the OpenDocument Formula document describes the RSQ() function: ''Summary: Returns the square of the Pearson product moment correlation coefficient through data points in known_y's and known_x's.'' Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be computationally unstable - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are notoriously unstable Although I did NOT do a formal calculation of the error term, having *two* occurrences of this unstable calculation, and both times as *divisors* makes the formula very unreliable. SUGGESTIONS: - remove formula from specification - consult statistician knowledgeable of stable coding algorithms Sincerely, Leonard Mada

**Follow-Ups**:**Re: [office-comment] RSQ - Unstable Algorithm***From:*Eike Rathke <erack@sun.com>

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]