OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

office-comment message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: RSQ - Unstable Algorithm


section 6.16.63 of the OpenDocument Formula document describes  the 
RSQ() function:

''Summary: Returns the square of the Pearson product moment correlation 
coefficient through data points in known_y's and known_x's.''

Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be 
computationally unstable
 - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are 
notoriously unstable

Although I did NOT do a formal calculation of the error term, having 
*two* occurrences of  this unstable calculation, and both times as 
*divisors* makes the formula very unreliable.

 - remove formula from specification
 - consult statistician knowledgeable of stable coding algorithms


Leonard Mada

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]