[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RSQ - Unstable Algorithm
Hi, section 6.16.63 of the OpenDocument Formula document describes the RSQ() function: ''Summary: Returns the square of the Pearson product moment correlation coefficient through data points in known_y's and known_x's.'' Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be computationally unstable - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are notoriously unstable Although I did NOT do a formal calculation of the error term, having *two* occurrences of this unstable calculation, and both times as *divisors* makes the formula very unreliable. SUGGESTIONS: - remove formula from specification - consult statistician knowledgeable of stable coding algorithms Sincerely, Leonard Mada
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]