[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [office-comment] RSQ - Unstable Algorithm
Hi Leonard, On Thursday, 2007-06-07 23:04:10 +0300, Leonard Mada wrote: > Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be > computationally unstable > - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are > notoriously unstable > > Although I did NOT do a formal calculation of the error term, having > *two* occurrences of this unstable calculation, and both times as > *divisors* makes the formula very unreliable. Thanks for pointing out. Did you try it out in OOoCalc and noticed shortcomings? > SUGGESTIONS: > - remove formula from specification I think that is not a good option because it would leave the definition without a mathematical description. Even if the given formula may numerically be instable at some point it is mathematically correct. However, we could add a note saying so. > - consult statistician knowledgeable of stable coding algorithms That of course would be favourable. As I also know from the OOo mailing lists that you're interested in that area, could you help us or are you in contact with one? Eike -- OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]