[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

*Subject*: **Re: [office-comment] RSQ - Unstable Algorithm**

*From*:**Eike Rathke <erack@sun.com>***To*: Leonard Mada <discoleo@gmx.net>*Date*: Fri, 8 Jun 2007 18:04:36 +0200

Hi Leonard, On Thursday, 2007-06-07 23:04:10 +0300, Leonard Mada wrote: > Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be > computationally unstable > - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are > notoriously unstable > > Although I did NOT do a formal calculation of the error term, having > *two* occurrences of this unstable calculation, and both times as > *divisors* makes the formula very unreliable. Thanks for pointing out. Did you try it out in OOoCalc and noticed shortcomings? > SUGGESTIONS: > - remove formula from specification I think that is not a good option because it would leave the definition without a mathematical description. Even if the given formula may numerically be instable at some point it is mathematically correct. However, we could add a note saying so. > - consult statistician knowledgeable of stable coding algorithms That of course would be favourable. As I also know from the OOo mailing lists that you're interested in that area, could you help us or are you in contact with one? Eike -- OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS

**Follow-Ups**:**Re: [office-comment] RSQ - Unstable Algorithm***From:*Leonard Mada <discoleo@gmx.net>

**References**:**RSQ - Unstable Algorithm***From:*Leonard Mada <discoleo@gmx.net>

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]