OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

office-comment message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [office-comment] RSQ - Unstable Algorithm

Hi Leonard,

On Thursday, 2007-06-07 23:04:10 +0300, Leonard Mada wrote:

> Unfortunately, the formulas/algorithms given for 'a' and 'b' seem to be 
> computationally unstable
> - they both use terms of the form n*sum(x^2) - (sum(x))^2 which are 
> notoriously unstable
> Although I did NOT do a formal calculation of the error term, having 
> *two* occurrences of  this unstable calculation, and both times as 
> *divisors* makes the formula very unreliable.

Thanks for pointing out. Did you try it out in OOoCalc and noticed

> - remove formula from specification

I think that is not a good option because it would leave the definition
without a mathematical description. Even if the given formula may
numerically be instable at some point it is mathematically correct.
However, we could add a note saying so.

> - consult statistician knowledgeable of stable coding algorithms

That of course would be favourable. As I also know from the OOo mailing
lists that you're interested in that area, could you help us or are you
in contact with one?


 OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]