FOSS 2.0 - was: Add FFT() to formula spec?

["Leonard Mada" <discoleo@gmx.net>]

FOSS 2.0 - Was: Add FFT() to formula spec?


Hi to everyone,

I will briefly discuss the FFT-issue, adopting later a more comprehensive
approach to the problem.

Because the FOSS 2.0 concept and my general approach to the problem
do touch some usability aspects, I have forwarded this mail to the
OOo-UX mailing list, too.


A.) FFT
=======

To cut the story short, I support in general any additional features.

So, in principle I would support also the implementation of FFT.
BUT, in the 2nd part of this mail I will describe a more useful
approach.

Surely, FFT is very important in various science-fields, so some users
will benefit from such a function. IF another solution is NOT feasible,
than I fully agree to implement FFT directly into the spreadsheet.


B.) THE PROBLEM
===============

What is the problem?
Well, there are many useful features. Way too many.

While FFT is highly used in signal processing, numerous other techniques
are employed even in this one particular field. As a short example, the
Empirical Mode Decomposition (Huang et al.) addresses various shortcomings
of FFT. Therefore it is sensible to think of implementing EMP sometimes
in the future. (Unfortunately EMP is patented by NASA, so it might be
difficult to get it actually implemented).

Moreover, some methods are not easy to implement and necessitate complex
algorithms and special validation tools (e.g. the validation of Bayesian
techniques).

Often free alternatives do exist, so this will create duplicate effort
and difficult to maintain code.


C. SOLUTION
===========

As I pointed out in an older e-mail to this list, I strongly advocate
reusing existing software to solve particular problems.

Instead of reinventing the wheel, spreadsheet applications should
be able to integrate tightly with existing software. There are excellent programs specifically suited for particular tasks. These features would
never be matched by spreadsheet applications. But this is really NOT
necessary.

What is really missing is a general and standardised way to call external
functions from within the spreadsheet.

See also my previous post on this issue:
http://lists.oasis-open.org/archives/office-comment/200706/msg00019.html

How should such a function look like?
This is only a very basic suggestion, but I can imagine something like:
  'FUNCTION_NAME' (
     'range',
     'EXTERNAL_FUNCTION_NAME',
     'EXTERNAL_PROGRAM/LOCATION',
     '...'
   )

where '...' stands for arguments passed to the external function/program.

There should be probably functions to pass:
  - single data values
  - simple arrays of same type (vectors)
  - matrices of same type
  - data lists / data frames
  - probably more complex objects
[I have constructed this list basically on the object models of R,
 see http://cran.R-project.org ]

Therefore, one could access external resources in a standardised and
documented way. Also, such spreadsheets would be much more easy to
maintain and debug. It will also foster collaboration between the various
developers.


D.) FOSS 2.0
============

The term FOSS 2.0 has previously surfaced on the internet, though it is
vaguely defined and mentioned only seldom. I will convey it a different
meaning.

What is FOSS 2.0?

Well, many people consider FOSS as free software. Yes, they are free,
you have the source code, you can change it and you can redistribute
the changed version.

BUT, may I ask, how many users can usefully modify the source-code
to implement their much needed features? definitely not many.

So basically, although this software was declared as free, most of us
are bound to use the existing/unmodified version.

IF I wish to use one software to perform some tasks, and then use
a different software to perform some other tasks (because the 2nd
program is better suited for those tasks), there is NOT much hope
in accomplishing this. There are sometimes some workarounds, to
save as an external file, often in an inferior format, where most
of the additional information is lost. This is just a workaround.

REAL FREEDOM
============
Real freedom means that for every task during a work-pathway, I can
freely choose whichever software suits my particular needs for that
specific task. And IF I need for 2 tasks, 2 different programs,
then I shall be able to use both programs. Both programs should work
together in a standardised way, without any limitations. This is real
freedom. Not bound to a particular program, being proprietary or "free".

In FOSS 2.0 I expect to have the ability to use any external program
within the first one. I expect standardised interfaces for such usage.
That one can create a complex hierarchical structure of simpler programs
in order to solve more complex tasks. That there are NO bounds or limits
in the ability to combine simpler programs into a complex data-pipeline.

Standardising the accessibility to external applications is a sensible
task towards FOSS 2.0.

NOTE
====
Please note, that standardised file formats are NOT a way towards
FOSS 2.0 (there was recently a discussion on the gnumeric list on the
use of gretl, a very powerful package for financial analysis, emphsising
how far we are from FOSS 2.0).
While they may allow implementing a 2nd spreadsheet program, they
do NOT foster interactivity between 2 different programs:
 - in order to read the file, you will have basically to implement
   the whole spreadsheet engine
   (duplicate effort, bugs, maintenance of code, ...)
 - the 2 programs run completely separately, so you loose basically ALL
   INTERACTIVITY

There are many reasons for a FOSS 2.0 goal. Unfortunately, such a move
is barely visible, but I hope to see some improvements at least in
the filed of spreadsheet applications.

Sincerely,

Leonard Mada

======================

Initial e-mail:

> Betreff: [office-comment] Add FFT() to formula spec?

> Hi - We've received a proposal to add a Fast Fourier Transform (FFT)
> function to the OpenFormula spec.  The proposal itself is from Steven G.
> Johnson; Rob Weir wrote a public reply to it.
> 
> What does everyone think?  Pro? Con? Don't care?  I don't think this
> should be required for small or medium; heck, it doesn't even need to be
> included in a group at all.  Details below.
> 
> Arguments for it: Excel has an FFT in a data pack; FFT is common in
> engineering.  By spec'ing it, we increase portability in this area.
> Arguments against it: Many spreadsheets don't have it, or have it as a
> standalone function; most spreadsheet users don't know or care what it is. 
> Implementations can add their own functions like FFT, they don't need
> OpenFormula's permission to do so.
> 
> In college I wrote a paper comparing parallel FFT implementations, and
> like most electronics engineers I've used FFTs.  So I'm quite aware that FFTs
> have many users.  I'm inclined to say "yes, if the proposer writes the
> initial spec, and isn't required for small or medium".  But I'd like to hear
> what others think.
> 
> --- David A. Wheeler 
> 
> 
> ----- Start Original Message -----
> > From: "Steven G. Johnson" stevenj, at, fftw.org
> > 01/28/2008 05:58 PM
> > Please respond to stevenj, at, math.mit.edu
> > 
> > To  office-comment at, lists.oasis-open.org
> > Subject [office-comment] office-formula: FFT functions?
> > 
> > Hi, I was wondering if there was any interest in including a discrete 
> > Fourier transform function in OpenFormula (and, based on that, one can 
> > easily implement convolutions, correlations, autocorrelations, etcetera
> as 
> > 
> > desired). e.g. Excel has some FFT functionality via the data analysis 
> > toolpack, as I understand it, but I didn't see anything in the
> OpenFormula 
> > draft.
> > 
> > I'm co-author of FFTW (www.fftw.org), a popular free-software FFT 
> > implementation (used e.g. in Matlab, GNU Octave, etc.), and I wanted to 
> > send you a note in case you wanted any help with this.  I would be happy
> > to provide any advice on specification, a reference implementation, 
> > etcetera that you might need.  I'm not a spreadsheet expert, but I hope 
> > that my knowledge of FFTs and their applications would be helpful.
> > 
> > We get a sizeable amount of email at fftw.org from people who just want
> to 
> > 
> > transform some data in a spreadsheet, and so I get the impression that 
> > that there is a fair demand for this kind of capability; also, if you 
> > google "excel fft" you will get a lot of pages.  FFTs themselves are
> kind 
> > of hard for a user to implement in a spreadsheet language, but once you 
> > have them there are all sorts of things that can be done directly in the
> > formula language (windowing, filtering, correlations, etcetera).
> > 
> > Regards,
> > Steven G. Johnson [stevenj at math dot mit dot edu]
> > 
> > PS. If you do specify FFT functions, you don't want to make the same 
> > mistakes that Excel did, e.g. you don't want to artificially limit
> support 
> > 
> > to power-of-two sizes (a very inconvenient restriction for analysis of 
> > user-generated data).  Given a power-of-two FFT, you can implement
> support 
> > 
> > for arbitrary sizes with a couple dozen more lines of code (via 
> > Bluestein's algorithm), so there's really no excuse not to support any 
> > size with O(n log n) complexity.
> > 
> > This publicly archived list offers a means to provide input to the
> > OASIS Open Document Format for Office Applications (OpenDocument) TC.
> > 
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> 
> 
> Robert Weir replied:
> > Hi Steven,
> > 
> > This is an interesting idea.  In practice,do you think this makes sense
> as 
> > spreadsheet array functions which take an array and return an array?  Or
> > as an application comment that prompts the user to select an input and 
> > output range,along the lines of what Excel does with its Analysis
> Toolkit?
> > 
> > My experience is that most users are not even aware of array functions
> in 
> > Excel and how they use.  On the other hand, most users are not aware of 
> > FFT's either.
> > 
> > If done as a spreadsheet function, the FFT would obviously need to be 
> > recalculated every time one of the cells in the input range changed.  At
> > O(N log N) this isn't so bad.
> > 
> > Do you know if there are any ways to calculate an FFT with incremental 
> > changes?  So I calculate once in O(N log N) and then change only some of
> > the input cells, a small portion of them -- is there a much faster way
> of 
> > calculating the new results?
> > 
> > Note also that complex number support in spreadsheets is rather poor in 
> > general, and only a handful of functions work with them.
> > 
> > What functions would you recommend?  Maybe FFT() and the inverse IFFT().
> > Each one might have a parameter that specifies the normalization 
> > convention.  Maybe also a Power() function?  I could imagine in the time
> > domain CrossCorrelation()and AutoCorrelation() would be useful as well. 
> > These are interesting for fields beyond signal processing, including 
> > economics.
> > 
> > Note however that it is always a debatable point on whether a particular
> > feature request should be placed in the standard, or whether it is best 
> > done as an extension.  I'd be more encouraged to add features like this
> to 
> > the ODF standard if we had one or more vendors committed to supporting 
> > this feature in their product. 
> > 
> > If it were purely up to me, I'd love to create the ultimate 
> > engineer's/scientist's spreadsheet application.   There are a lot of 
> > things we could do, like adding units support, integration with R, etc. 
> > But we would need a set of volunteers dedicated to implementing this in 
> > OpenOffice or Gnumeric or someplace.
> > 
> > Regards,
> > 
> > -Rob
> > 
> > ___________________________
> > 
> > Rob Weir
> > Software Architect
> > Workplace, Portal and Collaboration Software
> > IBM Software Group
> > 
> > email: robert_weir, at.., us.ibm.com
> > phone: 1-978-399-7122
> > blog: http://www.robweir.com/blog/
> 
> 
> ----- End Original Message -----
> 
> -- 
> This publicly archived list offers a means to provide input to the
> OASIS Open Document Format for Office Applications (OpenDocument) TC.
> 
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
> 
> Subscribe: office-comment-subscribe@lists.oasis-open.org
> Unsubscribe: office-comment-unsubscribe@lists.oasis-open.org
> List help: office-comment-help@lists.oasis-open.org
> List archive: http://lists.oasis-open.org/archives/office-comment/
> Feedback License: http://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> Committee:
> http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=office

-- 
Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten 
Browser-Versionen downloaden: http://www.gmx.net/de/go/browser