The need for speed... and how to do it.

["David A. Wheeler" <dwheeler@dwheeler.com>]

Massachusetts plans to implement its OpenDocument
policy by January 1, 2007.  By that time, I would
like to see not only the formula specification passed...
but I'd like to see it IMPLEMENTED in at least
two implementations and demonstrating interoperability.

Which means we need to develop a specification quickly,
AND have it be high-quality, AND have it implemented.
Is that impossible?  No, I think it's quite practical,
if we do things carefully; we just have to plan out
how we do things.

Here are the steps I think we should take that,
in my opinion, will make that possible:
* Have multiple implementors represented (well, that's
   obvious)... and we have that now.
* In general, start with "what is done now".  ODF spreadsheets
   generally use the OpenOffice.org syntax, for example, so let's
   start there... there's no strong technical reason to do otherwise.
   Standards groups should avoid creating new
   constructs, if we reasonably can.  HOWEVER, that does NOT mean
   that we should spec "whatever it happens to do"; the real
   challenge of a standards group is to specify what MUST happen,
   and NOTHING more, so that improvements aren't stifled.
   Balancing between under and over definition will be our challenge.
   I think everyone here knows the difference between an implementation
   and a standard... let's make sure others know it too.
* Define the spec with multiple "levels", so that implementors
   don't have to implement 500 functions just to pass at all.
   Most users don't need that; heck, a number would be happy
   with just + - * / and SUM.
* Include test cases for EVERY function, operator, syntactic
   requirement, and semantic requirement.  That way, implementors
   can tell us IMMEDIATELY when a requirement is wrong or
   unreasonable.  If it's reasonable, implementors can see
   EXACTLY what they need to change to meet the requirement, and
   they'll know RIGHT AWAY... so that they can fix it quickly.
* Include non-normative implementation notes to help
   implementors meet the requirements.  A few functions,
   if implemented "blindly", will get wrong answers on
   typical hardware floating point.  So let's warn them about that.
   (The OpenFormula spec warns about STDEV... if you do it
   naively, you'll often end up with square roots of negative numbers).
   That way, we'll have correct interoperable implementations, sooner.
   Other notes are fine too... because once they're written down,
   they'll keep us from constantly revisiting the issue.

I can't commit any implementor to actually IMPLEMENT this,
but I can hope that several will.  After all, we have a
number represented.  Ideally, some of you will implement the
spec WHILE we create it (as was done with OpenFormula)... because
implementing a spec is the BEST way to ensure that it's a good spec.

This is why I want to see the spec COMPLETED in 6 months.
If at least some implementors work to implement it WHILE we're
writing it, that will give them a little time to finish last-minute
work and release an upgrade, if they choose to. Result:
great spec, implemented and available.

A reminder to the outsiders who think that you can't
even use OpenDocument until this is all done: nonsense.
The .xls format has NO public specification at all, and
MOOX doesn't define formulas either.  But we aren't
satisfied with the status quo... we want something better.
And so, we're going to make something better happen.

Comments? Do these ideas make sense?

Thanks, everyone.

--- David A. Wheeler