Re: [xmile-comment] Draft Comments

[Scott Fortmann-Roe <scottfr@gmail.com>]

Hi Karim,

I have seen your response to the comments I raised and that you are moving forward to put this to a formal vote without making changes to the spec.

Briefly:

1) Your response is incomplete. You only address the high-level comments listed in the email, not the numerous line-by-line ones in the doc itself (linked to in the email). These additional comments raise numerous issues with the spec that should be addressed (in addition to other things like various typos that would be trivial to fix).

2) For the comments you do address your responses are often not sufficient. For example, releasing a spec for an _expression_ language without a formal grammar simply doesn't make sense. Defining a grammar really isn't that complex and I don't see how it could be omitted. Omitting details on simulation algorithms and order is also simply not accessible. These types of things are simply not items that can be delay to "version 2".

3) As I note in my email to the comment mailing list, there does not appear to actually be three actual conformant implementations of the spec (I'm not sure there is even 1). According to my understanding of OASIS procedures, this should not have been brought to a final proposed standard yet alone be brought up for a final approval vote. Given the issues I raise in my comments, I don't believe it would be possible to get three different conforming implementations simply based on the current spec. There are simply too many under-specifications and ambitious I have identified as is.

I think the issues are fixable and I hope you choose to address them. I will not be pushing this any further, but I wanted to make sure my comments and feedback were registered as it appears they may not have been.

-Scott

On Fri, Oct 23, 2015 at 9:38 AM, Karim Chichakly <kchichakly@iseesystems.com> wrote:

Hi Scott,

The XMILE Technical Committee has received your comments.  Thank you for taking the time.  We will review and consider your suggestions.

Best regards,

Karim Chichakly

Co-Chair, OASIS XMILE Technical Committee

On Sat, Oct 10, 2015 at 2:03 PM, Scott Fortmann-Roe <scottfr@gmail.com> wrote:

Thanks for to Karim and everyone else for their hard work on this standard! There is a lot to like and a shared spec is very important for the community.

However, as is I do not think this is ready to release.

Here are my item-by-item comments:

  https://dl.dropboxusercontent.com/u/94002/xmile-SFR%20Review.doc

Here are the main weaknesses of the current draft that I took away from my quick reading. Please let me know if I misunderstood anything.

1) The simulation details are underspecified. More specificity is needed there from precisely defining numerical diff-eq solivers to defining how behaviors like non-negative stocks or macro-filters interplay with the solution algorithms. The most important thing this spec can do is ensure simulation results are the same between programs, and as-is it doesn't do that.

2) The unit support is inadequate and needs to be beefed up. Namely, unit conversion should be directly supported.

3) A full _expression_ grammar needs to be provided in the doc.

4) I estimate 80 of the text is OPTIONAL. This is very dangerous as it means those sections aren't being fully evaluated and critiqued. There is a lot here that has no business being in an SD spec (e.g. transition animations) and the stuff that should be should be added later on as separate specs (e.g. the queue). In my opinion, everything that is OPTIONAL should be CUT from the v1 version of the spec, and added piecemeal in a later version or as additional specs. 

5) Related to (4) and (1), there is generally a lot of underspecification throughout especially in the optional sections. E.g. for "leak_integers" how is this applied is it rounded, do the deltas accumulate, etc? E.g. what is "flow_concept" it has one brief line describing it? Since this is OPTIONAL, it is very easy to let this pass as a reviewer and say it is not important, but in reality everything is and must be full specified if it is in the spec of cut.

6) Existing standards need to be used when possible. There are a number of cases of ad-Hoc custom rules are defined when existing standards have wide adoption that could be used instead. This greatly reduces the features in the spec and also makes it harder to implement. E.g. the spec has tags like <video> <image>, etc.. all could and should be replaced with a standard HTML field to render arbitrary contents. The styling attributes need to replaced completely with CSS. MathML or another standard should define the equation syntax tree. This would open up a lot more flexibility and also decrease the size of the spec itself.

7) In general, the display section has too many hard coded widgets and seems to be replicating an outdated (hypercard-esque) model. This both creates a burden for the vendor and also limits what they can implement. Everything is specified with absolute position (in pixels only! how do we handle retina displays?). This isn't compatable with much of modern UI design and development. Most of this should be cut.

Anyways, those are my two cents. I hope this email doesn't come across as too negative, I'm focusing on what needs to be improved rather than what works well.

Cheers, Scott