Re: [emergency-msg] Compliance Tests

[Art Botterell <acb@incident.com>]

Rex -

That's great.  I'm going to be out of pocket for the next couple of 
weeks, so could you please help the group move ahead on this?  Thanks!

- Art


At 6:51 AM -0800 10/29/03, Rex Brooks wrote:
>Thanks, Allen,
>
>I think I understand, and as long as we stick to
>
>>"... it is only to assist implementors (not act as a
>>certification process), and ideas on how to craft this should start to
>>take form. It really is nothing complex. Take a look at what other
>>standards have done (http://www.w3.org/QA/Tools/#validators) -
>>especially those that have written validators for CSS, HTML/XHTML, or
>P3P...."
>
>I have no difficulty addressing such a suite of assistance tools, 
>guides, etc. and I think examples, tutorials, validators, and best 
>practices (like asking that there be a criteria for usefulness that 
>is clear even if unofficial). I'm up to my neck in a primer right 
>now, so I have at least some understanding of this thicket that may 
>help.
>
>Ciao,
>Rex
>At 11:31 PM -0500 10/28/03, R. Allen Wyke wrote:
>>Fair enough questions, so let me try to address...
>>
>>First, the request to deliver the compliance test suite is actually part
>>of the Charter for the EM TC
>>(http://www.oasis-open.org/committees/emergency/charter.php). It was to
>>be delivered in Q3 2003 to support our first standard's effort, which
>>ended up being CAP.
>>
>>Your next question seems to revolve around expectations, which ties to
>>your thoughts/comments around how it should be cited for use - ie: is it
>>some kind of certification authority. This is somewhat of a two part
>>question, so let me answer the later first.
>>
>>As for its intended use, per the Charter, it is "to assist
>>implementers". It is in no way suppose to be some authority on
>>certifying a CAP message as "legal" - its not a certification suite. Its
>>only a compliance suite. Hold this thought, and I will come back to it.
>>
>>When it comes to expectations, you guys (MSG SC) are really the ones
>>best equipped to answer that question - not the TC. In other words,
>>during the test and demo, what would have helped you? What kind of
>>guidelines, or maybe even some basic validation tools, would have
>>allowed you to see if your implementation worked as the spec intended?
>>Did you find SHOULDs or MAYs that "SHOULD" be used, but didn't have to?
>>For instance, I generated a "minimal" CAP message using XML Spy the
>>other day, it basically has nothing useful in it. So, while it was an
>>XML document that would have validated against the CAP schema, it
>>provided no useful information.
>>
>>Ok, now couple the experience you guys had during the demo/tests with
>>the fact that it is only to assist implementors (not act as a
>>certification process), and ideas on how to craft this should start to
>>take form. It really is nothing complex. Take a look at what other
>>standards have done (http://www.w3.org/QA/Tools/#validators) -
>>especially those that have written validators for CSS, HTML/XHTML, or
>>P3P.
>>
>>Just think about what you would have liked to have had during your
>>test/demo, and go from there. Again, this may be a document outlining a
>>testing process, or it could be a tool (like the ones at the W3C). To
>>get your minds stimulated, I would think the following ideas might be a
>>good place to start:
>>
>>1. XML Validation: nothing more than validating (upload) an instance doc
>>against the schema for starters. Over time if we identify transports to
>>support, this could have various "interfaces" for those transports.
>>Think of a Web Service you could send a CAP alert too, for example.
>>
>>2. GIS Display: this maybe something the GIS SC can help
>>recommend/provide, but basically a place you could upload a CAP alert
>>with GIS info and see it displayed on a map. Allow people to see how it
>>looked, and if it made sense.
>>
>>3. Variation of Art's EDIS Demo: something distributed via Java Web
>>Start that allows you to browse to a local CAP alert, rather than grab
>>off a server, and see it displayed in the application.
>>
>>Clearly you can go wild with variations of these, and do all kinds of
>>things that will be helpful for implementors. You guys are the CAP
>>experts, so I would recommend you pick a couple of areas you feel are
>>important and go from there.
>>
>>Hope this helps - Allen
>>
>>On Tue, 2003-10-28 at 15:14, Rex Brooks wrote:
>>>  Hi Allen,
>>>
>>>  I was asked by the message and notification subcommittee to draft
>>>  this message to you concerning the action item from the TC, as it was
>>>  referred to in the msg-sc mtg today, to design or consider what might
>>>  be required for a "compliance" test suite for CAP.
>>>
>>>  To make sure I understood what was meant by "compliance" I went to
>>>  the TC page and looked under Action Items, and did not find a
>>>  specific item of this name or effect. So I reviewed the minutes, and
>>>  also did not find a reference. There was a discussion about the
>>>  operational and/or "intent" tests that have already been done, but no
>>>  specific mention of what is or was meant by designing or developing
>>>  or discovering requirements for CAP "compliance" tests.
>>>
>>>  One of the reasons why I volunteered to take on the task of writing
>>>  this message is that I have some familiarity with a conformance test
>>>  suite that has been devised by the Web Services for Remote Portlets
>>>  TC. So I, at least, know something, as little as it is, about this
>>>  particular topic.
>>>
>>>  No one was able to really narrow down what was expected for this test
>>>  suite. I mentioned that WSRP tested "MUST" assertions In the WSRP TC
>>>  a test suite that has been developed by IBM based on the work of the
>>>  Conformance Subcommittee, which is its own entity, rather than a part
>>>  of another subcommittee.
>>>
>>>  That's about the extent of what I know, outside of some of the ins
>>>  and outs of what is tested for. However, as far as I can tell all it
>>>  tests for is if an application conforms to the spec. It does not
>>>  "certify" that an application is compliant or conformant. So, insofar
>>>  as testing at all is concerned, it is not intended to be cited for
>>>  public purposes, at least as far as I know.
>>>
>>>  My sense of the consensus of the Messages and Notifications SC is
>>>  that we do not have enough of an idea of what is needed or wanted in
>>>  a compliance test suite. So, we suggest that asking Karl might be the
>>>  wisest course of action for a number of reasons.
>>>
>>>  Not least of these reasons is that there may well be OASIS-wide
>>>  policy that bears on the issue or liability concerns over the notion
>>>  that a TC might offer formal or informal testing, and what purposes
>>>  such testing would serve.
>>>
>>>  So that is what the Messages and Notifications Subcommittee requests:
>>>  Formal guidance from the TC and OASIS on what is needed or requested
>>>  in a suite of tests for compliance with CAP. What should be tested,
>>>  and to what degree.
>>>
>>>  Thanks,
>>>  Rex Brooks
>>--
>>R. Allen Wyke
>>Chair, OASIS Emergency Management Technical Committee
>>http://www.oasis-open.org/committees/emergency
>
>
>--
>Rex Brooks
>GeoAddress: 1361-A Addison, Berkeley, CA, 94702 USA, Earth
>W3Address: http://www.starbourne.com
>Email: rexb@starbourne.com
>Tel: 510-849-2309
>Fax: By Request
>
>To unsubscribe from this mailing list (and be removed from the 
>roster of the OASIS TC), go to 
>http://www.oasis-open.org/apps/org/workgroup/emergency-msg/members/leave_workgroup.php.