NIEM & EDXL-RIM; WAS: RE: [emergency] EDXL HAVE and NIEM 2.1dictionary alignment?

[Rex Brooks <rexb@starbourne.com>]

Thanks Gary,

I've been waiting  for this connection to be made 
without me beating the drum about it.

We have to be very careful in the RIM to ensure 
that we do not create more problems than we solve 
when we attempt to make this alignment.

We are starting with the DE and the ValueListURN 
for several very basic reasons. Among those 
reasons is the fact that the DE gives us the 
functional container element for the various EDXL 
message payloads. It is important, IMO, to start 
with the DE because it came about as a response 
to the first major problem we faced after working 
CAP.

This response came about when we understood that 
one of the key rationales underlying CAP, e.g. 
that it is a transport-independent format for 
messages, was a two-way door enabling any medium 
but not specifying how to distribute those 
messages in detail.

It turned out that the DE then became the wrapper 
or header element of any EDXL payload message, 
thus setting the stage for the family of 
specifications in such a way that a de facto 
Service-Oriented Architecture was established.

Having a distribution specification that uses a 
modular-managed-list approach, which reuses those 
lists and needs a registry system of systems and 
network of networks to do this, is as close to a 
textbook example of SOA as we are likely to find.

The RIM should make this SOA explicit. NIEM is 
also a de facto SOA, but because it has not yet 
been strongly framed in this context, we have to 
be careful not to fall into the trap of pushing 
the issue of establishing an underlying 
architecture off because it is not currently 
reflected in NIEM as a major component concept, 
at least not as a fundamental end-to-end 
principle.

More importantly, we should avoid formalizing a 
framework (architecture) that is not explicitly 
SOA in order to avoid any future conflict as SOA 
is adopted by the marketplace as the dominant IT 
architecture for the next few decades.

In the upcoming NIEM Training Event, with which 
the EM Adoption TC will be colocating the 
"Emergency Interoperability Summit" there are 
only four presentations that deal with SOA 
explicitly unless you count the one on the 
Justice Reference Architecture (JRA) which is 
based on the OASIS SOA Reference Model. However, 
I expect to see that expand over the next several 
years because:
1. It builds on an existing base in the Federal 
Enterprise Architecture (FEA); and,
2. It's just plain sensible and has a lot of 
momentum going for it, despite the industry 
scramble to "own" the SOA mantle by hyping a 
bunch of short-lived acronyms and buzz words.

Cheers,
Rex

At 7:15 AM -0400 7/15/09, Gary Ham wrote:
>David,
>
>My understanding of the IEPD process is that a 
>wantlist is required only for brand new 
>elements.  Reuse of an approved external 
>standard does not require a wantlist. In fact it 
>would be confusing because it would appear that 
>we are submitting individual HAVE elements for 
>inclusion in the NIEM vice HAVE as a whole. The 
>wantlist is for LOCAL data structures that have 
>not been standardized and need to be. We do not 
>have to create a wantlist for the HAVE standard. 
> Just get it approved as an external standard 
>and define a NIEM adapter.
>
>You may be confused because the subschema 
>generator uses the same "wantlist" format for 
>storing the items you need out of NIEM prior to 
>generating the one ore more subset schemas that 
>you build in order back the specific (separately 
>name spaced) reference schema that you use for 
>your exchange schema.  But, you do not need to 
>submit a "wantlist" separately for elements that 
>the subschema generator can build for you. Just 
>for the new stuff. And approved external 
>standards are not new stuff. In fact, the HAVE 
>schema with its adapter is actually just another 
>subset schema as far as the IEPD process goes.
>
>The larger question is alignment of our RIM with 
>NIEM, which is a RIM of its own.  We may need to 
>think about something there, because there could 
>be value in aligning basic constructs. But my 
>warning is simple:  "there be monsters 
>lurking!!!!"  It is not simple at all.
>
>Respectfully,
>
>Gary
>
>
>From: David RR Webber (XML) [mailto:david@drrw.info]
>Sent: Tuesday, July 14, 2009 10:28 PM
>To: Gary Ham
>Cc: emergency@lists.oasis-open.org; 'Timothy Grapes'; 'Lee Tincher'
>Subject: RE: [emergency] EDXL HAVE and NIEM 2.1 dictionary alignment?
>
>Gary,
>
>Oh boy - this got wildly overly complex!!!
>
>Can we have a time out please?!?
>
>The wantlist.xml is a simple construct used by 
>the online NIEM SSGT.  It is REQUIRED as part of 
>an IEPD submission for those folks who know and 
>need IEPDs.  They are an absolutely SOB to 
>create manually online - but having CAM create 
>one of these for you in 30 seconds by pushing a 
>button - is likely to make adults breakdown, 
>laugh and cry.  Feels like cheating.
>
>Once you have a Wantlist.xml done for you its 
>easy to then delete and tweak as desired.  Hence 
>the supplied base EDXL HAVE wantlist.xml 
>uploaded earlier.  That's probably saving people 
>days of work manually.
>
>Now the magic is - once you have the 
>wantlist.xml - you can have the NIEM SSGT create 
>a subset schema for you.  Of course in the 
>context of OASIS EDXL this is irrelevant - 
>because you are not using it.  However
>developers of NIEM exchange schemas - normally need it.
>
>Again - we're back to what is formally required 
>for a valid IEPD.  A wantlist.xml, and 
>crossreference schema, an exchange schema and a 
>XML example instance, and you're business rules 
>documentation.
>
>So the good news is we can create all of these 
>for EDXL HAVE - and that is really important for 
>federal and state implementors who will need 
>these things for their project documentation to 
>justify what they are doing.
>
>Simple - nothing more than that.
>
>Thanks, DW
>
>p.s. If you want to try out the SSGT with the 
>wantlist.xml - go to this link - click on the 
>top right "Options" - and then upload it.
>
><http://niem.gtri.gatech.edu/niemtools/ssgt/SSGT-LoadWantList.iepd>http://niem.gtri.gatech.edu/niemtools/ssgt/SSGT-LoadWantList.iepd
>
>
>
>
>
>
>Thanks, DW
>
>-------- Original Message --------
>Subject: RE: [emergency] EDXL HAVE and NIEM 2.1 dictionary alignment?
>From: "Gary Ham" <hamgva@cox.net>
>Date: Tue, July 14, 2009 6:14 pm
>To: "'Lee Tincher'" <ltincher@evotecinc.com>, "'David RR Webber (XML)'"
><david@drrw.info>
>Cc: <emergency@lists.oasis-open.org>, "'Timothy Grapes'"
><tgrapes@evotecinc.com>
>
>Lee is correct, but did not fully explain the 
>reasoning in my mind. The NIEM NDR recognizes 
>that differences in context are inevitable, and 
>even valuable. These differences in context are 
>encapsulated in different name spaces and even 
>different standards organizations.  This 
>encapsulation is very important because it 
>facilitates maintenance and change as standards 
>are updated and improved (or even abandoned for 
>better alternatives).  Enforced integration 
>introduces rigidity and mutual dependencies to 
>the point that brittle structures are created 
>that destroy growth and innovation.  The point 
>is that we need to recognize each other, and 
>make use of each others capabilities through 
>mapping and combined (aggregated) exchange 
>schemas, but not try to combine incompatible 
>reference schemas.  The combined reference 
>schema is a path that has been tried over and 
>over again in one form or another, always with 
>eventual failure.  The NIEM federated approach 
>(using adapters) is a breath of fresh air, 
>because it allows encapsulation to work.
>
>So, if you are indeed trying to create a 
>wantlist from a defined context to be inserted 
>in the NIEM context, you are essentially 
>following the same road as a programmer who does 
>a code cut-and-paste from one code module to 
>another and then faces the difficult task of 
>maintaining the same logic in both modules.  It 
>is a daunting task, particularly because NIEM 
>naming rules WILL require you to modify the 
>items in your wantlist to some extent.   So now 
>you will have to maintain two almost the same, 
>but slightly different structures, in two 
>different maintenance schedules (NIEM and OASIS) 
>and try to keep them consistent. OUCH!
>
>So, lets use NIEM's excellent encapsulation 
>concepts and only reuse data structures that are 
>exactly the same in meaning AND structure. 
> According to the NIEM coursework and the NIEM 
>Naming and Design Rules, what I am advocating is 
>indeed the proper NIEM way, which is why I 
>support NIEM.
>
>Respectfully,
>
>Gary A. Ham
><http://grandpaham.com>http://grandpaham.com
>703-899-6241
>Grandpa can do IT!
>
>
>Gary A. Ham
><http://grandpaham.com>http://grandpaham.com
>703-899-6241
>Grandpa can do IT!
>
>
>From: Lee Tincher 
>[<mailto:ltincher@evotecinc.com>mailto:ltincher@evotecinc.com]
>Sent: Tuesday, July 14, 2009 5:19 PM
>To: 'David RR Webber (XML)'; 'Gary Ham'
>Cc: emergency@lists.oasis-open.org; 'Timothy Grapes'
>Subject: RE: [emergency] EDXL HAVE and NIEM 2.1 dictionary alignment?
>
>Actually - Gary is supporting the stance I have 
>been trying to get across - we do not need a 
>wantlist - We created a HAVE adapter and 
>submitted it to NIEM.  That approach supports 
>federation of namespacesŠre-creating HAVE from a 
>wantlist does not, it promotes integration, 
>which will not work as the standards would not 
>be able to exchange between each otherŠ.
>
>Thanks,
>Lee
>
>"I was wondering why that Frisbee was getting bigger - then it hit me."
>
>From: David RR Webber (XML) [<mailto:david@drrw.info>mailto:david@drrw.info]
>Sent: Tuesday, July 14, 2009 4:53 PM
>To: Gary Ham
>Cc: emergency@lists.oasis-open.org; 'Lee Tincher'; 'Timothy Grapes'
>Subject: RE: [emergency] EDXL HAVE and NIEM 2.1 dictionary alignment?
>
>Gary,
>
>Good thoughts.
>
>Attached is a revised wantlist.xml for EDXL HAVE / NIEM
>
>I found on closer inspection that there were 
>things the earlier iteration from the weekend 
>missed.
>
>Plus I've attached a copy of the EDXL-dictionary.xml - for all domains used.
>
>You can drag and drop that into Excel and it 
>opens as a spreadsheet - and then filter on 
>namespace add desired.
>
>Enjoy, DW
>
>-------- Original Message --------
>Subject: RE: [emergency] EDXL HAVE and NIEM 2.1 dictionary alignment?
>From: "Gary Ham" <hamgva@cox.net>
>Date: Tue, July 14, 2009 4:19 pm
>To: "'David RR Webber (XML)'" <david@drrw.info>, "'Timothy Grapes'"
><tgrapes@evotecinc.com>
>Cc: <emergency@lists.oasis-open.org>, "'Lee Tincher'"
><ltincher@evotecinc.com>
>
>Folks,
>
>This thread happened to coincide with a rather 
>detailed study of NIEM that I have been doing 
>recently.  In fact, I just posted to my blog 
><>http://grandpaham.com a comment on what I like 
>most about NIEM and a warning not to force like 
>concepts from differing contexts into the same 
>structure. I am going to repeat the blog post 
>here (although if I wanted to drive traffic I 
>would make you go to my site):
>
>I have been studying the 
><http://www.niem.gov>National Information 
>Exchange Model (NIEM) fairly extensively over 
>the last few weeks. I have even read the NIEM 
>Naming and Design Rules document from beginning 
>to end. I will admit that I went into it with 
>something of a jaundiced view.  As a veteran 
>contributor to the DoD data model and an outside 
>observer of the GJXDM (recently), and a large 
>scale IBM model (a long time ago), I have real 
>reservations about the usability and 
>maintainability of any all-knowing, all-seeing 
>model.  I have, at least at this point, become a 
>believer in NIEM.  Why?   Because NIEM accepts 
>the notion that a federation between separately 
>name-spaced models makes sense, both within 
>NIEM, and with external standards defined 
>outside the heavy NIEM NDR discipline (or 
>defined with a different heavy discipline). 
>The notion of defining an Adapter for NIEM use 
>of other standards is a brilliant concept. 
>This, combined with the Information Exchange 
>Package Documentation (IEPD) methodology for 
>documenting the contextual use of data used in 
>exchanges has made me a fan.
>The problem with this "federation of standards" 
>concept is that it makes tools (and "auto-magic" 
>validation) harder to build.  As a result there 
>is a tendency to try and force all of the 
>standards back into the all-knowing, all-seeing 
>model. It is a seductive idea, but not a good 
>idea.  Let's look at a very simple example: EDXL 
>Resource Management uses the Customer 
>Information Quality (CIQ standard) for Person 
>Names. This allows internationalization for all 
>kinds of different Naming structures and for a 
>wide variety of Addressing schemes.  NIEM (as a 
>national model) is much more U.S. centric, 
>particularly in the use of PersonName tag. Both 
>CIQ and NIEM are appropriate in their respective 
>namespaces (and the NIEM NDR respects this fact 
>by allowing for the adapter wrapper for external 
>standards).   If we try to combine the two 
>standards by defining CIQ elements as NIEM 
>elements directly in order to make the subschema 
>generator work more easily, we blur important 
>distinctions that were developed for good reason.
>So, we need to use NIEM IEPD methods. They are 
>excellent. But we must resist the desire to 
>force single definitions for concepts that may 
>appear to be the same, but actually differ due 
>to the context in which they were defined.  In 
>other words, do not force a merger of conceptual 
>domains, unless they actually are the same. 
>NIEM lets us federate in the building of an 
>IEPD.   We should take advantage of that 
>capability.
>
>
>Gary A. Ham
><http://grandpaham.com>http://grandpaham.com
>703-899-6241
>Grandpa can do IT!
>
>
>


--
Rex Brooks
President, CEO
Starbourne Communications Design
GeoAddress: 1361-A Addison
Berkeley, CA 94702
Tel: 510-898-0670