Re: [ubl] Discussion of substitution groups

["Anthony B. Coates" <abcoates@londonmarketsystems.com>]

On Mon, 18 Jul 2005 16:08:50 +0100, CRAWFORD, Mark <MCRAWFORD@lmi.org>  
wrote:

> But if our schema follow our NDRs and are created from models as
> expressed in the spreadsheets, why do we need to use substitution groups
> at all?

Sorry I'm a little late in responding to this thread.  In the SSC group,  
we spent a lot of time, during the UBL 1.1 days, looking at pros and cons  
of different methods of managing minor versioning.  In particular, we were  
looking at ways of being able to provide some reasonable proof or  
guarantee that a new minor version of the UBL Schemas really is backwards  
compatible with the previous major/minor version.  Before we finished this  
discussion, it was decided that we would skip 1.1. and go straight to 2.0,  
so we never really brought this discussion back to the main group.

There are two key issues around backwards compatibility: how does UBL  
satisfy itself that new minor version Schemas are backwards compatible,  
and how does UBL convince its users that new minor version Schemas are  
backwards compatible?  The latter is the more difficult question.   
Although the UBL models (spreadsheets) are the source of the definitions,  
the Schemas are the deliverable that users really use, so it is at the  
Schema level that users will judge whether the Schemas appear to be  
backwards compatible.  The one advantage that the Scheme  
extension/restriction/redefine/substitution constructs have is that they  
make explicit how the new Schema version is derived from the previous one,  
so you only have to examine the changes to assess backwards  
compatibility.  If you just generate the full Schemas anew each time, you  
get into the difficult area of trying to do diffs between full Schemas,  
something that tools don't give you any help to do (none that I've seen,  
anyway).

Mind you, generating minor version Schemas as "deltas" on top of the  
previous major/minor version isn't without some problems either, since it  
suggests that the spreadsheets also need to express deltas, and that isn't  
entirely straightforwards either, particular if you have multiple minor  
versions, and need to track which fields were created or changed in which  
minor versions.

There other ways that people sometimes deal with the backwards  
compatibility issues, but the question is whether they are applicable to  
UBL.  For example, a comprehensive set of test examples can provided a  
sufficiently good regression test of backwards compatibility, good enough  
to satisfy users.  However, we don't have such a set of test examples  
right now.  I've also worked on a project where the Schemas were generated  
 from a model, and in that case I could trace backwards compatibility, at  
the level that was important to my users, by "compiling" the Schemas into  
Java and checking (by compilation) that each new minor version interface  
was a valid extension of the previous.  However, I don't see this method  
being suitable either.  I just wanted to point out that there are various  
mechanisms one can use to establish backwards compatibility; we need to  
find at least 1 that works for UBL.

It seems to me that UBL has a good set of design rules for creating new  
major versions of models/Schemas.  However, I do see that there is still a  
lot to do in order to provide the necessary support for minor versions,  
which is something we never had much opportunity to test until we  
(temporarily) started 1.1, and which we won't really have an opportunity  
to get to grips to until we do have to release a 2.1, 3.1, whatever.  My  
experience is that you always learn lessons from doing backwards  
compatible minor versions that you won't have foreseen when doing the  
initial major version.  It's the normal flow of things, but we have to be  
prepared for the idea that it might require some changes to the NDRs as  
they stand now.

Anyway, I hope this provides some context as to why these issues are  
coming up, and weren't necessarily resolved when the NDRs were done for  
1.0.  Ultimately, UBL needs to agree on a solution for backwards  
compatible minor versions that (a) satisfy's UBL's needs to have a  
workable process, and (b) provides sufficient confidence to its users that  
minor versions really are backwards compatible.

Cheers, Tony.
-- 
Anthony B. Coates
London Market Systems Limited
33 Throgmorton Street, London, EC2N 2BR, UK
http://www.londonmarketsystems.com/
mailto:abcoates@londonmarketsystems.com
Mobile/Cell: +44 (79) 0543 9026
[MDDL Editor (Market Data Definition Language), http://www.mddl.org/]
[FpML Arch WG Member (Financial Products Markup Language),  
http://www.fpml.org/]
-----------------------------------------------------------------------
This Email may contain confidential information and/or copyright material  
and is intended for the use of the addressee only.  Any unauthorised use  
may be unlawful. If you receive this Email by mistake please advise the  
sender immediately by using the reply  facility in your e-mail software.   
Email is not a secure method of communication and London Market Systems  
Limited cannot accept responsibility for the accuracy or completeness of  
this message or any attachment(s). Please examine this email for virus  
infection, for which London Market Systems Limited accepts no  
responsibility. If verification of this email is sought then please  
request a hard copy. Unless otherwise stated any views or opinions  
presented are solely those of the author and do not represent those of  
London Market Systems Limited.