EnergyInteropTC Action Item: propose wording for sec. 3.2.3

["Horst, Gale" <ghorst@epri.com>]

Team:

 

I noted this on the actions items that this was assigned to me  “propose wording for sec. 3.2.3”

 

I don’t recall volunteering … but went ahead and created the verbiage below.  We may want to change this a bit or state it differently.  But perhaps this text will provide a talking point on the verification and compliance section. 

 

- Gale

 

 

 

3.2.3   Verification and Compliance.

Verification and compliance testing goes beyond interoperability which is focused on getting products from various vendors working with each other as outlined. The focus during development is on correctness and finding and isolating problems of interoperability of one product connected to another product.   Following this, the focus is on the appropriate response of the device or system. Generally we could identify two broad phases of development:

1.      The first phase encompasses a number of issues surrounding the successful communication of the data.  Was the data received?  Was the information in the packets formatted correctly?  This may uncover areas where the specification was unclear, inconsistent, or missing information.

2.      Phase two focuses on whether or not the device or entity, that is the recipient of the data exchange, correctly responds and/or reports the actions taken.  This tends to be the compliance stage and could move toward a certification process development.

As part of the first phase, prior to development of formal testing procedures, it is common practice for a group of product developers to get together in a controlled forum to perform initial testing of interfaces. A term “plugfest” is often used for this initial group testing period. This is an opportunity for developers to come together and test their products against other implementations of the protocol being developed early in the process.   While still in final stages of development, products from different vendors can connect through the defined standards that both vendors have implemented.  This may be conducted in a somewhat unstructured manner providing an effective way to verify the specifications and interoperability early in development.  A key advantage is that a “plug fest” could be considered as part of the development process and also done early enough to also uncover potential issues with the implementation as well as the protocol specification. 

The scope of this technical committee is to produce a specification that is verifiable in order to enable the measurement of compliance.  While phase one can be done with a skeleton system representing the communications components, the second phase involves a load and measureable response as well as measurement of the response.  The OASIS EnergyInterop TC recognizes that this specification enables an opportunity for development of compliance testing and a potential for certification of the responding system or entity.  Due to the variability of the nature of the responding system or entity, compliance testing will be variable and potentially complex with the need to tailor testing to the specific application as well as for the intervals and accuracy needed.  The resources to build the infrastructure needed for the compliance testing could also be significant requiring specialized tools or access to the internals of the product under test.  As noted in the specific scope statements, this level of performance validation and verification is not in the scope of this work.  Once the messages have been sent and received, the compliance should be measureable via whatever means the involved entity utilizes for verification or billing and may be tied to a variety of contractual obligations.