OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

soa-rm-ra message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [soa-rm-ra] Comments on Section 5.4 SOA Testing Model


I respectfully disagree that we should also look at the deeper issues. I would prefer saying that Service Mocks are one kind of testing, full scale test harnesses are another and there are issues involved with accommodating unanticipated users and versioning that need to be considered. I would leave it at that and move on.

Cheers,
Rex

On 11/10/10 7:08 AM, Francis McCabe wrote:
BDA4672B-E679-4C2C-9C88-90A6D2641204@gmail.com" type="cite">What we *should* be doing in an architectural description is identifying the key concepts and relationships - so far as they relate to testing in a SOA ecosystem. What we *should not* be doing is mandating best practice.

So, in so far as mock service is an important concept in testing, we should identify it.

Personally, I think we should also look at the deeper issues of testing; much as Boris has alluded to. For example, if Amazon gives a consumer $100 to spend on buying and selling stuff on its site and to report on the experience; that too is testing. 

In addition, we *should* do this in a style that promotes rigor and clarity. IMO, this is where we fall down currently.

On Nov 10, 2010, at 6:13 AM, Lublinsky, Boris wrote:

Gents,
Granted, service testing is extremely important, but unfortunately there is no cut and dry answer to this. When talking about testing there is a huge difference between internally and externally owned services and between a functional role of a software – service provider vs. service consumer. Mocks are not the answer to all problems, but they can be extremely useful in the cases where service consumers are either created in parallel with service providers or service providers are external and usage negotiations are in progress.
On another hand, a service designer/developer rarely knows upfront all possible consumers of the service, so he has to test for his SLA with a specialized test driver, simulating advertized load and validating advertised SLAs.
So technically the approach here is using different tools and techniques raging from  load runners to mocks to completely proprietary approaches.
The hard issues here are:
·         Onboarding of new non anticipated service consumers
·         Service versioning in the presence of significant amount of consumers
So I would be very careful advocating a single solution here. Its more of the case of “Whatever works”. May be the right approach is to describe a set of use cases for testing and allow readers to decide on their specific approaches
 
From: Ken Laskey [mailto:klaskey@mitre.org] 
Sent: Wednesday, November 10, 2010 6:17 AM
To: mpoulin@usa.com; danny_thornton2@yahoo.com; soa-rm-ra@lists.oasis-open.org
Subject: RE: [soa-rm-ra] Comments on Section 5.4 SOA Testing Model
 
I understand and am sympathetic to the point being made.  There is a strong sense about mocks for people with a traditional testing background, and I felt the important points for SOA were
-          SOA testing as a transition rather than a step change
-          Monitoring in the SOA ecosystem is integral to continuous testing in an environment with unanticipated (but authorized) users and unanticipated (but justified) uses.
If there are specific suggestions, I’m willing to work with folks offline to improve.
 
Ken
 
---------------------------------------------------------------------------
Dr. Kenneth Laskey
MITRE Corporation, M/S H305              phone: 703-983-7934
7515 Colshire Drive                                    fax:        703-983-1379
McLean VA 22102-7508
 
From: mpoulin@usa.com [mailto:mpoulin@usa.com] 
Sent: Wednesday, November 10, 2010 5:07 AM
To: danny_thornton2@yahoo.com; soa-rm-ra@lists.oasis-open.org
Subject: Re: [soa-rm-ra] Comments on Section 5.4 SOA Testing Model
 

I fully support Danny in "I have a strong opposition to the implication that Service Mocks are required fortesting.
 
Using Service Mocks is oversimplification leading to majority of mistakes in SOA implementation but it is very convenient for PM and developers because they run Unit-tests. Many (including myself) wrote about this from personal practical experience (for example, iTKO, Parasoft, etc.). The SOA requirement to the service testing is: the test environment must include all engaged services (the services, that the tested service interacts and all services the engaged services interact in chain and down to the final resources); only in this way we can test whether new service functionality works properly. This requirement also based on the assumption that all services are fully independent and may be under different authorities. Testing environment for SOA is not cheap.

 

 Service Mocks (as well as only-interface testing) for SOA services may be allowed only for the developer's Unit-testing, which must be followed by intensive integration and regression testing. If you want, I can send you a chapter from my book dedicated to this topic.

 

- Michael

 

-----Original Message-----
From: Danny Thornton <danny_thornton2@yahoo.com>
To: soa-rm-ra@lists.oasis-open.org
Sent: Wed, Nov 10, 2010 6:10 am
Subject: [soa-rm-ra] Comments on Section 5.4 SOA Testing Model

I am grappling with the idea of a testing model being foundational to a 
reference architecture.  For me, a test model is more closely related to a type 
of realization of a reference architecture.
 
There are a lot of good relationships made between IEEE-829 and SOA Testing.  c  In the real world, it is often the case that test access to a service 
uses the same service as production access to the service.  This is mentioned in 
the beginning of section 5.4, I would emphasize this again in relation to 
service mocks.
 
Danny
 
--- On Tue, 11/9/10, Ken Laskey <klaskey@mitre.org> wrote:
 
> From: Ken Laskey <klaskey@mitre.org>
> Subject: RE: [soa-rm-ra] services in the ecosystem
> To: "'Francis McCabe'" <fmccabe@gmail.com>
> Cc: soa-rm-ra@lists.oasis-open.org
> Date: Tuesday, November 9, 2010, 5:11 PM
> The definition is in the RM. 
> The interpretation of the RM definition for
> extension to the ecosystem is here.
> 
> Ken
> 
> ---------------------------------------------------------------------------
> Dr. Kenneth Laskey
> MITRE Corporation, M/S H305       
>       phone: 703-983-7934
> 7515 Colshire Drive         
>                
>           fax:
> 703-983-1379
> McLean VA 22102-7508
> 
> 
> -----Original Message-----
> From: Francis McCabe [mailto:fmccabe@gmail.com]
> 
> Sent: Tuesday, November 09, 2010 7:40 PM
> To: Laskey, Ken
> Cc: soa-rm-ra@lists.oasis-open.org
> Subject: Re: [soa-rm-ra] services in the ecosystem
> 
> I still do not see a definition of service here
> On Nov 9, 2010, at 2:44 PM, Ken Laskey wrote:
> 
> > Boris, Michael, and I (with some input from Jeff) have
> been working with
> > wording from the RM and extending it to better cover
> the concept of
> services
> > in the ecosystem.  The attached is the result of
> that collaboration.  We
> > believe this responds to the need to ground the term
> service in the
> present
> > work without introducing yet another definition to the
> already abundant
> > confusion.  Given we approached this task with
> different concerns, I am
> > pleased that we came to a satisfactory agreement and
> hope the rest of you
> > will agree.
> > 
> > 
> > 
> > On behalf of those who contributed to this,
> > 
> > 
> > 
> > Ken
> > 
> > 
> > 
> >
> ---------------------------------------------------------------------------
> > 
> > Dr. Kenneth Laskey
> > 
> > MITRE Corporation, M/S H305       
>       phone: 703-983-7934
> > 
> > 7515 Colshire Drive         
>                
>           fax:
> > 703-983-1379
> > 
> > McLean VA 22102-7508
> > 
> > 
> > 
> > <service in the ecosystem 20101109PM.doc>
> 
> 
 
 
      
 
---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 


The information contained in this communication may be CONFIDENTIAL and is intended only for the use of the recipient(s) named above. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please notify the sender and delete/destroy the original message and any copy of it from your computer or paper files.




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]