[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [soa-rm-ra] Comments on Section 5.4 SOA Testing Model
I am with Frank. Its either cover a set of cases or take it out. By simplifying this we will do disservice to the document From: Francis McCabe
[mailto:fmccabe@gmail.com] Well, in that case, I would vote the testing section off the
island. Personally, I think that that would be a shame; because testing is not
so obvious; and software engineers are not that good at testing. On Nov 10, 2010, at 7:33 AM, Rex Brooks wrote:
I respectfully disagree that we should also look at the
deeper issues. I would prefer saying that Service Mocks are one kind of
testing, full scale test harnesses are another and there are issues involved
with accommodating unanticipated users and versioning that need to be
considered. I would leave it at that and move on. What we *should* be doing in an architectural description is
identifying the key concepts and relationships - so far as they relate to
testing in a SOA ecosystem. What we *should not* be doing is mandating best
practice. So, in so far as mock service is an important concept in
testing, we should identify it. Personally, I think we should also look at the deeper issues
of testing; much as Boris has alluded to. For example, if Amazon gives a
consumer $100 to spend on buying and selling stuff on its site and to report on
the experience; that too is testing. In addition, we *should* do this in a style that promotes
rigor and clarity. IMO, this is where we fall down currently. On Nov 10, 2010, at 6:13 AM, Lublinsky, Boris wrote:
Gents, Granted, service testing is extremely important, but
unfortunately there is no cut and dry answer to this. When talking about
testing there is a huge difference between internally and externally owned
services and between a functional role of a software – service provider vs.
service consumer. Mocks are not the answer to all problems, but they can be
extremely useful in the cases where service consumers are either created in
parallel with service providers or service providers are external and usage
negotiations are in progress. On another hand, a service designer/developer rarely knows
upfront all possible consumers of the service, so he has to test for his SLA
with a specialized test driver, simulating advertized load and validating
advertised SLAs. So technically the approach here is using different tools and
techniques raging from load runners to mocks to completely proprietary
approaches. The hard issues here are: · Onboarding of new non
anticipated service consumers · Service versioning in the
presence of significant amount of consumers So I would be very careful advocating a single solution here.
Its more of the case of “Whatever works”. May be the right approach is to
describe a set of use cases for testing and allow readers to decide on their
specific approaches From: Ken Laskey [mailto:klaskey@mitre.org] I understand and am sympathetic to the point being made.
There is a strong sense about mocks for people with a traditional testing
background, and I felt the important points for SOA were - SOA testing as a transition
rather than a step change - Monitoring in the SOA
ecosystem is integral to continuous testing in an environment with
unanticipated (but authorized) users and unanticipated (but justified) uses. If there are specific suggestions, I’m willing to work with
folks offline to improve. Ken --------------------------------------------------------------------------- Dr. Kenneth Laskey MITRE Corporation, M/S
H305
phone: 703-983-7934 7515 Colshire
Drive
fax: 703-983-1379 McLean VA 22102-7508 From: mpoulin@usa.com [mailto:mpoulin@usa.com]
Using Service Mocks is oversimplification
leading to majority of mistakes in SOA implementation but it is very convenient
for PM and developers because they run Unit-tests. Many (including myself)
wrote about this from personal practical experience (for example, iTKO,
Parasoft, etc.). The SOA requirement to the service testing is: the test
environment must include all engaged services (the services, that the tested service
interacts and all services the engaged services interact in chain and down to
the final resources); only in this way we can test whether new service
functionality works properly. This requirement also based on the assumption
that all services are fully independent and may be under different authorities.
Testing environment for SOA is not cheap. Service Mocks (as well as only-interface
testing) for SOA services may be allowed only for the developer's Unit-testing,
which must be followed by intensive integration and regression testing. If you
want, I can send you a chapter from my book dedicated to this topic. - Michael -----Original Message----- I am grappling with the idea of a testing model being foundational to a reference architecture. For me, a test model is more closely related to a type of realization of a reference architecture. There are a lot of good relationships made between IEEE-829 and SOA Testing. c In the real world, it is often the case that test access to a service uses the same service as production access to the service. This is mentioned in the beginning of section 5.4, I would emphasize this again in relation to service mocks. Danny --- On Tue, 11/9/10, Ken Laskey <klaskey@mitre.org> wrote: > From: Ken Laskey <klaskey@mitre.org> > Subject: RE: [soa-rm-ra] services in the ecosystem > To: "'Francis McCabe'" <fmccabe@gmail.com> > Cc: soa-rm-ra@lists.oasis-open.org > Date: Tuesday, November 9, 2010, 5:11 PM > The definition is in the RM. > The interpretation of the RM definition for > extension to the ecosystem is here. > > Ken > > --------------------------------------------------------------------------- > Dr. Kenneth Laskey > MITRE Corporation, M/S H305 > phone: 703-983-7934 > 7515 Colshire Drive > > fax: > 703-983-1379 > McLean VA 22102-7508 > > > -----Original Message----- > From: Francis McCabe [mailto:fmccabe@gmail.com] > > Sent: Tuesday, November 09, 2010 7:40 PM > To: Laskey, Ken > Cc: soa-rm-ra@lists.oasis-open.org > Subject: Re: [soa-rm-ra] services in the ecosystem > > I still do not see a definition of service here > On Nov 9, 2010, at 2:44 PM, Ken Laskey wrote: > > > Boris, Michael, and I (with some input from Jeff) have > been working with > > wording from the RM and extending it to better cover > the concept of > services > > in the ecosystem. The attached is the result of > that collaboration. We > > believe this responds to the need to ground the term > service in the > present > > work without introducing yet another definition to the > already abundant > > confusion. Given we approached this task with > different concerns, I am > > pleased that we came to a satisfactory agreement and > hope the rest of you > > will agree. > > > > > > > > On behalf of those who contributed to this, > > > > > > > > Ken > > > > > > > > > --------------------------------------------------------------------------- > > > > Dr. Kenneth Laskey > > > > MITRE Corporation, M/S H305 > phone: 703-983-7934 > > > > 7515 Colshire Drive > > fax: > > 703-983-1379 > > > > McLean VA 22102-7508 > > > > > > > > <service in the ecosystem 20101109PM.doc> > > --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php The
information contained in this communication may be CONFIDENTIAL and is intended
only for the use of the recipient(s) named above. If you are not the intended
recipient, you are hereby notified that any dissemination, distribution, or
copying of this communication, or any of its contents, is strictly prohibited.
If you have received this communication in error, please notify the sender and
delete/destroy the original message and any copy of it from your computer or
paper files. The information contained in this communication may be CONFIDENTIAL and is intended only for the use of the recipient(s) named above. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please notify the sender and delete/destroy the original message and any copy of it from your computer or paper files. |
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]