Subject: Re: [security-services] SAML Testing Referral Program Proposalfor SSTC Review & 30 Day Feedback - Response to SSTC Feedback
I talked to some colleagues here at Novell and we have mixed thoughts about the conflict of interest questions. IMHO, If there existed 3 or four different referral providers, we (SSTC) wouldn't care whether one is also a vendor. Let implementors worry about conflict of interest and NDA's. In any case this should be an OASIS question and not a TC question.
I think our (SSTC) emphasis should be on specifying what the tests should be. That way if my implementation passes Ping's tests, I should also pass IEEE-Liberty tests. Prateek called for some 'champions' to come forward and help create these tests: http://lists.oasis-open.org/archives/security-services/200503/msg00055.html.
OK, ping just stepped up and gave us some tests - I think. Is ping letting us use these tests, or did they just want feedback? My concern is that no on has commented on the tests (me included - because I thought they just wanted feedback). Can we approve a referral provider without specifying what they should test?
That said, we should actively seek additional independent third parties to also do conformance testing. I'm thinking of IEEE-Liberty, GSA, and the Open-Group. I mention the Open-group because they hosted a SAML plugfest along with their LDAP certification event last year: http://lists.oasis-open.org/archives/security-services/200408/msg00198.html
Then once they are on board, lets have ping/opengroup/gsa/ieee-liberty hammer out the exact conformance tests details.
- Cameron Morris
>>>Greg Whitehead <firstname.lastname@example.org> 04/24/05 3:58 am >>>
I haven't seen an answer to Tony's question about the process going
forward. If you've replied privately, would you please post the answer
to the list?
Trustgenix has been a strong supporter of SAML interoperability testing
in both Liberty and Oasis and of independent certification programs,
such as the ones run by IEEE for Liberty and by the GSA for the US
Government. However, we continue to see a fundamental problem with a
vendor of SAML products running a certification program for other
vendors of SAML products (their competitors). I don't know of any other
industry that operates this way.
I finally got a chance to read through your response last night and
here are some initial comments (by number from your response):
1) You say that Oasis defines the test suite and that changes can't be
made without a vote, but in the general background info on PingDeploy
it is made clear that it exists independently of Oasis and is owned and
managed by Ping. I don't understand how both can be true. How does
Oasis know that PingDeploy implements the test suite specified by
Oasis, or that it does not favor some implementations over others?
2) The complexity of the attached "Privacy Directive" just reinforces
the fact that all parties acknowledge a fundamental conflict of
interest in having a vendor of SAML products run the SAML certification
program. It raises many more questions than it answers. How can we be
sure that the Privacy Directive is sufficient or can even be
3) It's that Ping, a vendor of SAML products, would be selected to run
an Oasis branded SAML certification program that is the problem. As
noted in (2), the "Privacy Directive" raises more questions that it
6) I don't understand this. If this is not an Oasis program, why is
Oasis involved at all?
7) This seems like something that should be corrected in the CURRENT
program, not left to future programs.