Re: [xdi] Proposal for XDI Test Tool

On Wed, Apr 11, 2012 at 7:00 PM, Yuriy Zabrovarnyy <yzabrovarniy@gmail.com> wrote:

Thanks for questions, please find comments/remarks inlined.

On Wed, Apr 11, 2012 at 6:28 PM, Markus Sabadello <markus.sabadello@xdi.org> wrote:

Looks good to me, I also just put together a few notes about testing in XDI²: https://github.com/peacekeeper/xdi2/wiki/Testing

- Yes I think testing based on XDI messaging is the way to go, so the same test suite can be used against different servers.

Agreed :).

- I think OpenID tokens and link contracts should not be required for basic tests that simply check if graph operations work correctly.

Main idea was to use XDI Messaging to communicate with XDI Server. Since XDI Messaging requires tokens and link contracts XDI Server needs to support them too. I agree that maybe it looks not so granular but it should check request/response according to specs. For first shot it's not planned to support "special" cases (e.g. check operations without tokens or link contracts).

I don't think I agree that "XDI Messaging requires tokens and link contracts".

Of course OpenID Connect is one (and probably the preferred and best) way of authenticating, and of course there isn't much point in doing XDI without authentication/authorization.

But I think that theoretically there could be different mechanisms for doing this.

In my mind, the authentication/authorization process happens on a different "layer" than the core graph operations, and should therefore also be tested separately.

For example, let's say you send a $mod message to modify the value of a literal.

You could get an error message because you didn't send a valid token. But you could also get an error because the literal simply doesn't exist in the XDI graph.

These are 2 very different reasons why the message doesn't succeed.

I don't agree that OpenID Connect is a fundamental part of XDI Messaging, but I agree it fits well.

Well, so I guess it's okay if we have tokens in all test messages.

I'm not sure I quite understand the diagram on your page..

Does this mean a server that wants to be tested has to provide a "test.xml" file and an "executor" component to support the testing process?

No, you shouldn't provide any "tests.xml" or do any "extra" steps. The only thing you need is just specify configuration, click "Run" button and enjoy :).

Also, maybe it would be good to come up with some declarative way of doing testing, e.g. by putting XDI messages and the expected results into XML files, rather than writing specific testing code such as GetLiteralNode or DelNode.

Main idea was to do Test Tool as black box which tests XDI Server implementation based on messaging. I do not agree about declarative way of testing. Think about very simple use case, you need to check $del operation:
a) send request with $del operation
b) check whether it contains $true/$d!/(data:,<data>)
c) ATTENTION: maybe server just kidding you and send on all request $true statement!... to check it send $get operation and check whether node is really removed (in more precise case test tool before $del need to check with $get operation whether it exists or not.)

This could still be done in a declarative way, e.g. see Bill's message about pre conditions and post conditions.

But I guess we should do whatever is simplest and makes most sense :)

Anyway, I'm really excited about seeing and trying this test tool.

The UI design looks great :)

Markus

Yuriy Z

On Wed, Apr 11, 2012 at 4:00 PM, Michael Schwartz <mike@gluu.org> wrote:

Yuriy Zabrovarnyy has proposed a design for a test suite that would be developed as part of the OX project:

http://ox.gluu.org/doku.php?id=oxtestool:draft

- Mike

---------------------------------------------------

Michael Schwartz
Gluu
Founder / CEO
mike@gluu.org
+1 646-810-8761

---------------------------------------------------------------------
To unsubscribe, e-mail: xdi-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: xdi-help@lists.oasis-open.org

xdi message