[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [oic] The Hague expense sheet scenario, continued
GENERAL COMMENTS I think we should take David Marston's advice about this, and not get hung up on an aggregate score or counting. We should never use "score," only test outcomes. - <http://wiki.oasis-open.org/oic/TestingThoughts1/200_Conformance_Principles> - <http://wiki.oasis-open.org/oic/TestingThoughts1/300_Test_Suite> - <http://wiki.oasis-open.org/oic/TestingThoughts1/400_Test_Lab_Obligations> Also, I notice that Marston mentions test scenarios and we might also, but a test scenario and an application scenario are different (even though ability to support an application (or usage scenario, if you think the application is only the software) scenario might be the subject of a test scenario). I think we are stumbling over unqualified use of "scenario." I've become uncomfortable with "feature point." I am not sure that is commonly understood or obvious. I think you want a collection of predicate-like conditions for which the individual outcomes are "yes" (true), "no" (false), or "not done" (inapplicable). The last case is if prerequisites are not satisfied or it is not possible to verify the particular condition for some other reason. Maybe these are just "test(-point) outcomes," stated as predicates that can be determined to be satisfied (true) or not (not true). "Test point" in the sense of a specific measurement point in an electronic circuit is something I think of like that. In our case, the test point definition would include the threshhold condition. It appears that we are asking that an ODF document of a particular nature (more or less) be created, but the tester has to know how to interact with the application in a way that achieves that result. I guess we are going to learn how to specify what the outcome should be in a way where a test developer would have to identify how that outcome is achievable, or not, with their product. For a test suite to be developed, it seems that the test scenario here would have to be translated by the test lab into a specific test procedure that applied to the product under test. This seems to be something that Marston anticipates also. With regard to reporting the outcomes of the test, I would put the results down a left column because they make it easier to scan and don't depend on having a particular width to view a right-side column easily. I think they need to be two-level, so that blocks of conditions that depend on a particular stage having been reachable or even exercisable are together. I realize that this is experimental, and there is a lot we will be learning about this. This is what I noticed. - Dennis LOOKING AT THE EXAMPLE TEST SCENARIO I looked at the latest oic-expensesheet scenario and there are a large number of conditions in the scenario that apply well before the conditions under Feature Points come into play. I also think we need to look at how those are measurable. For example, how is it to be determined that the document conforms to ODF 1.1 (and that it is the intention to create an ODF 1.1 document)? Also, how are satisfaction of layout measures to be determined? I assume the document must be rendered (on a printer?) in this case. There are language and locale conditions that need to be established in the setup too. Finally, I notice a tremendous number of dependencies on nomenclature having to do with ODF conveyed-notions that may be reflected in an user interfaces in a variety of ways (including not at all). (For example, what exactly does "Apply 'oic-page' to current page" mean? What is the intended outcome. Likewise, "Create a new page style 'oic-page'" has a high requirement for tacit knowledge. I think experts in formats and office applications would be able to infer what is required without much difficulty, but we are depending on them getting that "right." We might want to look at what the desired outcome is and then allow a tester to determine what implementation feature there is for accomplishing that. (For example, test scenario step 1.1 assumes there is a UI provision for independently defining and naming page styles. That's a big item with tons of dependencies (A4, portrait, metric dimensions, and UI language and locality-specific behavior). -----Original Message----- From: Hanssens Bart [mailto:Bart.Hanssens@fedict.be] http://lists.oasis-open.org/archives/oic/200906/msg00019.html Sent: Thursday, June 04, 2009 06:08 To: oic@lists.oasis-open.org Subject: [oic] The Hague expense sheet scenario, continued Hi, As an experiment for the The Hague "expense sheet" application scenario, I've added a "feature point" section at the bottom, with checkboxes for the first part (creating a test document) and a simple javascript (might not work locally) to count the number of check boxes being checked. Now I do realize that this isn't an exact science, one can always argue about what to check and what not, the scenario itself can be flawed, there is no "weight" involved etc, so that's why I used the term "Feature Points" instead of "Score". Perhaps I should reformat the page, so you'd have part the scenario with the steps on, and the associated checks on the right (would be easier to read and check). What do you think ? Best regards, Bart -----Original Message----- From: workgroup_mailer@lists.oasis-open.org [mailto:workgroup_mailer@lists.oasis-open.org] http://lists.oasis-open.org/archives/oic/200906/msg00018.html Sent: donderdag 4 juni 2009 14:50 To: oic@lists.oasis-open.org Subject: [oic] Version Control Commit by bart.hanssens Author: bart.hanssens Date: 2009-06-04 08:50:03 -0400 (Thu, 04 Jun 2009) New Revision: 91 Web View: http://tools.oasis-open.org/version-control/browse/wsvn/oic/?rev=91&sc=1 Modified: AppScenarios/branches/TheHague/appscenarios/oic-expensesheet.html Log: - added "feature points" section --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]