Preconditions

In a few hours every day I have to test a Death Star-like application. I know there is nothing special about it, but this DS is a naughty one. It has billions of settings, and because it is a shared environment it often happens that something has been changed in the cms shortly before testing, and the QA does not know about it. So when we run the tests we always have hundreds of false positive results. (it is better than a false negative but it is annoying)

So on a quiet day I've decided to create a Precondition framework.
It has two main targets:

1. Make possible to add preconditions to every test we have, in a simple and common format.
2. Make possible to evaluate those preconditions before test run and skip the tests with a message if the precondition fails.

The framework have to be somewhat independent from the test framework (at least the first part), and could be independent from the actual level and type of testing (unit, integration, UI, API...).

So how does it is look like? Like a Thermozodium esakii.

The marking part is annotation based. You can annotate your test methods and your test classes with one @Preconditions which may have more (at least one) @Precondition. The @Precondition has the following parameters: id = the unique id (in class level) of this precondition, these can be used for linking. type = any value from the CondiotionType enum (to collect the precondition types in one place). args = the required arguments for the evaluation. skip = for skipping the evaluation for a given value of a given test parameter (see later). description = a short description which will be in the message of the SkipException.
You can mark your class fields with a @TestParam annotation which can be used for a simple value checking of the args value of its @Precondition (see examples below).

The evaluation is triggered by a TestNG listener class which implements the IInvokedMethodListener, in its beforeInvokation method gathers the required annotations and calls the required class for the evaluation and throws and exception if it did not met.

So you can define any type of conditions you only have to implement it in the *.conditions package (or elsewhere) and create an enum value for it, and add it to the beforeInvokation method of the listener (its a huge else-if... no better solution yet).

Here is a few annotation examples (using TestNG):

 public class MyGreatTest{  
   /**
   * The test field what should have a specified value
   */
   @TestParameter(conditionID="testParamCondition")  
   public String myTestParameter;  

   /**
   * The field gets its value here
   */
   @BeforeMethod  
   public void setUp(){  
     //... do something  
     myTestParameter = someMethod();  
   }  

   /**
   * A precondition which describes the check of a simple value of a parameter (of course the implementation has to be created before)
   */
   @Preconditions(@Precondition(type=ConditionTypes.parameterCheck,args={"YouCanDoIt"},id="testParamCondition",description="To check the field has the required value"))  
   @Test  
   public void firstTest(){ 

   }  

   /**
   * This example is one of our most common one. Based on the values of the parameters of the test method we check something in the CMS and if it has a NULL set skip the test. For some reason we can skip the evaluation if we define the skip argument. It contains the value and the position of the test parameter we want to check. So in this case, we define: if the first parameter (which is firstParam in this case) has the value 'Atlantis' skip the checking of this condition.
   */
   @Preconditions(@Precondition(type=ConditionType.checkSomethingInACMSByXpath,args={"/universe//planet//region[@='Aqua']"},skip={"1","Atlantis"},description="This test requires an Aqua type region, except when the first parameter has the value: Atlantis"))
   @Test
   public void secondTest(String firstParam, Integer secondParam){

   }

 }  

Why is it good for us? You can check the precondition in the before method and skip the test if it does not match... If you have 1000+ tests it is useful to part your tests from precondition checking, and you don't have to change the code on a lot of places, and you can easily skip the evaluation if you remove the Listener for the actual run.