What about negative testing? Usually, when working in the Agile world, users test at the minimum, and then extend test coverage after the feature has been built. But, when using a data-driven testing model, users can cover both the positive and negative testing scenarios all at once. This opens the door to testing the boundary and limits of the feature, testing the exceptions that may occur when exceeded.

Let's look at how this is done!

When we developed the JSON datasets, we really only talked about positive testing data. Now, we can easily extend the positive tests to negative testing by adding an exception message field to the JSON object, setting it to null for the positive tests, and then including the error for the negative test cases.

Here's an example:

{  
   "tc001_gmailLoginCreds":[  
      {  
         "rowID":"tc001_gmailLoginCreds.01",
         "description":"Gmail Login Test - Positive",
         "username":"[email protected]",
         "password":"password",
         "error":null
      },
      {  
         "rowID":"tc001_gmailLoginCreds.02",
         "description":"Gmail Login Test - Negative (Invalid Account)",
         "username":"[email protected]",
         "password":"password",
         "error":"Couldn't find your Google Account"
      },
      {  
         "rowID":"tc001_gmailLoginCreds.03",
         "description":"Gmail Login Test - Force Exception",
         "username":"[email protected]",
         "password":"password",
         "error":null
      }
   ]
}

In this example, there are three sets of data included: one for the positive test, one for the negative test, and one to force an exception to test the error handling of the login method.

In the positive and force exception tests, the error field was set to null. In the negative test, the actual error was included. That's all that was required for the dataset. Now, let's look at the test method:

@Test(groups={"LOGIN"},
      dataProvider="fetchData_JSON",
      dataProviderClass=JSONDataProvider.class,
      enabled=true)
public void tc001_gmailLoginCreds(String rowID,
                                  String description,
                                  JSONObject testData)
                                  throws Exception {

    String user, password;
    GmailLoginPO gmail = new GmailLoginPO();

    // test the login or credentials error
    user = testData.get("username").toString();
    password = testData.get("password").toString();

    if ( testData.get("error") == null ) {
        gmail.login(user, password);
        gmail.signOut();
    }

    else {
        gmail.login(user,
                    password,
                    testData.get("error").toString());
    }
}

In this example, the testData object was queried to see whether the error field was null, and if not, the positive test case was run. If it wasn't null, then an overloaded method was used to take the additional parameter and instead of throwing an exception, as would be done in the first login method, it will verify the error.

When testing boundary and limit conditions in test methods, users should pass in the first and last valid choices for a field that can be used, for instance, an integer value. Then, add in a negative test case to use a value beyond the limit of the feature, and verify an error is thrown.

So, it is fairly simple to design the test methods in a way that allows users to add positive, negative, boundary, and limit tests by simply varying the data. Keep in mind, when creating the page object methods, they should always include exception handling to catch an error that occurs during the test. Whether the common method allows you to test the error or an overloaded method is created for testing errors, is up to the user.