Automated UI testing

While unit tests ensure that a significant amount of code is tested, it is specifically focused on testing the logic within the app, leaving the user interface largely untested. This is where UI testing comes into play. UI testing allows us to automate specific actions within the app's user interface to assert that they function properly—just like regular unit tests assert that business logic delivers the desired results or functionality.

Xamarin provides a very rich set of tools for performing automated UI tests—both locally and in the Xamarin Test Cloud. UI tests can be written in C# and NUnit using Xamarin's UITest framework or in Ruby using Xamarin's Calabash framework. For the purposes of this book, we will only focus on using the C# approach using Xamarin.UITest.

The Xamarin UITest framework

The UITest framework enables you to automate interactions with an app using C# and the NUnit testing suite. All interactions take place through an instance of IApp. The ConfigureApp class is used to create iOS and Android IApp instances. We will take a closer look at creating IApp instances with ConfigureApp in the next couple of sections.

At the time of writing this book, UITest only supports iOS and Android.

Common UITest methods

The UITest framework contains many APIs that can be used for interacting with an app's user interface. Here are some of the more commonly used methods and the ones we will specifically use to test the TripLog app in this chapter.

These methods are members of the IApp interface:

  • Screenshot(): This is used to take a screenshot of the current state of the app
  • Tap(): This is used to send a tap interaction to a specific element on the app's current screen
  • EnterText() and ClearText(): These are used to add and remove text from input elements such as Entry views in Xamarin.Forms
  • Query(): This is used to find elements on the app's current screen
  • Repl: This is used to interact in real-time with the app through the terminal using the UITest API
  • WaitForElement(): This is used to hold up the test until a specific element appears on the app's current screen within a specific timeout period

Many of the previous methods, including Query and WaitForElement, return an AppResult[] object that can be used to determine the results of the call. For example, if a Query call returns an empty result set, then we can assert that the element does not exist. We will see a couple uses of this in the next section when we start writing actual tests.

These methods are members of the AppQuery class and are typically used in the AppQuery Func parameters of the IApp methods mentioned previously:

  • Class(): This is used to find elements on the app's current screen based on their class type
  • Marked(): This is used to find elements on the app's current screen based on their text or identifier
  • Css(): This is used to perform CSS selector operations on the contents of a webview on the app's current screen

Creating a UITest project in Xamarin Studio

Our UI tests will run side-by-side the normal unit tests in our solution:

Create a new Xamarin UITest project within the Tests solution folder and name it TripLog.UITests:

Creating a UITest project in Xamarin Studio
Creating a UITest project in Xamarin Studio

By default, we are supplied with an initial file named Tests.cs. We will place our UI tests in this single Tests.cs file for now. In some cases, it might make sense to break your UI tests up into separate files. In addition to the Tests.cs class, we are also provided with a class named AppInitializer.cs, which is used by the Tests.cs class to create an IApp instance and start the app under the test.

Initializing the TripLog app for UITest

In order to start an app and interact with it via UITest, we must perform some initialization steps, which occur in the AppInitializer class. The AppInitializer class has a static method called StartApp, which is called by the Tests setup method to get an IApp instance. It can be either an iOSApp or an AndroidApp using the ConfigureApp class:

public class AppInitializer
{
    public static IApp StartApp (Platform platform)
    {
        if (platform == Platform.Android) {
            return ConfigureApp
                .Android
             // TODO: Update this path ...
             //.ApkFile ("../../../Droid/bin/Debug/TripLog.apk")
                .StartApp ();
        }

        return ConfigureApp
            .iOS
         // TODO: Update this path ...
         //.AppBundle ("../../../iOS/bin/iPhoneSimulator/Debug/TripLog.iOS.app")
            .StartApp ();
    }
}

Notice that the ApkFile and AppBundle methods are currently commented out. This is because it is not required to run the tests locally within the unit test pane in Xamarin Studio.

Testing views

We will create tests for specific scenarios within the app to assert that screens and elements within them appear when expected and to capture screenshots of each step of each scenario for visual inspection.

When testing the UI, we will use a combination of the Xamarin.UITest and NUnit frameworks. UITest allows us to query and interact with elements on the app's user interface, whereas NUnit allows us to apply assertions to the results of the UITest operations resulting in either a pass or fail of the test—similar to typical unit tests.

The first scenario to test in our TripLog app is signing in. The main steps in our sign in scenario are as follows:

  • Tap the Sign in with Facebook button on the first screen
  • The OAuth webview authentication screen appears with the Facebook sign in form
  • Enter valid Facebook credentials and tap on the log in button within the webview
  • The main screen of the app containing the TripLog entries list appears

In order to create a UI test for the sign in scenario, perform the following steps:

  1. First, create a new method in the Tests file named SignIn and decorate it with the [Test] attribute:
    [Test]
    public void SignIn ()
    { }
  2. Add a private method named PerformFacebookSignInSteps to the Tests class that will handle the test steps specific to the Facebook sign in process. By placing these test steps into a separate method, they are easily reused in other tests that will also require automating the login process before carrying out other steps within the UI:
    void PerformFacebookSignInSteps (string email,
        string password)
    {
        // Wait for sign in with Facebook button to appear
        app.WaitForElement (x => x.Text ("Sign in with Facebook"));
    
        app.Tap (x => x.Text ("Sign in with Facebook"));
    
        // Wait for Login button within Facebook oAuth webview to appear
        app.WaitForElement (x => x.WebView().Css ("[name=login]"));
    
        app.Screenshot ("Facebook oAuth login screen");
    
        // Enter text in the element within the webview with name="email"
        app.EnterText (x => x.WebView ().Css ("[name=email]"), email);
    
        // Enter text in the element within the webview with name="email"
        app.EnterText (x => x.WebView ().Css ("[name=pass]"), password);
    
        // Tap the button in the webview with name="login"
        app.Tap (x => x.WebView ().Css ("[name=login]"));
    }
  3. Next, update the SignIn test method to call the PerformFacebookSignInSteps method, and validate that the main screen appears after successfully signing in:
    [Test]
    public void SignIn ()
    {
        app.Screenshot ("Sign in screen");
    
        PerformFacebookSignInSteps ("<your-fb-email>", "<your-fb-password>");
    
        var mainScreen = app.WaitForElement (x => x.Marked ("TripLog").Class ("UINavigationBar"));
    
        app.Screenshot ("Main screen");
    
        // Assert main screen is shown
        Assert.IsTrue (mainScreen.Any (), "Main screen wasn't shown after signing in");
    }

We can also write UI tests for alternative scenarios such as when the user taps the Cancel button on the OAuth web view authentication screen:

[Test]
public void CancelSignIn ()
{
    app.Screenshot ("Sign in screen");

    app.Tap (x => x.Text ("Sign in with Facebook"));

    // Wait for Login button within Facebook oAuth webview to appear
    app.WaitForElement (x => x.WebView().Css ("[name=login]"));

    // According to REPL there are 3 elements marked as Cancel - the one at the second 2nd index is a UIButtonLabel which we need
    var cancel = app.Query (x => x.Marked ("Cancel").Index (2));

    // Assert there is a Cancel button the UINavigationBar
    Assert.IsTrue (cancel.Any ());

    app.Screenshot ("Authentication screen with Cancel button");

    app.Tap (x => x.Marked ("Cancel").Index (2));

    // Wait for sign in screen to appear
    var mainScreen = app.WaitForElement (x => x.Marked ("Sign in with Facebook"));

    // Assert sign in screen is shown after auth webview screen is canceled
    Assert.IsTrue (mainScreen.Any ());

    app.Screenshot ("Sign in screen after auth is canceled");
}

The second scenario that we can test is adding a new entry. The main steps in this scenario are as follows:

  • Tap the Sign in with Facebook button on the first screen
  • The OAuth webview authentication screen appears with the Facebook sign in form
  • Enter valid Facebook credentials and tap on the log in button within the webview
  • The main screen of the app containing the TripLog entries list appears
  • Tap the New button on the main screen
  • The new entry screen appears
  • Fill out the form on the new entry screen
  • Tap the Save button on the new entry screen
  • The main screen of the app containing the TripLog entries list appears

We will need to use the PerformFacebookSignInSteps method again to advance through the sign in steps:

[Test]
public void AddNewEntry ()
{
    app.Screenshot ("Sign in screen");

    PerformFacebookSignInSteps ("<your-fb-email>", "<your-fb-password>");

    // Wait for main screen to appear
    var mainScreen = app.WaitForElement (x => x.Marked ("TripLog").Class ("UINavigationBar"));

    app.Screenshot ("Main screen");

    // Assert main screen is shown
    Assert.IsTrue (mainScreen.Any (), "Main screen wasn't shown after signing in");

    app.Tap (x => x.Marked ("New"));

    // Wait for New Entry screen to appear
    var newEntryScreen = app.WaitForElement (x => x.Marked ("New Entry").Class ("UINavigationBar"));

    app.Screenshot ("New entry screen");

    // Assert New Entry screen is shown
    Assert.IsTrue (newEntryScreen.Any (), "New Entry screen was not shown after tapping New on main screen");

    // Enter title text into Title EntryCell (first EntryCell)
    app.EnterText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell").Index (0),
        "Test Entry");

    // Tap into Date EntryCell (fourth EntryCell)
    app.Tap (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell").Index (3));

    // Tap Done on the date picker
    app.Tap (x => x.Marked ("Done"));

    // Tap into Rating EntryCell (fifth EntryCell) and assert numeric keyboard
    app.Tap (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell").Index (4));

    var numericKeyboard = app.Query (x => x.Class("UIKBKeyView"));

    // Assert numeric keyboard is shown for Rating EntryCell - numeric keyboard has 12 UIKBKeyViews
    Assert.IsTrue (numericKeyboard.Count () == 12, "Rating EntryCell doesn't use numeric keyboard");

    // Clear default text in Rating EntryCell (fourth EntryCell)
    app.ClearText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell")
                        .Index (4)
                        .Descendant ("UITextField"));

    // Enter rating text into Rating EntryCell (fifth EntryCell)
    app.EnterText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell")
                        .Index (4)
                        .Descendant ("UITextField"),
        "3");

    // Enter notes text into Notes EntryCell (sixth EntryCell)
    app.EnterText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell")
        .Index (5)
        .Descendant ("UITextField"),
        "Test entry notes.");

    // Tap Save to exit the new entry screen
    app.Tap (x => x.Marked ("Save"));

    // Wait for main screen to appear
    mainScreen = app.WaitForElement (x => x.Marked ("TripLog").Class ("UINavigationBar"));

    // Assert main screen is shown after saving a new entry
    Assert.IsTrue (mainScreen.Any (), "Main screen was not shown after saving a new entry");
}

Notice that a common pattern used to validate screens and elements is to use the WaitForElement method in the UITest API along with the NUnit Assert.IsTrue method.

Note

There are a few references to iOS specifics throughout these tests. Use the private platform class variable to check the platform and handle accordingly for Android. Alternatively, if these tests will only be run for iOS, then remove the [TestFixture (Platform.Android)] class attribute; the tests will be locked to iOS only.

Running UI tests

In order to run UI tests, Xamarin Test Cloud Agent needs to be included in the app project. For Android, Test Cloud Agent is provided by the UITest framework, and so, it does not need to be manually added. For iOS, Test Cloud Agent needs to be manually added to the project either via NuGet or Xamarin component:

  1. Add a reference to the Xamarin.TestCloud.Agent NuGet package to the TripLog.iOS project.
    Running UI tests
  2. Update the FinishedLaunching method of the TripLog app AppDelegate class to start Test Cloud Agent. Again, this step is only for iOS app projects:
    public override bool FinishedLaunching (UIApplication app, NSDictionary options)
    {
        global::Xamarin.Forms.Forms.Init ();
    
        Xamarin.FormsMaps.Init ();
    
        #if ENABLE_TEST_CLOUD
        Xamarin.Calabash.Start ();
        #endif
    
        LoadApplication (new App (new TripLogPlatformModule()));
    
        return base.FinishedLaunching (app, options);
    }

Notice the use of the ENABLE_TEST_CLOUD compiler symbol in the #if directive surrounding the Calabash.Start method call. This ensures that Test Cloud Agent is only started under specific configurations. Be sure to include the ENABLE_TEST_CLOUD symbol in the Debug configuration only and not in the Release configuration.

Running UI tests

Note

In most cases, when creating a new iOS project, the Test Cloud Agent NuGet package will already be added to the project references and the Calabash.Start method call will already be included in the AppDelegate.

Running UI tests locally

UI tests can be run from the Unit Tests pane within Xamarin Studio, just like normal unit tests. However, you will notice that there is a Test Apps node within the TripLog.UITests section.

Running UI tests locally

You need to either add your app to the Test Apps node of the Unit Tests pane or specify the path to the app in the AppInitializer class mentioned earlier; otherwise, your tests will fail to run.

To add your apps to the Tests Apps node within the Unit Tests pane, simply right-click on the Test Apps item and click on Add App Project. This will bring up a dialog that allows you to select one or many UITest compatible projects that can be added.

Running UI tests locally

Select the project(s) that you want to add to the tests and click on OK. For the purpose of this chapter, we only need to select the TripLog.iOS app project since our tests are specific to iOS. You will now see the selected app listed under the Test Apps node in the Unit Tests pane.

If you don't see your iOS app project listed in the dialog when trying to add to the Test Apps node, ensure that you have added the Xamarin Test Cloud Agent NuGet package to your iOS project.

Once you have either added your app projects to the Unit Tests or referenced the app package within AppInitializer, simply run the UI tests within the Unit Tests pane, just as you do for normal unit tests.

When your UI tests start to run, the UITest framework will automatically deploy your app to the simulator, run the app, and step through each of the steps specified in the test methods. The results of the tests will appear in the Test Results pane in Xamarin Studio.

Enabling screenshots

By default, screenshots are not stored when UI tests are run locally. In order to store the screenshots captured during your tests, you must add the EnableLocalScreenshots method to the ConfigureApp method chain in the AppInitializer class:

return ConfigureApp
    .iOS
    .EnableLocalScreenshots ()
    .StartApp ();

Once enabled, the screenshot images will be stored in the bin output folder where the tests are run from. Beware that these images will be overwritten the next time the tests run. So, copy them out to a different location if they need to be stored for each local test run.

Running UI tests in Xamarin Test Cloud

Once you have run your UI tests locally and verified the steps within them, you can broaden the number of devices, screen sizes, and operating system versions that they run on by uploading them to Xamarin Test Cloud.

You can upload your UI tests to Xamarin Test Cloud directly from the Unit Tests pane in Xamarin Studio.

Running UI tests in Xamarin Test Cloud

Xamarin Studio will build and upload your app and UI tests to Test Cloud. Once it has finished uploading the app and tests to Test Cloud, it will automatically launch the Test Cloud interface in your browser and allow you to select the parameters for the test run including specific devices and operating system versions. Finally, it will validate the tests and run them on the selected devices, providing the test results at completion.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.226.185.196