Chapter 14. Testing mobile apps using Xamarin UITest

This chapter covers

• What UI testing is
• Using Xamarin UITest to do UI testing
• Using the REPL
• Interacting with controls
• Asserting that the UI is correct

As we’ve built our apps, we’ve written a lot of unit tests to verify the model and view-model layers, but we’ve written nothing to test the view layer. Although this layer is small, there could still be issues that aren’t spotted until the app is out in the wild. Fortunately, we can write automated tests to help catch such bugs before users see them, causing them to uninstall our apps and move to a competitor’s offerings.

14.1. Introduction to UI testing

One of the great things about the MVVM design pattern is that it allows us to maximize the code in our cross-platform model and view-model layers. Not only have we written the bulk of our code just once, but we’ve managed to write unit tests for it, so we have some degree of confidence that the code works. These unit tests are great, but they don’t cover two important questions: have we used the correct controls on our view and bound them correctly, and does our app actually run on the device?

It’s great to have a property on a view model that’s bound to a text field so that the user can enter the name of a counter, but what if we accidentally used the wrong control, such as a label instead of a text box? Or maybe we used the right control but forgot to add the binding code? What if in our app we used a feature that was only added to the Android SDK in API 21, but our app manifest shows that our app will run on API 19 and higher? This is where UI testing comes in—it allows us to run our apps on emulators, simulators, and devices and write automated tests in code against it.

The concept behind UI testing is simple: you run your app and have something interact with it as if it were a user by using the user interface components (such as tapping buttons or entering text in text boxes), and you validate that everything is working by ensuring the app doesn’t crash and that the results of the user’s actions are shown on the UI as expected. This kind of testing was first used with desktop apps, where the aim was to make testing more reliable and cheaper—after all, humans are expensive, and they can get bored and make mistakes or miss problems after testing the same screen many, many times. Automated UI testing also allowed for better time usage, with tests being run overnight and developers discovering whether they’d broken anything the next morning.

For desktop apps, UI testing was reasonably simple—launch the app and test it, maybe testing on a few different screen sizes but always on one OS with maybe one or two different versions, because desktop OSs don’t change very often. With mobile, things have become more complicated. There are two major OSs that you’ll want to support with your cross-platform apps, and there are multiple versions of these OSs available. You also have different hardware with different screen sizes. On iOS this isn’t too bad—you only need to support a small number of OS versions (maybe just the current and previous one) and a small number of different devices. On Android, however, as you’ve already seen, it’s a mess with multiple OS versions in regular use, a huge range of screen sizes available, and worst of all, customizations to the OS from both the hardware manufacturers and carriers.

This is why UI testing is hugely important for mobile apps. There’s no way a human could test on a wide range of devices without needing a lot of time or lots of humans involved in the process (expensive) and without them all going mad as they install the app on yet another device and run the same test for the millionth time.

14.1.1. Writing UI tests using Xamarin UITest

You write UI tests in essentially the same way that you’d write a unit test: you decide what you want to test and then write some code to create the test. This code will use some kind of framework that can launch your app and interact with it as if it were a real user.

There are a number of different frameworks for testing, and table 14.1 lists some of these.

Table 14.1. UI testing frameworks

Framework	Platforms	Language	Description
Espresso	Android	Java	This is Google’s testing framework for Android apps, so it has deep integration with Android: https://google.github.io/android-testing-support-library/docs/espresso/.
XCTest	iOS	Objective-C/Swift	This is Apple’s UI testing framework, so it has deep integration with iOS: https://developer.apple.com/reference/xctest.
Appium	iOS/Android	Any (Java/C#/PHP/Ruby, and so on)	An open source cross-platform testing framework based on Selenium, a web UI testing framework: http://appium.io.
UITest	iOS/Android	C#/F#	This is Xamarin’s testing framework, which is heavily integrated into Visual Studio for Windows (Android) and Mac (iOS and Android): https://docs.microsoft.com/en-us/appcenter/test-cloud/uitest/?WT.mc_id=xamarininaction-book-jabenn.

In this book we’ll be focusing on Xamarin UITest because it’s very well integrated into Visual Studio. For testing Android apps, you can use either Windows or Mac, but for testing iOS apps you’ll need to use a Mac—iOS testing isn’t supported on Windows at the moment.

Xamarin UITest is based on a testing framework called Calabash that was written in Ruby and is fully open source and maintained by Xamarin. UITest is a layer on top of Calabash that allows you to write your tests in C# and run them using NUnit. These tests are written in the same way as unit tests using the arrange, act, assert pattern (shown in figure 14.1), with the arrange part launching the app and getting it to the relevant state for testing, the act part interacting with the UI as if it were a user, and the assert part querying the UI to ensure it’s in the correct state.

Figure 14.1. UI tests, like unit tests, follow the arrange, act, assert pattern.

14.1.2. Setting up your app for UI testing

In this chapter we’ll just be focusing on the Countr app, as there’s more to test there, so you can open the completed Countr solution from the previous chapter.

When you built the model layer in the app you added a new unit test project that tested both the model and view-model layers. For our UI tests, we’ll need a new project that will contain and run the UI tests.

Creating the UI test project

Add a new UITest project to the Countr solution: On Visual Studio for Windows, right-click the solution, select Add > New Project, and from the Add New Project dialog box select Visual C# > Cross-Platform on the left, and select UI Test App in the middle (figure 14.2). On Mac, right-click the solution, select Add > Add New Project, and select Multiplatform > Tests on the left and UI Test App in the middle, and then click Next. Name your project Countr.UITests and click OK (on Windows) or Create (on Mac).

Figure 14.2. Adding a new UITest project using Visual Studio for Mac (left) and Windows (right)

Once the test project has been added, it will install two NuGet packages that UITest needs: NUnit and Xamarin.UITest. It’s worth at this point updating the Xamarin.UITest NuGet package to the latest version, as they often push out bug fixes to ensure it works on the latest mobile OS versions. Do not, however, update NUnit. UITest will only work with NUnit 2, not NUnit 3, so if you update this package, your tests won’t work and you’ll need to remove the package and re-install NUnit 2.

The UI test project has two files autogenerated for you: AppInitializer.cs and Tests.cs:

• AppInitializer.cs— This is a static helper class with a single static method that’s used to start your app. UITest has an IApp interface that represents your running app, and this interface has methods on it for interacting with the UI elements in your app or, to a limited extent, the device hardware (for example, rotating the device). The StartApp method in the AppInitializer class returns an instance of IApp that your tests can use. This method uses a helper class from UITest called ConfigureApp to start the app, and this helper class has a fluent API that allows you to configure and run your app. The autogenerated code in the app initializer doesn’t do much to configure the app; it just specifies the app’s type (Android or iOS) based on the platform passed in to the method.
• Tests.cs— This file contains a UI test fixture that you can run. This test fixture class has a parameterized constructor that takes the platform to run the tests on as one of the values from the Platform enum, either Platform.iOS or Platform.Android. It also has two TestFixture attributes, one for each platform. This means that you really have two test fixtures—one Android and one iOS. This fixture has a setup method that uses AppInitializer to start the app before each test and a single test that calls the Screenshot method on the IApp instance, which is returned from the app initializer, to take a screenshot.

Setting up your Android apps for UI testing

By default, Xamarin Android apps are configured in debug builds to use the shared mono runtime (mono being the cross-platform version of .NET that Xamarin is based on). When you deploy your app to a device or emulator, it takes time to copy the code over, so anything that can make your app smaller reduces install time, which is good.

Xamarin Android apps use a mono runtime to provide the .NET Framework, and this is a large piece of the code bundled in your app. Rather than bundling it in, you can use a shared version that is installed separately for debug builds, making your app smaller. Unfortunately, when doing UI tests, you can’t use the shared runtime, so you have two options:

• Don’t use the shared runtime— You can turn off the shared mono runtime from the project properties. In Visual Studio for Windows, you’ll find it in the Android Options tab at the top of the Packaging page. On Mac, it’s on the Android Build tab at the top of the General page. Untick the Use Shared Mono Runtime box to turn this off, but be aware that this will increase your build times.
• Release builds— Release builds don’t have the shared mono runtime turned on. After all, when you build a release version, it’s usually for deployment, such as to the store, and your users won’t have the shared mono runtime installed. The downside to using a release build is that you need to grant your app permission to access the internet so that it can talk to UITest. This isn’t a problem if your app already accesses the internet, but if it doesn’t, you many not want to ask your users for this extra permission, as they might not want to grant it. If you want to use a release build, you can grant this permission in Visual Studio by opening the project properties, heading to the Android Manifest tab, and finding the Internet permission in the Required Permissions list and ticking it. On Mac, double-click the AndroidManifest.xml file in the Properties folder and tick the permission (figure 14.3).

  Figure 14.3. Adding the internet permission to AndroidManifest.xml

  

For the purposes of this book, we’ll use release builds for Android, so open the Android manifest and add the Internet permission.

Setting up your iOS apps for UI testing

Apart from the issue of the shared mono runtime, UITest just works with Android out of the box—UITest can connect to your running Android app on a device or an emulator and interact with it. iOS, on the other hand, isn’t quite as simple. Due to the stricter security on iOS, you can’t simply have anything connect to a simulator or device and interact with the app. Instead you need to install an extra component into your iOS apps that you initialize before your UI tests can run. To do this, add the Xamarin.TestCloud.Agent NuGet package (figure 14.4) to the Countr iOS app (Test Cloud is the Xamarin cloud-based testing service that we’ll look at in the next chapter, and you’ll see the name used in a few places with UITest).

Figure 14.4. Adding the Xamarin Test Cloud Agent NuGet package

Once this NuGet package is installed, you’ll need to add a single line of code to initialize it. Open AppDelegate.cs and add the code in the following listing.

Listing 14.1. Enabling the Calabash server for debug builds only

public override bool FinishedLaunching(UIApplication app,
                                       NSDictionary options)
{
   #if DEBUG                        1
   Xamarin.Calabash.Start();        1
   #endif                           1
   ...
}

• 1 For debug builds, start the Calabash server

This code starts the Calabash server for debug builds only. The Calabash server is an HTTP server that runs inside your app, and the UITest framework connects to it so it can interact with your app. Apple is very strict about security in its apps and would never allow an app with an open HTTP port like this on the App Store. To avoid this, the Calabash server is only enabled for debug builds—for release builds this code won’t get run, the linker will strip it out, and your app won’t be rejected from the App Store (at least not due to the Calabash server).

14.1.3. Running the auto-generated tests

UI tests are run in the same way as any other unit tests—you can run them from the Test Explorer (Windows) or Test pad (Mac).

UI tests rely on having a compiled and packaged app to run, so the first step is to build and either deploy to a device, emulator, or simulator or run the app you want to test. Note that just doing a build isn’t enough for Android—a build compiles the code, but it doesn’t package it up. The easiest way to ensure you have a compiled and packaged app is to run it once.

On Android, create a release build; for iOS use a debug build to enable the Calabash server. You also need to set what device, emulator, or simulator you want to run your tests on in the same way that you’d select the target for debugging.

Getting ready to run the tests

If you open the Test pad or Test Explorer, you may not see the UI tests if the project hasn’t been built (figure 14.5). If you don’t see the tests, build the UITest project, and the tests should appear.

Figure 14.5. The Test Explorer (Windows) and Test pad (Mac)

If you expand the test tree in Visual Studio for Mac, you’ll see two fixtures: Tests(Android) and Tests(iOS). These are the two fixtures declared with the two TestFixture attributes on the Tests class. When you run the tests from Tests(Android), it will construct the test fixture by passing Platform.Android to the constructor, which in turn will use the AppInitializer to start the Android app. Tests(iOS) is the same, but for the iOS app. Under each fixture you’ll see the same test, called AppLaunches.

In Visual Studio, you won’t see the same hierarchy out of the box, so drop down the Group By box and select Class to see the tests grouped by test fixture (figure 14.6). You can only test Android apps with Visual Studio, so feel free to comment out the [TestFixture(Platform.iOS)] attribute in the Tests class to remove these tests from the Test Explorer.

Figure 14.6. Grouping tests in the Visual Studio Test Explorer

Before you can run the test, make a small tweak. Despite the fact that the test calls app.Screenshot, this test won’t spit out a screenshot. For some reason, UITest is configured to only create screenshots if the tests are run on Xamarin’s Test Cloud, so you need to change this configuration to always generate screenshots. To do so, add the following code to AppInitializer.cs.

Listing 14.2. Enabling local screenshots for UI tests

public static IApp StartApp(Platform platform)
{
   if (platform == Platform.Android)
   {
      return ConfigureApp
         .Android
         .EnableLocalScreenshots()           1
         .StartApp();
   }

   return ConfigureApp
      .iOS
      .EnableLocalScreenshots()              1
      .StartApp();
}

• 1 Calls EnableLocalScreenshots on the fluent configuration API to turn on screenshots

By default, the StartApp method doesn’t do anything to configure the app that’s being tested, which means that it expects that the app to be tested is a part of the current solution. You also need to configure UITest so it knows which apps in the solution it should use, as there could be multiple apps.

Setting the app to test in Visual Studio for Mac

Open the Test pad and expand the Countr.UITests node. Under this you’ll see the test fixtures, as well as a node called Test Apps, shown next to a stack of green arrows. Right-click this and select Add App Project. In the dialog box that appears, tick Countr.Droid and Countr.iOS, and then click OK. You’ll see these two apps appear under the Test Apps node.

If you right-click one of them, you’ll see a number of options, including a list of possible target devices to run against, with Current Device ticked. This list is used to set which device the UI tests should be run against when you run them from the pad. If you leave Current Device selected, it will use whatever target is set from the main toolbar, but if you always want the tests to run against a particular emulator, simulator, or device, you can select it from here.

You can also configure the app in code

If you want to run an app outside the solution, or run the tests outside the IDE (such as from a CI server), you can configure which app to run and which device to run it on using the Configure App fluent API. See the Xamarin developer docs at http://mng.bz/fE5S for more information on how to do this.

Setting the app to test in Visual Studio

UITest uses NUnit to run tests, so you need to ensure Visual Studio is configured to run NUnit tests. Back in chapter 7 we installed the NUnit 3 test adapter, but to use UITest you’ll also need to install the NUnit 2 adapter. Select Tools > Extensions and Updates, and then select the Online tab on the left and search for NUnit 2. Select NUnit 2 Test Adapter in the list in the middle and click the Download button (figure 14.7). You’ll need to close Visual Studio for this to be installed, so relaunch it after the install and reload the Countr solution.

Figure 14.7. Installing the NUnit 2 test adapter

Visual Studio only supports testing Android, so delete the [TestFixture(Platform .iOS)] attribute from the Tests class. This will stop iOS tests from showing up in the Test Explorer.

Unlike Visual Studio for Mac, there’s no way on Windows to set the test apps. Instead you need to configure this in code by giving it the path to the Android APK, which is in the output folder and is named based on the Android package name with the .apk file extension. Release builds also have a suffix of -Signed to indicate that they’ve been signed with your keystore. You set the package name in the Android manifest in the last chapter, based on a reverse domain name (mine was set to io.jimbobbennett.Countr), and you can find this file in Countr.DroidinRelease if you’ve built using the release configuration or Countr.DroidinDebug for the debug configuration. We’ll be using release Android builds for the purposes of this book, so add the following code to point UITest to the right APK, substituting in your package name.

Listing 14.3. Configuring UITest to use the Countr.Droid APK file

if (platform == Platform.Android)
{
   return ConfigureApp
      .Android
      .EnableLocalScreenshots()
      .ApkFile ("../../../Countr.Droid/bin/Release/       1
       <your package name>-Signed.apk")                 1
      .StartApp();
}

• 1 Configures the app to use the release APK

Replace <your package name> in this code with the name of your package (for example, I’d use io.jimbobbennett.Countr). This assumes your UI test project has been created in the same directory as all the other projects, so the folder for Countr.UITests is at the same level as the folder for Countr.Droid. If not, change the path in this code to match your folder structure.

When tests are run, they’ll be run on the device or emulator that you’ve selected for the build configuration, just as for debugging your apps. This makes it easy to change the device that tests are run on by changing the drop-down in the toolbar, just as you’d change the target for debugging, as shown in figure 14.8.

Figure 14.8. The device to test on is set in the same way as the device for debugging.

Running the test

Once the test apps have been configured, you can run the tests by double-clicking them in the Test pad in Visual Studio for Mac, or by right-clicking them in the Visual Studio for Windows Test Explorer and selecting Run Selected Tests. If you’re testing Android, set the build configuration to release, and for iOS set it to debug.

When the test is run, the following things happen:

1. The unit test runner loads the test fixture, passing the relevant platform to the constructor.
2. Before each test, the BeforeEachTest method marked with the SetUp attribute is called, and this in turn calls the StartApp method on AppInitializer. This will configure the app for the relevant platform, enable screenshots, and launch the app, returning an IApp instance that’s stored in the app field on the test class and that can be used to interact with the app.
3. UITest knows which app to start and what device to launch it on based on the TestApps setting or the configured APK, so it will launch the relevant emulator, simulator, or device and start the app.
4. On iOS, before the app is launched, another app will be installed on the simulator or device, called “Device Agent”. This is part of the iOS XCTest framework, which UITest uses under the hood and is needed to control your app. You can just ignore this app; if you delete it, it will be reinstalled next time the test is run.
5. The test case is run—in our case, a test called AppLaunches, which calls the Screenshot method to capture a screenshot.
6. The screenshot is captured and placed in the output directory of the UI test project. This screenshot can be found at Countr.UITests/bin/Debug/screenshot-1.png.
7. The test will pass, as nothing went wrong.

As a first test this is OK, but not great. It shows that the app launched, which in itself is a valuable test, but it doesn’t tell us much more than that. The screenshot is not of much use either—on iOS it will be the first screen, but on Android it might just be the splash screen, depending on how long the app takes to launch. Let’s now look at writing some proper tests.

14.2. Writing tests

Our apps are running inside a simple UI test, but how can we take this further and write some useful tests? First, let’s define a couple of tests that we want to run, and then we’ll look at how we can implement them:

• Adding a counter— This test should add a new counter and verify that it has been added. It would involve the following steps:

  Step	Description
  Arrange	Start the app (this happens in the BeforeEachTest method).
  Act	Tap the Add button, add a name for the counter, and tap the Done button.
  Assert	Verify that the new counter is visible with a count of 0.

• Incrementing a counter— This test should take an existing counter, increment it, and verify that the count has gone up by one. It would follow these steps:

  Step	Description
  Arrange	Start the app (this happens in the BeforeEachTest method) and add a new counter.
  Act	Tap the Increment button for the new counter.
  Assert	Verify that the new counter has a count of 1.

These are simple tests. We’ll start by building the structure of the test methods, and then we’ll look at how we can fill in the tests. Add the following code to Tests.cs.

Listing 14.4. Test methods ready for implementing our UI tests

[Test]
public void AddingACounterAddsItToTheCountersScreen()
{
    // Arrange
    // Act
    // Assert
}

[Test]
public void IncrementingACounterAddsOneToItsCount()
{
    // Arrange
    // Act
    // Assert
}

14.2.1. The visual tree

UITest views your app as a visual tree. This concept should be familiar to you if you’ve done UI development before with WPF or HTML, but essentially it’s a hierarchical representation of everything that’s visible on the screen, and it should map to the UI that you’ve designed in your storyboard or Android layout file. For example, for the counters screen in Countr, the visual tree on Android when showing two counters would be something like the tree shown in figure 14.9.

Figure 14.9. The visual tree on Android and how it maps to the UI

This tree is a hierarchy, so there are parent/child and sibling relationships. The RelativeLayout has a TextView as one of its children, a RecyclerView as its parent, and another RelativeLayout as its sibling.

UITest has a number of methods on the IApp interface that interact with your app, and they know which controls to use based on an app query. App queries are functions that look inside the visual tree of your app for a control that matches a specific thing—this can be based on some kind of identifier, text inside the control, or its class type. You can also query based on relationships, such as finding parent, child, or sibling controls. When you write a UITest test, you use app queries to find the controls that you want to interact with. Working out exactly what app query to write can be easy when you know the visual tree for your app, but sometimes it’s more difficult and requires writing complicated queries. Luckily, there’s a REPL that makes it easy to see your app’s visual tree and try out queries while your app is running.

14.2.2. The REPL

A Read-Evaluate-Print loop (REPL) is a command-line tool that allows you to execute commands inside some kind of context. You may have used something similar before, such as the Interactive window in Visual Studio, where you can run C# code that has access to the current application stack while debugging.

You can launch the UITest REPL by using the IApp.Repl() method, so update one of the new tests to call this method with the following code.

Listing 14.5. Launching the REPL from a UI test

[Test]
public void AddingACounterAddsItToTheCountersScreen()
{
   app.Repl();        1
   ...
}

• 1 Adds a call to launch the REPL

Once this code has been added, run the test. It will hit this line, launch the REPL in a terminal or command-line window, and wait (figure 14.10). The Repl() method will only return once the REPL has closed, either by closing the command-line window or using the exit command, so you should only use this method when building your tests. You should remove it once your tests are written, or your tests will never finish.

Figure 14.10. The REPL running in a command-line window

You may also find that you can’t stop your tests from inside Visual Studio while the REPL is running, so if you need to stop your tests, you’ll need to manually close the REPL.

Once you’re in the REPL, the most useful command is tree, which shows the visual tree. Type this and press Enter. Figure 14.11 shows the visual tree on iOS, and figure 14.12 shows the visual tree on Android.

Figure 14.11. The visual tree for Countr on iOS inside the REPL

Figure 14.12. The visual tree for Countr on Android in the REPL

This visual tree is a representation of the visual tree inside your app, flattened a bit to be more useful. It shows the controls using the underlying types from the OS, and it shows some string-based properties on the controls, such as the text of a button. The tree will look different on iOS and Android because the controls are different on each platform.

The REPL is a full-featured C# REPL as well

UITest app queries are written in C#, so to make the REPL work, Xamarin had to provide full C# capabilities inside it, with a field called app pointing to your IApp instance. This means you can run other C# code if you want, just as in the Immediate window in Visual Studio, and if you call a variable by its name with no operators or methods, the value will be printed to the console.

The first step of our first test involves tapping the Add button to add a new counter, so we need to write a query for this.

14.2.3. Identifying controls

There are a number of different ways to query for a control, such as based on the class or the text inside it, but for our Add counter button, this might be problematic. On iOS, it’s a text-based button showing the text Add, but on Android it’s a floating action button with an image inside it and no text. We could write different queries for different platforms, but this isn’t ideal—we want to have as much cross-platform code as possible, both in our apps and our tests.

The easiest thing to do is to assign some kind of unique ID to the navigation bar button on iOS and the floating action button on Android, and use this to identify the control. That way we can use this identifier in the test, and it will work on both platforms. Identifiers are easy to add—on Android you can use the id property in the layout AXML file, and on iOS you can add an AccessibilityIdentifier property.

Let’s start with Android. Open the counters_view.axml layout file and add the following code.

Listing 14.6. Adding an ID to an Android floating action button

<android.support.design.widget.FloatingActionButton
   android:id="@+id/add_counter_button"               1
   ...
   />

• 1 Adds the ID to the floating action button

On iOS, open the CountersView.cs file and add the following code. We’re doing this in code because the navigation bar button is created in code, but for controls on a storyboard, this can be set using the Properties pad.

Listing 14.7. Adding an ID to an iOS navigation bar button

public override void ViewDidLoad()
{
   ...
   var button = new UIBarButtonItem(UIBarButtonSystemItem.Add);
   button.AccessibilityIdentifier = "add_counter_button";           1
   ...
}

• 1 Sets the accessibility identifier on the navigation bar button

Casing for ID names

In this example I’ve used lowercase names with underscores for the IDs simply because this matches the Android standard, and it means we can stay consistent with the standard for IDs used for identifying controls in UI tests, as well as for relative layout references and other uses. It’s up to you what naming convention you use, but you must use the same value on iOS and Android for the UI tests to work.

After making these changes, build and deploy the app, and then rerun the UI tests and look at the tree in the REPL. In the Android tree you’ll see this:

[FloatingActionButton] id: "add_counter_button"

On iOS, you’ll see this:

[UINavigationButton > UIImageView] id: "add_counter_button", label: "Add"

Now that you have your IDs set, it’s time to write your first query. In the REPL, type app. and you’ll see a load of autocomplete options identifying the methods on the IApp instance. app is an instance of IApp pointing to your app, and it’s the same as the app field in the Tests class. That means everything you write in your REPL can be used in your tests. One of the auto-complete options is Query, which can be used to perform an app query and return all the controls that match the query.

App queries are lambda functions using the same syntax as C#, so start by typing app.Query(c => c.. The parameter to the function, c in this case, is of type AppQuery, which has a stack of methods that can be used to run different queries. All methods return the AppQuery instance, so you can chain multiple calls if needed.

After typing the period (.), you’ll see an autocomplete list showing all the possible methods, such as Child, Id, Marked, and Text. You need to use the Id method, passing the ID you’ve just set as the parameter. Run the command app.Query(c => c.Id ("add_counter_button")) and look at the results. Queries return a list of all items that match the query, and in this case there’s a single item: the floating action button or navigation bar button (depending on platform). This query and its result is shown in figure 14.13. We’ll look at a whole range of different app queries you can run later in this chapter.

Figure 14.13. Querying by ID returns only those items in the tree that match the ID.

14.2.4. Tapping the Add button

Once you’ve found the Add counter button, you can tap it easily. In the app field, there’s a method called Tap that takes an app query. This will run the query and tap the first item it finds that matches the query. If nothing matches the query, it will throw an exception.

If you type app. again, you’ll see Tap in the autocomplete list. Run the Tap command using the same app query as before: app.Tap(c => c.Id("add_counter_button")). The REPL has a command history, so you can always just tap the Up arrow to see the query again and edit the command to use Tap instead of Query. If you watch your app while this command is running, you’ll see the new counter screen appear. We’ll look at some of the other methods on IApp later in this chapter.

This gives you the first part of the Act step of your add new counter test, so let’s add this to the test. Add the following code to tap the button before the REPL is shown.

Listing 14.8. Tapping the add new counter button from the test

[Test]
public void AddingACounterAddsItToTheCountersScreen()
{
   // Arrange
   // Act
   app.Tap(c => c.Id("add_counter_button"));      1
   app.Repl();
   // Assert
}

• 1 Taps the add counter button

The REPL has a copy command

The REPL has a copy command that copies the entire command history to the clipboard. This is useful for writing tests in the REPL and then copying them to the clipboard to paste into your own tests.

The next step is to enter the counter name into the text box.

14.2.5. Entering text

Entering text is as simple as tapping buttons—there’s an EnterText method on the IApp interface that takes an app query and a string containing the text you want to add. Once again, though, how do you know which control to enter the text into. If you run the tree command again, you’ll see a different tree—after all, the app has navigated to the add new counter screen, but there’s nothing in common between the iOS UITextField and Android AppCompatEditText. As before, you need to add some IDs.

Add the following code to the counter_view.axml layout file in the Android project to add the Android IDs.

Listing 14.9. Add an ID to the Android text-entry field

<EditText
   android:id="@+id/new_counter_name"       1
   ...
   />

• 1 Adds an ID to the text-entry field

On iOS, this time you can do it from the storyboard instead of the code. Open CounterView.storyboard, select the counter name text field, and set the Identifier property in the Accessibility section of the Widget tab to be new_counter_name, as shown in figure 14.14.

Figure 14.14. Setting the Accessibility Identifier from a storyboard

Now if you build, deploy, and run the UI test to tap the Add button, and you launch the REPL and look at the tree, you’ll see that the text box has an identifier of new_counter_name. You can now use the EnterText method to set the text by running app.EnterText(c => c.Id("new_counter_name"), "My Counter"). This will find the first element matching the query, and enter the text, character by character, simulating what would happen when a user types the text.

Add the following code to the test before the REPL.

Listing 14.10. Entering text from the test

[Test]
public void AddingACounterAddsItToTheCountersScreen()
{
   // Arrange
   // Act
   app.Tap(c => c.Id("add_counter_button"));
   app.EnterText(c => c.Id("new_counter_name"), "My Counter");      1
   app.Repl();
   // Assert
}

• 1 Enters text into the counter name text box

Disconnect the hardware keyboard

On iOS, text is entered by tapping the keys on the software keyboard. This means that if you’re using a simulator to run your tests, you’ll need to disable the hardware keyboard from inside the simulator by ensuring Hardware > Keyboard > Connect Hardware Keyboard is unchecked. If you don’t, any UI test that enters text will fail with “Timed out waiting for keyboard.”

14.2.6. Finding controls based on their text

After setting the name, you need to tap the Done button to add the counter. This time there’s a similarity between the button on both platforms—they both have a button with the text set to “Done.” This means that rather than having to set an identifier, you can query for the text directly. From the REPL, query for this using app.Query(c => c.Text("Done")), and you’ll see the Done button.

If you do this on Android, you might notice something interesting—the text of the button is “Done” in the tree, but on screen it’s “DONE”, all in capitals. This is because the default button style on Android capitalizes the text, but just the text on the display, not on the underlying control. This is very helpful, as text queries are case sensitive: querying for “Done” gives one result, but querying for “DONE” gives none. When building your app, you can set the text for controls using normal casing on both iOS and Android, you can query for it in the same way on both platforms, but you can still have Android buttons automatically show on screen in uppercase.

Text queries can query for any text on a control—so for labels it will match the static text that’s showing, for text boxes it will match the text entered by the user, and for buttons it will match the button text. If you use this query with the Tap command, it will tap the Done button and add the counter, navigating back to the counters screen.

Try it out and then add it to the unit test so it matches the following listing.

Listing 14.11. Tapping the Done button

[Test]
public void AddingACounterAddsItToTheCountersScreen()
{
   // Arrange
   // Act
   app.Tap(c => c.Id("add_counter_button"));
   app.EnterText(c => c.Id("new_counter_name"), "My Counter");
   app.Tap(c => c.Text("Done"));                                  1
   app.Repl();
   // Assert
}

• 1 Taps the button with the text “Done”

14.2.7. Assertions

The Arrange (handled by the BeforeEachTest setup method) and Act steps are done. Now it’s time to move on to Assert, where you assert that you can see a new counter with the name of My Counter and a count of 0.

The easiest way to do asserts is to use an app query for what you’re looking for, and assert the results of the query by using some helper methods on the IApp interface. IApp has two useful methods for this: WaitForElement and WaitForNoElement. Both take an app query. The first will wait for at least one element that matches the query, timing out if nothing is found after 10 seconds and throwing an exception. The other will wait until no elements match the query, and again will throw an exception if there are elements that match the query after 10 seconds. The 10-second timeout is configurable, so it can be made longer if needed (such as if your app is running a slow action).

For this assertion, ensure that you can see the new counter with the name of My Counter on the counters screen. If you look at the tree in the REPL, you’ll see there’s a label with the text “My Counter” as part of the tree view or recycler view, but if you do a text query for “My Counter” you wouldn’t be able to distinguish between the counter showing on the counters screen or the text box on the add new counter screen showing the same text. Essentially, the test couldn’t assert that the Done button had actually done anything. What you want is to verify that you can see the text in the right place, and you can do this by using multiple app queries.

The first thing to do is to add some more IDs so you can identify the labels in the table view or recycler view that show the name and count. To do this on Android, add the code in the following listing to the counter_recycler_view.axml layout file to add the ID of “counter_name” to the name label and “counter_count” to the count label.

Listing 14.12. Adding IDs to the counter view used by the recycler view

<TextView
   ...
   local:MvxBind="Text Name"
   android:id="@+id/counter_name"/>        1
<TextView
   ...
   local:MvxBind="Text Count"
   android:id="@+id/counter_count"/>       2

• 1 Sets the counter name label ID
• 2 Sets the counter count label ID

For iOS, once again you can do this from the storyboard by selecting the labels in the cell prototype in the CountersView.storyboard file and setting the Identifier properties in the Accessibility section. Select the counter name label and set the identifier to be “counter_name”; then select the counter count label and set the identifier to be “counter_count”.

Once you have these identifiers in place, rebuild and redeploy the apps, and then rerun the UI test. Once the REPL loads, run the tree command and you’ll see that the two labels now have their IDs set. You now have two ways to query the counter name: you can query based on the ID (but that doesn’t tell you what text is showing), or you can query based on the text (but that won’t tell you if the text is on the counters screen, or if you’re still on the add new counter screen and the text is in the text box).

To solve this problem, you can combine the queries. The app query methods are part of a fluent interface—one that returns the same object that the method was called on, allowing you to essentially chain methods. This means you can combine queries for the ID and the text. Try it out by running the command app.Query(c => c.Id("counter_name").Text("My Counter")). This will search for all items that have an ID of counter_name, and from those results will return only the ones with the text set to My Counter (figure 14.15). You can chain as many queries as you like—the first query in the chain queries the entire visual tree, and each subsequent query queries the result of the previous query.

Figure 14.15. Querying by ID and text returns only those items in the tree that match the ID and text.

You’ll see that this query will return a single result, the label showing the new counter in the list, and you can write an identical query to prove that the count is 0 by querying for the ID “counter_count” and the text “0”. You can turn these queries into assertions in your test by using the WaitForElement method, as shown in listing 14.13. Also, you can remove the Repl call, because you don’t need it for this test anymore.

Listing 14.13. The complete test, asserting that the new counter has been found

[Test]
public void AddingACounterAddsItToTheCountersScreen()
{
   // Arrange
   // Act
   app.Tap(c => c.Id("add_counter_button"));
   app.EnterText(c => c.Id("new_counter_name"), "My Counter");
   app.Tap(c => c.Text("Done"));
   // Assert
   app.WaitForElement(c => c.Id("counter_name").Text("My Counter"));    1
   app.WaitForElement(c => c.Id("counter_count").Text("0"));            1
}

• 1 The assertions, verifying the text is correct for the new counter

After making this change, run the test, and you’ll see it pass.

14.2.8. Proving your test by breaking things

You now have a test that passes, but ideally you should make your test fail at least once, just to be sure that the test is working as expected. There are many ways to break the app, but a simple way would be to comment out one of the bindings (to simulate what would happen if you had a bug in your code due to forgetting a binding). Try commenting out the bindings in CountersView, as follows.

Listing 14.14. Commenting out some bindings to simulate a bug in the code

public override void ViewDidLoad()
{
   ...
   //var set = this.CreateBindingSet<CountersView, CountersViewModel>();  1
   //set.Bind(source).To(vm => vm.Counters);                              1
   //set.Bind(button).To(vm => vm.AddNewCounterCommand);                  1
   //set.Apply();                                                         1
}

• 1 Comments out the bindings

If you make this change, build and deploy the app, and re-run the test, you’ll see it fail. If you look in the output, you’ll see the error shown in figure 14.16.

Figure 14.16. The output of a test failure, showing that the query for “new_counter_name” didn’t give any results

This error shows you that the query for the control with the ID of “new_counter_name” didn’t find anything, and the stack trace points to the call to EnterText in the AddingACounterAddsItToTheCountersScreen test. This tells you that when the query was run, the new counter name text box wasn’t showing, so now you need to find out why.

There are multiple ways to debug a UI test. By debug, I mean see what’s happening on screen when your test runs. When a UI test is running through the debugger, you can only debug the test itself. You can’t step into the app or attach the debugger to the app, so if you need to debug a problem inside your app, you’ll need to run it and manually work through the tests steps to get there.

To see what’s happening when the test runs, you can do one of the following:

• Debug the UI test— UI tests can be debugged just like any other unit test. You can set breakpoints, step through, evaluate variables, and do anything else you’d normally do in a debugger. The usual way to debug a UI test is to set a breakpoint on the line that fails, then debug the test by right-clicking it in the Test Explorer or Test pad and selecting Debug Test. This will launch the test in a debugger and break on the breakpoint. To do this for your test, set a breakpoint on the call to app.EnterText(c => c.Id("new_counter_name"), "My Counter"), and debug the test. When the breakpoint is hit, take a look at the app and you’ll see that it’s still on the counters screen, not the add new counter screen. This suggests that tapping add_counter_button didn’t work, and you can test this theory by manually tapping the button and seeing what happens. When you run a UI test, it’s the same as if you manually followed all the steps, so you can interact with the app yourself at any time to try things out. (Some developers even use UI tests to get their app to a known state that takes multiple steps, saving themselves the boring repetition of getting there manually.)
• Use the REPL— The REPL is a great way to investigate issues. Debugging can help find the line that failed, but sometimes the REPL can help you find out why it failed. For example, if an automation ID was missing, you could see this from the tree (a regular scenario, especially when using multiple layout files on Android to support multiple screens). You can launch the REPL from any point in your test by making a call to app.Repl(), either by adding this manually to the code or by breaking on a breakpoint and calling it from the Immediate window. If you want to try the REPL as a debugging tool, try uncommenting the binding code in CountersView so that the add new counter button works again, but remove the ID from the add new counter button in the iOS CountersView or Android counters_view.axml. Build and deploy the app, and put a breakpoint on the call to EnterText. When the app breaks on this breakpoint, open the Immediate window (View > Debug Pads > Immediate in Visual Studio for Mac, or Debug > Windows > Immediate in Visual Studio) and run app.Repl(). You can then call tree to see that the Add button doesn’t have an ID set.
• Take screenshots— At each step in your test, you can save a screenshot by making a call to app.Screenshot("<name>"). The name you pass should be an identifier for where in the test the screenshot was taken—these names aren’t used for local screenshots, but if you use Test Cloud (we’ll look at it in the next chapter), these names will be shown against each screenshot. Screenshots are a great way to see the state of the app as the test is run, and they can be especially useful if you have a flappy test—one that passes most of the time but fails occasionally (for example, a test that relies on an external resource, such as a web service that’s sometimes not available or a network that times out or loses connection). To debug your issue using screenshots, add the following code to your test fixture.

  Listing 14.15. Taking a screenshot of the test as it runs

  [Test]
  public void AddingACounterAddsItToTheCountersScreen()
  {
     ...
     app.Tap(c => c.Id("add_counter_button"));
     app.Screenshot("About to enter text");                         1
     app.EnterText(c => c.Id("new_counter_name"), "My Counter");

  • 1 Take a screenshot
  If you look in the output folder of the UI tests, which will be Countr.UITests/bin/Debug if you run your tests as a debug build, you’ll see a screenshot called screenshot-1.png. This is the state of the app before the EnterText method was called, so if your app was working you’d see the add new counter screen. You can see main counters screen instead, so it’s obvious that something isn’t working when the Add button is tapped.
• App Center crash reports— If your app crashes during a UI test, you won’t be able to see, in a debugger, which line of code is causing the crash. If you wired up your app to App Center crash analytics, you would be able to see the crash details in App Center.

Once you’re done, revert all changes to the bindings and IDs, rebuild and redeploy the app, and re-run the test to ensure it passes.

14.3. Testing incrementing a counter

Time to finish the tests by adding the last one—testing incrementing a counter. This test increments a counter and ensures that the count has been correctly incremented, so as part of the Arrange step you’ll need to create a new counter. You can do this by reusing the test code from the previous test, as follows.

Listing 14.16. Creating a new counter in the Arrange step of the test

[Test]
public void IncrementingACounterAddsOneToItsCount()
{
   // Arrange
   app.Tap(c => c.Id("add_counter_button"));                       1
   app.Screenshot("About to enter text");                          1
   app.EnterText(c => c.Id("new_counter_name"), "My Counter");     1
   app.Tap(c => c.Text("Done"));                                   1
   // Act
   // Assert
}

• 1 Creates a new counter

For the Act step, you need to tap the Increment button. Once again, this is different on iOS and Android, so you need to set an ID. On Android, the ID for the image button in the counter_recycler_view.axml layout file has already been set to add_image, so that the count label could be positioned to the left of it, so you can reuse this ID on iOS. Open CountersView.storyboard and set the accessibility identifier for the Increment button to be “add_image”. Once this ID is set, you can add a call to tap the button, and an assert that the count text is now “1”, as shown in the following listing.

Listing 14.17. Tapping the Increment button and asserting on the new value

[Test]
public void IncrementingACounterAddsOneToItsCount()
{
   ...
   // Act
   app.Tap(c => c.Id("add_image"));                             1
   // Assert
   app.WaitForElement(c => c.Id("counter_count").Text("1"));    2
}

• 1 Taps the Increment counter button
• 2 Asserts that the counter count is now 1

Run this test, and it should be a nice green color. You now have two UI tests that ensure you can add counters and increment them, and you have some confidence that you’ve built an app that works.

UI tests should not be the only tests you run. They’re a great way to ensure that your view is bound correctly and that all the parts of your app work together, but they should always be used in conjunction with thorough unit testing.

14.4. The app interface and app queries

Let’s take some time to recap the IApp interface and look at what else is available on this interface. We’ll also look in more detail at app queries.

14.4.1. The IApp interface

The IApp interface has a number of methods that can interact with your app. Some take app queries and interact with the controls that match the query, and others act on the app as a whole. Table 14.2 shows some of the general methods on this interface.

Table 14.2. IApp interface methods

Method	Description
Back	Navigates back using the iOS navigation bar or Android Back button. On Android, this uses the hardware Back button, so if the keyboard is showing, this will dismiss the keyboard rather than going back. If your screen has any text-input controls that would show the keyboard, you should dismiss the keyboard first.
DismissKeyboard	Hides the keyboard if it’s visible.
Repl	Shows the REPL.
SetOrientationLandscape and SetOrientationPortrait	Changes the orientation of the device.
Screenshot	Takes a screenshot (only works in Test Cloud unless the app is configured to allow local screenshots).
Invoke	Invokes a backdoor method, allowing you to interact with hidden features in your app. You can read more about these in Xamarin’s documentation at http://mng.bz/MSXX.

Table 14.3 shows some of the methods that take app queries.

Table 14.3. IApp interface methods that take app queries

Method	Description
Query	Returns all the items in the visual tree that match the query.
Tap and DoubleTap	Taps (or double-taps) the first item in the visual tree that matches the query (if there are multiple matches, only the first is tapped), and throws an exception if none is found.
EnterText and ClearText	Enters text into or clears all text from the first item that can accept text and matches the query. If there are multiple matches, the text is only entered into the first one. If there are no matches, an exception is thrown.
Flash	Makes all controls that match the query flash. This is useful when writing complicated queries and you want to test them out.
ScrollDownTo and ScrollUpTo	These methods take two queries: one for a scrollable container, and one for an item to find. They scroll up or down inside the control that matches the scrollable container query for an item that matches the item query.
WaitForElement	Waits for an element that matches the query and throws an exception if none are found in a certain period of time (defaults to 10 seconds).
WaitForNoElement	Waits for no elements that match the query and throws an exception if any is found after a certain period of time (defaults to 10 seconds).

You can read more about this interface and about how the different methods work in Xamarin’s documentation at http://mng.bz/J9co.

14.4.2. Queries

An app query defines the criteria used to find items in the visual tree. App queries have a fluent interface, with methods you call to define criteria that all return another app query instance, which you can then call another method on to build up more detailed criteria. Some of these methods are shown in table 14.4.

Table 14.4. AppQuery methods

Method	Description
Id	Finds controls that have an Android resource ID or iOS accessibility identifier set to the specified ID.
Text	Finds controls that have text that matches the query, such as the text on labels, text boxes, or buttons.
Marked	This is a combination of Id and Text, so finds anything that has an ID or text matching the given string.
Button	The same as Marked, but only for buttons.
Switch	The same as Marked, but only for switches.
TextField	The same as Marked, but only for text fields.

You can also query using strings instead of app query objects, and this is the same as using Marked. For example app.Tap("FooBar") is the same as app.Tap(c => c.Marked ("FooBar")).

App queries can also be used to find parents, children, and siblings, so they can be used to walk around the tree if necessary. You can find more information on building up app queries in the Xamarin documentation at http://mng.bz/kyx0.

There’s a lot more you can do with app queries and UITest

In this chapter I’ve only lightly touched on app queries and the capabilities of UITest. You can write some pretty advanced queries, including walking the visual tree based on parent, child, and sibling relationships and even invoking iOS or Android methods on controls and querying based on the results (such as finding all switches that are on). You can also expose methods inside your app (called backdoors) that you can call from the IApp instance, or invoke native methods on existing controls. This invoking of native methods can be useful when UI testing components that draw on the screen, such as chart or graphing controls—you can’t check what’s drawn using a UI test, but you can expose data through a native method that you can call and verify. You can read more about the capabilities of UITest in the Xamarin developer docs at http://mng.bz/cpM5.

Your app is now well tested and well on the way to being production-ready. In the next chapter we’ll look at setting up Visual Studio App Center to automatically build your apps, run your UI tests on a range of real devices in the cloud, and set up user analytics and crash reporting so that you can track how your users use the apps and what issues they have once you release them to your beta and production users.

Summary

In this chapter you learned

• Xamarin UITest allows you to write automated UI tests in C#.
• UITest can be used to interact with controls, including tapping buttons or reading values.
• Apps have a visual tree made up of a hierarchy of controls.
• You can write tests against the visual tree.

You also learned how to

• Use the REPL to query your app’s visual tree and interact with it.
• Use app queries to find controls on screen.
• Write UI tests using the same arrange, act, assert pattern you’ve already used for unit tests.