Why Choose App Center for UI Testing?

It should be fairly obvious what door should be the least risky and least expensive for you, but I will break down the logic for you. If you pick door #1, you cannot know with 100% certainty if your app will behave as expected on every phone and every phone factor. If you risk it and your app misbehaves, you risk the chance that one errant UI design like a button too small for pressing or a panel not showing will give your users a bad experience. Bad experience means bad reviews and fewer downloads. Bad reviews can be a death sentence for a mobile app. Door #1 is a foolish choice.

Door #2 is viable but not a wise choice either because it is expensive to own and manage multiple physical phones. In order to develop the app I wrote for this book, I personally own two iPhones, an Android and an iPad Mini. Clearly, I could not possibly purchase every device, especially Android devices, which is why I had to rely on App Center to test on most flavors of Android on the market now. And do not forget you will rack up costs because phone devices go obsolete quickly because manufacturers release new phones into the market constantly.

Before you dive in to the actual configuration and UI testing, let’s get a couple of important concepts out of the way.

Pricing

Let’s get the bad news out of the way first. Nothing is free forever! Microsoft wants to give you a taste of how App Center UI testing works. You get unlimited free tests on a single concurrency for the first 30 days, meaning you can run UI tests on one device at a time, so go wild with as many UI tests as you want during this time. Hopefully, if you follow my instructions in this chapter, you can shave off a significant number of repeated attempts to get it working right, thus giving you more of the trial period to write your own UI tests.

Let’s cover the pricing briefly. All of the pricing and billing information is in the App Center site itself. Go to the GoGoGiphy organization, click Manage, and click Billing on the left. Your webpage should look similar to Figure 6-1. Keep in mind the page itself may change since App Center updates the site often.

You can click the Learn more button, which will take you to the Pricing site. Please read through the FAQs before committing to any of the subscription plans because they do provide useful information.

Microsoft will not bill you based on time spent downloading your app, taking screenshots, or generating test reports. Billed time is actual time spent running your test suites.

You can visit the sites below to learn more about pricing. These pages have more information on how Microsoft bills you if you want to use concurrent builds.

https://docs.microsoft.com/en-us/appcenter/general/pricing

https://visualstudio.microsoft.com/app-center/pricing/

Migration

Several important events occurred in 2018. If you were a Xamarin mobile developer before 2018, you may have used Xamarin’s important services and applications as part of your DevOps pipeline, such as Xamarin Test Cloud and the Xamarin Test Recorder. After Microsoft’s acquisition of Xamarin, the company slowly integrated Test Cloud into App Center so now it longer exists on its own. It is all part of the App Center ecosystem of services. In fact, if any of your former apps hit https://testcloud.xamarin.com/ , you will get redirected to the App Center site. This migration includes all calls to the Test Cloud CLI because those calls will now get rerouted to the App Center CLI. And this migration includes migration of all existing data in Test Cloud into App Center.

On a sad note, Microsoft discontinued development and support for Xamarin Test Recorder, including the Visual Studio extension on Windows. If you remember what this extension was about, it watched and recorded all your actions on the app so you could paste them as actions for your UI tests. This app made the process of creating UI tests effortless. As one of the most loved tools among Xamarin developers, it is also one of the most missed. App Center does provide another tool called Repl, which I will cover later in this chapter. Honestly, if you ask me, it does not do justice to Test Recorder because Test Recorder was easier to use since it offered a GUI. But don’t get me wrong. The Repl tool is handy tool in its own right because you can still do most of the operations of Test Recorder, just through a console interface.

You can read more about Microsoft’s events and important notices about their migration from older frameworks to App Center at https://docs.microsoft.com/en-us/appcenter/migration/test-cloud/ .

Configuring Projects for Xamarin.UITest

As mentioned at the start, this book is not about teaching you how to write a Xamarin app. Similarly, this chapter is not about teaching you how to write UI tests. You should have a basic understanding of UI testing coming into this chapter.

Please visit https://docs.microsoft.com/en-us/appcenter/test-cloud/preparing-for-upload/ to learn more about each framework.

Testing on every framework gets redundant and is out of scope for this chapter, so I will show you how to write, integrate, and run UI tests using the Xamarin UITest framework.

Furthermore, if you are coming from another mobile development background like React Native, Cordova, or Unity, even though this book is about App Center using Xamarin Forms, I will show you the steps to run UI tests using the App Center CLI. Developers from other platforms should get value from this chapter because the CLI commands are identical on every platform.

Since you have been following along using the sample app from my GitHub repository, refer to the GoGoGiphy.UITest project in the Visual Studio solution. This project has multiple sample UI tests and some tests that you will run in App Center.

Installing the Necessary NuGet Packages

The first action you must perform is to install the Xamarin.UITest NuGet package for the GoGoGiphy.UITest project. Since you pulled the source code from the repository, these references should already be set up for you. But in the event that you set up your own test project, it is important to know the basic steps.

• Right-click the project and select Manage NuGet Packages.

• Go to Browse and search for Xamarin.UITest. Install the package.

• Return to Browse, search for NUnit, and install version 2.6.4, since at the time of this writing, version 2.6.4 is the latest version of NUnit that is compatible with Xamarin.UITest. If you install a later version of NUnit, your UI tests in later sections will fail because Xamarin.UITest will ask for this version. In the future, the Xamarin team may update Xamarin.UITest to reference a later version of NUnit.

• Return to Browse, search for NUnitTestAdapter, and install the latest version.

• Your NuGet packages and their versions should be similar to Figure 6-2.

Or if you prefer doing the same actions from the command line, open the Package Manager Console and run the following commands with their specified versions:

Install-Package NUnit -Version 2.6.4

Install-Package NUnitTestAdapter -Version 2.1.1

Install-Package Xamarin.UITest -Version 2.2.6

Setting Up Android

You need to disable the Use Shared Runtime setting in Android before running tests. What is this setting? This setting and its child setting, Use Fast Deployment, help manage and optimize the application size when deploying a Debug version of the app. A Debug version of the app deploys two large packages, Shared Runtime and Shared Platform, to the physical device. This setting allows you to deploy the Shared Runtime package once and every subsequent Debug version of the app reuses the package. In summary, it speeds up your deployment. You can find more information about this setting at https://docs.microsoft.com/en-us/xamarin/android/deploy-test/app-package-size .

But if this setting is enabled when the code is committed before running the Android build in App Center, it will prevent any UI tests from running. Therefore, you need to disable it before running any Android builds.

Follow these steps:

• Right-click the Android project.

• Go to Properties.

• Go to the Android Options tab.

• Unselect the Use Shared Runtime option, as shown in Figure 6-3.

You can find more information about this setting at https://docs.microsoft.com/en-us/appcenter/test-cloud/preparing-for-upload/uitest .

Setting Up iOS

Setting up your iOS project is a little more involved than with Android. The first step is to download and install the Xamarin Test Cloud NuGet package. The Test Cloud agent runs an HTTP server that exposes the iOS user interface so you can interact with it from your UI tests.

Follow these steps in iOS:

• Right-click the iOS project and select Manage NuGet Packages.

• Go to Browse, search for Xamarin.TestCloud.Agent, and install the latest version.

Your installed NuGet package should appear like Figure 6-4.

Or you can install the package from the Package Manager Console as follows:

Install-Package Xamarin.TestCloud.Agent -Version 0.21.7

The second step is to add a new conditional compilation symbol named ENABLE_TEST_CLOUD in the Build properties for the iOS project. This symbol instructs the iOS project to look for this additional symbol during the build when it creates the .IPA artifact according to the active build configuration. You will launch the Test Cloud Agent in the next section. You want to encapsulate this launching code within a condition so it only launches the agent for a debug build and not a release build.

Follow these steps:

• Right-click iOS project.

• Go to Properties.

• Go to Build tab.

• Add ENABLE_TEST_CLOUD to the end of the text in the compilation symbols textbox, as shown in Figure 6-5.

The last step is to launch the Test Cloud Agent in the AppDelegate class before the app itself launches. Add the following code section to the start of the FinishedLaunching(..) function in the AppDelegate.cs file:

#if ENABLE_TEST_CLOUD

  Xamarin.Calabash.Start();

#endif

The conditional compilation symbol ensures that the Test Cloud Agent only launches for Debug builds. The Test Cloud Agent only works on apps that are signed with a development provisioning profile. When running a UI test on your iOS app in App Center, the service will use one of their preexisting development provisioning profiles to sign your app before running. The above #if statement ensures that the Test Cloud Agent will launch when an App Center provisioning profile is used to sign your app before running a UI test.

Note

Remember that you cannot execute and run UI tests locally on Windows using the iOS Simulator. But you can run them locally on a Mac. From a Windows machine, you need to upload your UI tests to App Center to run your iOS tests. The best workflow tip is to run your Android tests locally on Windows against the Android simulator, upload your tests, run both Android and iOS builds in App Center, and verify that your iOS tests run as expected. The UI tests themselves will be identical for both platforms. The only difference is the attributes you apply to each UI test function to differentiate between the platforms. You will see this attribute later.

Installing the App Center CLI Locally

This section is not a mandatory section to run your UI tests. It is optional because you will run your UI tests using build scripts in App Center servers later in this chapter, but the commands are the same CLI commands described in this section. However, your UI tests will not work locally against App Center because the UI test commands use path references on the server and not your local system.

But this section is important because the CLI gives you the ability to query and execute multiple actions in App Center from a terminal window and not logging into the website. If you ever want to automate App Center from a PowerShell script or a .NET application, CLI is the answer. At the time of this writing, not all services that you can perform on the website are supported on the CLI, but the App Center team is adding more and more commands to the CLI all the time, which is part of the reason why I devote a small section to this topic.

You can read more about the CLI documentation and its commands at https://github.com/Microsoft/appcenter-cli and https://docs.microsoft.com/en-us/appcenter/cli/index .

Setting Up Device Sets in App Center

App Center lets you set up your own device sets where you can organize what devices your tests will run on. For example, one device set may be called Tier 1 and it may contain the most recently released phones on the market. A device set called iPhoneX may include all the iPhone X devices and the separate iOS versions. Remember to name them something descriptive. At the time of this writing, iPhone X was the biggest cat in the jungle in the Apple family of phone products.

Now you will learn how to set up a device set for iPhone X phones (Figure 6-6).

Enter a name for the device set. Search for iPhone x in the search bar (Figure 6-7).

Note that App Center does a wonderful job of giving you the same device but running different iOS versions, as shown in Figure 6-8.

Select the latest two OS versions of the iPhone X. iOS 12.0.1 and 11.4.1 were the latest two OS versions at the time of this writing. Click the New Device Set button at the bottom right. You should then see your new device set (Figure 6-9).

In Figure 6-9, you can see another device set for iPhone 8 devices. I will leave it to you set one up for yourself.

Now let’s set up a device set for Android. Figure 6-10 shows the setting of a name, searching for pixel 3, finding two devices matching the search, and selecting both devices. Note that I use the latest device using the latest Android version here, which is Android 9 at the time of this writing.

Your new device set should appear like Figure 6-11.

You can see the complete list of supported physical devices at https://docs.microsoft.com/en-us/appcenter/test-cloud/devices .

About App Center Variables

As mentioned, the App Center CLI commands will run within Bash scripts on the App Center server. The commands will be included in the post-build script because you want the UI tests to run after the projects have been built.

App Center provides more system variables that you can read about at https://docs.microsoft.com/en-us/appcenter/build/custom/scripts/#app-center-variables .

Setting Up Post-Build Scripts

You can find more information about build scripts at https://docs.microsoft.com/en-us/appcenter/build/custom/scripts/ .

Microsoft also maintains a public GitHub repository offering helpful examples of build scripts at https://github.com/Microsoft/appcenter-build-scripts-examples .

For the purpose of UI testing, you will set up two post-build scripts, one for iOS and one for Android. You will return later to these scripts and enter your commands to run in App Center, recommit, and repeat. Trust me. You will find yourself testing these commands repeatedly to see the output in App Center.

Keep in mind that the commands in these scripts are written in the Bash language. Don’t worry if you do not know Bash because the commands that you will use in this chapter are simple and easy to follow.

Open the Start sub-folder in your Git folder. Open the solution file in Visual Studio. In the Solution Explorer, you will find two files replicated in two places, one in the root location and the second in the Android project. The first file is called appcenter-post-clone.rename for the Post-Clone script and the second file is named appcenter-post-build.rename for the Post-Build script. Later in this chapter, you will rename the file extension to “.sh” so App Center can recognize the Bash script.

Create the file in the same location as the solution file, as shown in Figure 6-12.

Copy the above file. Navigate to the GoGoGiphy.Android folder. Paste the file into the location shown in Figure 6-13.

Commit the two files into your repository. Return to App Center. Select either the Android or the iOS build. I show the Android build definition in Figure 6-14. Open the build definition. You should see the Post-Build icon indicating that App Center sees the post-build script. Save the build definition just once and not Save & Build so App Center can retain the script as part of the build process. Open the build definition again and now click on Save & Build.

After saving and running the build, your build output will list the contents of the three folders you included in the script. However, the build output window on the website has a character limit count, so it will truncate the output after a certain point. It is not reliable if you want to inspect the entire output. The best approach to take is to download the logs and inspect the files (Figure 6-15).

Therefore, click the Download logs button, save the zip file, unzip it, and view the Post Build Script.txt file, as selected in Figure 6-16.

Saving the Access Token into an Environment Variable

Since every call to the UITest command will require a login from App Center for security purposes, you need to authenticate yourself with App Center first to prove that you are an authorized user of the service. As a result, you need to automate this step by saving the access token as an environment variable and passing in the access token when invoking the UITest command.

I covered the steps of saving the access token back in Chapter 4. If you forgot the steps or if you forgot to save the value of the token, please revisit the chapter again.

Please refer to Figure 6-17 for the following steps.

• Open either the Android or the iOS build definition.

• Toggle the Environment variables section to On.

• Give the variable a name like AppCenterToken.

• Paste the value of the token.

• Click the lock icon to redact the value because you do not want other members of your team to see this value when they open the build definition.

• Click Save.

• Repeat the above steps for the other build definition, iOS or Android.

You can always click the environment variables link in App Center, shown in Figure 6-17, to learn more about them.

Note the use of the $ before the name of the variable.

Testing on a Real Device

I know, I know. You want to get UI testing already. But you have one last topic to get out of the way so you know what it is about and how it is different from UI testing. App Center gives you the option of launching your app on various devices just to verify it launches successfully.

Keep in mind a couple of things about this step. First, it will not run your post-build script and has nothing to do with running UI tests in App Center. Second, it will add about 10 minutes to your normal build time. This step is helpful when you build an app for the first time and you want to know quickly if it launches on various preconfigured devices.

Return to either your Android or iOS build definition and toggle the command shown in Figure 6-18.

Click the Save & Build option. Wait for the build output. You should see the output shown in Figure 6-19.

Notice that your build output will show links to your test run. You can navigate to these links or go to Test Runs in App Center to view the test runs. But I will show you later in more detail the layout of the Test Runs section.

If only running UI tests were as easy as toggling an option and clicking a button, this chapter would just be a few pages long! Maybe the App Center team will make UI testing easier for developers in the future.

Getting the UITest Command Template

You are finally here: your first step in a long list of steps to get UI tests up and running! Be prepared because the journey is only beginning. In order to make the journey easier, you will let App Center do most of the work for you by using the UITest command template and then returning to the command template by adding new parameters and modifying the parameters App Center provides for you. In this way, you will make fewer mistakes with syntax errors or misspellings.

Return to App Center, toggle the Test tab open, and find Test runs, as shown in Figure 6-20.

You should see the Start testing your app button if you have never run a test before. Since you launched your app on a test device in the prior section, you will see your test run on this page. If you have never executed a test run, click the Start testing your app option (Figure 6-21). If you have launched your app before on a test device, click the New test run button at the top right.

Figure 6-22 shows the iOS app. Remember how you set up two device sets before, one for iPhone 8 devices and the second for iPhone X devices. You can select one of the preconfigured device sets or the Choose other device configuration option.

You will see a list of test series. If you enabled the test on a real device option in the build and ran it in the section before, you will see a test series like launch-tests, as shown in Figure 6-18. You can always define a new test series. A test series is a way for App Center to organize your test runs after your UI tests have completed. For example, you can use HomePageTests. But for now, keep it as the default master. And select Xamarin.UITest. Click Next. See Figure 6-23.

The third and final page shows a lot of relevant and useful information. First, it shows you instructions on how to install NodeJS and the App Center CLI. You completed both steps in a prior section, so you can skip them. Let’s focus on the section called Running tests. You are interested in the template for running your UI tests at the bottom of this page.

Please note that the template App Center gives you by default will not run as it currently exists. The remainder of this chapter is devoted to customizing this single command. Do not feel discouraged if you get stuck or if you find yourself customizing and repeating multiple times. Failure is the best teacher. This is the part when you get to try 100 times, fail 100 times, and succeed on the 101st! See Figure 6-24.

Let’s break down the command template and understand the purpose of each of the command arguments before moving ahead to customize the command.

The coming sections will focus more specifically on the --app-path and --build-dir arguments because those are the arguments that will require the most trial-and-error attention while the other arguments can be left to their default values. In addition, you will need to supply the CLI with more arguments because the above template will not work as it currently exists. I will explain soon.

Configuring the iOS UITest Command

Let’s begin by modifying the appcenter-post-build.sh.rename file for iOS first. Rename the existing file extension from .rename to .sh. If you created the Bash script on your own, locate the file next to the solution file. Open the bash script in either Visual Studio or your favorite text editor.

You need to remember a couple of important points to help you customize the command template. First, every file path will reference paths in the App Center server environment and not your local machine. Second, you will make use of the App Center variables that I presented in a prior section because they will allow you to reference key locations in the App Center server.

Final iOS UITest Command

If you completed all of the above steps, you should be close to the finish line! Do not get discouraged if it does not work the first time. Some things may have changed in App Center since the time of this writing. I summarized all the steps that I performed after getting my UI tests to run.

Your full CLI command should now look like Listing 6-2.

appcenter test run uitest

--app "GoGoGiphyGroup/GoGoGiphy-iOS"

--devices "GoGoGiphyGroup/iphonex"

--app-path $APPCENTER_OUTPUT_DIRECTORY/GoGoGiphy.iOS.ipa

--test-series "master"

--locale "en_US"

--build-dir $APPCENTER_SOURCE_DIRECTORY/GoGoGiphy/GoGoGiphy.UITest/bin/Debug

--uitest-tools-dir /Users/vsts/.nuget/packages/xamarin.uitest/2.2.6/lib/tools

--token $AppCenterToken

Listing 6-2

Full CLI Command

Feel free to customize the --devices argument to point to another device set or the --test-series argument if you want to call your test run another name; I used the iPhoneX device set that was customized earlier. Leave the other arguments as they exist now.

• Open the appcenter-post-build.sh file (adjacent to the solution file). Remember that this file is the post-build script used for the iOS build definition.

• Paste the above command to the end of the file. You can comment out the other commands with a # character. Personally, I leave earlier commands commented out because you may need them later.

• Save the file and commit it to your repository.

• Run the build if it is set to manual build.

• Wait for the build to reach the end, at which point it will execute the post-build script.

• If you configured the command correctly, you will see the start of your UI tests, similar to Figure 6-25.

  

• Remain patient because this step will take a long time. You will see the Current test status message repeat. This is normal because App Center runs the UI test on a physical device in your device set. App Center continues pinging this physical device to get the status of the UI test on a regular interval, hence the repeated output.

• Eventually, you will see the UI test finish, as shown in Figure 6-26.

Notice above how the output shows how many scenarios ran, how many passed, and how many failed. In this test run, my UI test failed. This test run failed because one test function is set up deliberately to fail. I will show you how to fix this test function written with Xamarin.UITest in a later section.

Viewing Test Run Results

Now head over to the Test Run section in App Center. You will view the test that just ran. Keep in mind that this is a real UI test that is set up to fail on purpose. You need to know the important areas of this section so you do not miss out on what may help you in the future when writing your own UI tests. Your Test Runs page should look like Figure 6-27.

Click the Test Run to view the details, as shown in Figure 6-28. Notice how App Center shows on how many devices the UI test ran. In this example, it failed on both devices.

Click Test Search, which is the name of my UI test. Figure 6-29 shows an actual screenshot of the app that ran on both devices.

Click one of the devices. You will see the detailed page, shown in Figure 6-30.

You can view the details of the device by clicking Details, in which case you may get a pop-up showing the specifications of the device that ran your test (Figure 6-31).

Click Logs at the top right. This is the most critical section because you will see three areas, shown in Figure 6-32.

The first section, Test Failures, will show the UI test that ran and what failed. In this example, the Assert command expected a True value but got a False value instead.

The above output is good but not good enough since you want to see why it failed. Click the Test Logs option. You can view the output in the webpage itself or click the Download test logs option (Figure 6-33).

The output may be difficult to see in the image, but the relevant lines that explain the reasons for the failure are as follows:

01-11-2018 03:17:24.352 +01:00 - 27335 - Entering text 'Iron Man' in element matching Marked("SearchBar") at coordinates [ 184.5, 113 ].

01-11-2018 03:17:25.019 +01:00 - 28002 - Pressing enter.

01-11-2018 03:17:25.234 +01:00 - 28217 - Query for Marked("CachedImage") gave 0 results.

Notice how the device shows the timestamp, the action that was taken, and the reason for the failure. In this simple example, I entered Iron Man in the search bar, instructed the UITest to press Enter, and instructed the test to search for any image marked with CachedImage from the results. It failed because it didn’t find any matching results.

If you do not own an iPhone X device yourself, you have just tested your first UI test on an iPhone X in the cloud! Now you can say you have tested a UI test officially in the cloud!

Configuring the Android UITest Command

Thankfully, you are most of the way through your journey in configuring the CLI command for Android because you can reuse most of the command parameters from iOS. Keep the other argument values the same. I will show only the argument values that you need to change and the one command you need to add.

Final Android UITest Command

After following all the above instructions, your final Bash commands should now look like Listing 6-4.

/Library/Frameworks/Mono.framework/Versions/5_12_0/bin/msbuild $APPCENTER_SOURCE_DIRECTORY/GoGoGiphy/GoGoGiphy.UITest/GoGoGiphy.UITest.csproj /t:Build /p:Configuration=Debug

appcenter test run uitest

--app "GoGoGiphyGroup/GoGoGiphy-Android"

--devices "GoGoGiphyGroup/GooglePixel3Android9"

--app-path $APPCENTER_OUTPUT_DIRECTORY/com.companyname.GoGoGiphy.apk

--test-series "master"

--locale "en_US"

--build-dir $APPCENTER_SOURCE_DIRECTORY/GoGoGiphy/GoGoGiphy.UITest/bin/Debug

--uitest-tools-dir /Users/vsts/.nuget/packages/xamarin.uitest/2.2.6/lib/tools

--token $AppCenterToken

Listing 6-4

Add Caption

Similar to how you tested the post-build script, feel free to change a few things in the above command, which may be different in your setup. My device set is called GooglePixel3Android9; change it to your own device set name. If you labeled your test series to another name, change the name as well.

Follow the steps below to save and run your final CLI command:

• Open the appcenter-post-build.sh file next to the Android .csproj file.

• Paste the above msbuild command before the ls commands. You need to paste it before these commands so you can verify that the Bin/Debug folder is created when the project is built.

• Paste the UITest command after the ls commands. You can comment out the ls commands with the # character if you are confident that the UITest command will work. I would leave it uncommented if I were you for troubleshooting purposes.

• Save and commit the file into the repository.

• Run the build.

If everything was set up correctly, your build output will look like Figure 6-34.

Notice how the output is similar to the iOS output. The only difference here is the accepted devices where the UI tests are run, in this case, Google Pixel 3 and Pixel 3 XL.

Viewing Test Run Results

Similar to the iOS test runs, return to the Test runs sections for the Android app. Your test run may look like Figure 6-35.

Notice how the same UI test succeeded for both Android devices but failed for iOS. This difference reveals the power of App Center because you have just proven that the same test can yield different results in different devices. If you attempted to reproduce the same result on physical devices, you would need to purchase at least two separate devices, one for iOS and one for Android, to notice that your app behaves differently. Now imagine how much money you can save with App Center!

Follow the same steps you performed for the iOS test run. Click Test Search, click one of the devices, and click Test Logs (Figure 6-36).

The following are the contents of the log message:

Android test running Xamarin.UITest version: 2.2.6

02-11-2018 02:28:50.819 +01:00 - 216 - Using XAMARIN_TEST_CLOUD configuration override.

02-11-2018 02:28:52.719 +01:00 - 2117 - Initializing Android app with apk: /Volumes/Data/xamarin/workspaces/25372e3c-44b3-4597-afa1-e0bcd4bd3f80/workspace/app.apk

02-11-2018 02:30:01.830 +01:00 - 71229 - Using element matching Marked("SearchBar").

02-11-2018 02:30:01.832 +01:00 - 71231 - Tapping coordinates [ 709, 706 ].

02-11-2018 02:30:04.299 +01:00 - 73699 - Pressing enter.

02-11-2018 02:30:04.734 +01:00 - 74133 - Query for Marked("CachedImage") gave 6 results.

Notice how the UITest detects six objects with the CachedImage identifier, which is why the test succeeds.

Whew! If you got this far and got both your iOS and Android test runs to execute successfully, congratulate yourself on a spectacular job! Now you hold the crown of officially running UI tests on both iOS and Android in the cloud!

Running UI Tests Asynchronously

One thing you may notice while running your UI tests is that your builds now take a long time to run. You may want to start your build and simply want to see your build finish and trigger the test run instead of waiting for the test runs to finish while holding up the build.

Save your post-build script, commit to the repository, run the build, and wait for the build to finish. The output should look like Figure 6-37.

Notice a couple of important details in Figure 6-37. First, it runs your UI tests the same as before. Second, it provides a link to the Test Report that you can follow to view the results of the test run, which is the same webpage I showed you before. Third, the build actually succeeds and finishes a lot sooner than before.

Now your build success is not dependent on the results of your test run. It is a matter of developer preference whether the build should be dependent on the results of the test run. Just keep in mind the trade-off. If the build depends on the test run result, your build will take longer. If you want your builds to finish sooner, your builds will succeed regardless of the outcome of your test run, even a failed test run.

Using REPL (Read-Eval-Print-Loop)

If you remember, the same UI test succeeded for Android devices but not for iOS devices. This difference in results showed you the power of testing in App Center without having to purchase physical devices to run the same tests.

But why did iPhone devices give a different result in the first place? The answer is simple. Return to the solution in Visual Studio. Open HomePage.xaml. Collapse the content. You can press Control+M and Control+L to toggle the expansion and collapsing of all XAML elements. Scroll down the XAML page until you reach a control called PullToRefreshLayoutControl and the grid below it. Notice how the IsVisible bindable property on this control is bound to IsTrendingImagesPanelVisible. This property in the ViewModel collapses the grid and expands the PullToRefreshLayoutControl after the user presses Enter on the search bar. On Android, this set of actions completes in time before reaching the Assert statement in the UI test. On iOS, this set of actions do not complete in time, which is why the Assert statement returns false and the UI test fails.

How can you fix this? First, I will show you a handy little console tool that the App Center team has provided for you. It is called REPL and it stands for Read-Eval-Print-Loop. If you have experience using the C# Interactive tool or the PowerShell Interactive tool within Visual Studio, you know what REPL is about. It allows you to test or read the commands, evaluate them, and print out the results in real time, hence the name REPL. Once you have tested your commands, you can then copy the commands over into your application. Now App Center provides you with this handy tool to make writing UI tests easier by giving you the ability to test out UI test commands, evaluate your commands on your app in the simulator, and print out the results.

You can find more information about the important commands at https://docs.microsoft.com/en-us/appcenter/test-cloud/uitest/working-with-repl .

Using app.Repl()

Here are the steps for diagnosing, troubleshooting, and fixing this UI test. Open HomePageTests.cs in the UITest project. As mentioned, TestSearchBroken() is the intentionally failing UI test function. The working example called TestSearchWorking() is below it.

Feel free to create another UI test function or modify TestSearchBroken(). But I want to show you how to diagnose the broken example to arrive at the final working result in TestSearchWorking().

See Listing 6-6 and add the command and strip out the existing commands. Verify that your code looks like it now.

/// <summary>

/// This UITest fails on purpose on iPhone devices to show the working example in the next UITest function below.

/// </summary>

[Test]

public void TestSearchBroken()

{

  app.Repl();

}

Listing 6-6

Add Caption

The call to app.Repl() opens the terminal window for the Repl console application and allows you to experiment and play with various UI commands. It will evaluate the console commands and display the results to the user.

Follow these steps in order to run the Repl tool:

• Rebuild the UITest project.

• Go to Test Explorer and verify the tests appear.

• Go to Tools ➤ Android ➤ Android Device Manager.

• Launch one of your preconfigured Android simulators. Wait for it to finish launching.

• Return to Test Explorer. Expand HomePageTests for Android.

• Add the following command either to TestSearchBroken() or to your own UI test function:

  app.Repl();

• Right-click TestSearchBroken and either run selected tests or debug selected tests. If you select debug, you can insert breakpoints and step through you UI test functions.

At this point, it will launch the app in the Android simulator and call your selected UI test function. Once it runs app.Repl(), it will launch the Repl console application shown in Figure 6-38.

Using the Tree Command

Type the tree command in the command-line window. The tree command shows you a hierarchical list of all the views visible on the screen. You can use this command to list out all the views. After running this command, scroll down until you see a list of controls with the CachedImage labels, as shown in Figure 6-39.

Final UI Test Commands

You are going to perform one final modification to your command before committing these commands as your final test.

.NET provides the ability to provide additional parameters to the WaitForElement(..) function. Feel free to play with the parameters. But I am going to use the following command. It instructs WaitForElement(..) to display a custom message if the element is not displayed in time. And it instructs the function to wait up to 2 minutes, even though the element will display a lot sooner than 2 minutes.

app.WaitForElement("CachedImage", "The CachedImage control was not available in time", new TimeSpan(0, 0, 2, 0, 0));

Your final commands should now look similar to the following:

app.EnterText("SearchBar", "IronMan");

app.PressEnter();

app.WaitForElement("CachedImage", "The CachedImage control was not available in time", new TimeSpan(0, 0, 2, 0, 0));

Save your changes to HomePageTests.cs. Commit and push your changes to the remote repository.

Once your app finishes building and running the UI tests, both of your test functions will now run successfully for iOS, as shown in Figure 6-40.

Summary

This is your accomplishment! Of course, if you jumped around from chapter to chapter instead of going sequentially or if some of the topics are a little hazy, feel free to revisit any chapter and repeat the material.

I structured the topics in this book to be valuable not just to Xamarin Forms developers but developers from other platforms. Most of the topics are platform-agnostic. Xamarin Forms was a nice fit for me given my background with this platform. Also, Xamarin Forms is the most popular cross-platform framework at the time of this writing, but it could have been done just as easily with React Native, Cordova, or Unity.

If you have any outstanding questions, feel free to contact me at my LinkedIn profile at www.linkedin.com/in/sunnymukherjee/ and I will be happy to answer your questions.

I would like to conclude this chapter and this book with a big thank you to you, the reader, for buying this book. I hope the material in this book has shed light on the next evolution of mobile DevOps using App Center. And I hope this book helps you and your team incorporate App Center into your existing DevOps pipeline.

Congratulations, thank you, and good luck in your journey ahead with App Center!