Chapter 7: Testing Your Apps

In the previous chapters, we covered developing multiple different types of apps using Uno Platform. Uno Platform not only allows apps to be written, however; it also allows automated UI tests to be written that will run on Android, iOS, and WebAssembly using the Uno.UITest framework. During this chapter, we will write our first test using Uno.UITest and run it on different platforms, including using emulators. After this, you will also learn how to write tests for Windows using WinAppDriver.

In this chapter, we'll cover the following topics:

  • Setting up the Uno.UITest project for your app
  • Authoring Uno.UITest tests for your Uno Platform app
  • Running your tests against the WASM, Android, and iOS versions of your app
  • Writing unit tests for your Uno Platform app
  • Using WinAppDriver to author automated tests for the UWP head of your app
  • Why manual testing is still important

By the end of this chapter, you'll have learned how to write tests using Uno.UITest and WinAppDriver for your app, how to run those tests on different platforms, and why manually testing your app is still important.

Technical requirements

This chapter assumes that you already have your development environment set up, including installing the project templates, as was covered in Chapter 1, Introducing Uno Platform. The source code for this chapter is available at https://github.com/PacktPublishing/Creating-Cross-Platform-C-Sharp-Applications-with-Uno-Platform/tree/main/Chapter07.

The code in this chapter makes use of the library from https://github.com/PacktPublishing/Creating-Cross-Platform-C-Sharp-Applications-with-Uno-Platform/tree/main/SharedLibrary.

Check out the following video to see the code in action: https://bit.ly/3iBFZ2e

Getting started with Uno.UITest

Before we get started with Uno.UITest, let's cover what Uno.UITest is and what its aim is. Uno.UITest is a library developed and maintained by the Uno Platform team to allow developers to write unified UI tests for their Uno Platform apps. These UI tests allow you to simulate users interacting with your app and verify the UI of your app to ensure that user interactions work correctly and that your app behaves as designed. With Uno.UITest, you can write UI tests (also sometimes referred to as interaction tests), which you can run against the Android, iOS, and WASM heads of your app.

Under the hood, Uno.UITest uses Xamarin.UITest to run tests against the Android and iOS head of the app. For the WASM version of the app, Uno.UITest uses Selenium and Google Chrome. Using these libraries, Uno.UITest allows you to write tests that mimic user interactions with the UI of your app, including mimicking mouse input such as clicking, and keyboard input such as entering text.

But when should you use UI tests? When writing complex apps, ensuring that changes to your code did not break existing features can often be difficult to test, especially as some changes only become noticeable when using the app, not when testing components or classes alone. UI tests are ideal for this kind of scenario as you can write tests simulating a normal user using your app without having to manually go through dozens or hundreds of steps. A common scenario for writing UI tests is to check whether users can successfully achieve certain tasks within your app, for example, sign in to your app or search for a specific thing. While UI tests are good for testing these kinds of scenarios, UI tests are no silver bullet and also have their drawbacks. Since UI tests simulate user input, they are slower to run compared to normal unit tests, which are only testing single objects or classes. In addition to this, since the UI testing framework or library needs to find a way to interact with your app, UI tests can sometimes break when updating the UI of your app or changing texts or names within your app.

Nonetheless, when developing an app, writing UI tests is very important when you try to ensure that no bugs have snuck into the app. This is especially useful when writing apps that will run on a variety of different devices having different screen sizes, capabilities, and OS versions as this makes it easier to test your app on a lot of different configurations as manual testing is slow and error-prone.

Before we use Uno.UITest, we first need an app that we can use to write tests for. For this, let's start by creating a new solution for our app that we will use to write tests for:

  1. Create a new project with the Multi-Platform App (Uno Platform) template.
  2. Name the project UnoAutomatedTestsApp. Of course, you can use a different name; however, in this chapter, we will assume that the project is named UnoAutomatedTestsApp.
  3. Remove all the platform head projects except for Android, iOS, UWP, and WASM.
  4. Now we need to add a reference to our shared library. To do this, right-click on the solution file, select Add > Existing Project…, navigate to the UnoBookRail.Common.csproj file, and then click Open.
  5. Reference the shared library project in every head project. For this, right-click on the head project, select Add > Reference… > Projects, check UnoBookRail.Common, and click OK. Since we need a reference to the library in every head, repeat this process for every head, in other words, Android, iOS, UWP, and WASM.

Now that we have created the project, let's add some content to our app that we can test:

  1. Add xmlns:toolkit="using:Uno.UI.Toolkit" to the Page element at the root of MainPage.xaml.
  2. Replace the Grid control inside your MainPage.xaml file with the following:

    <StackPanel Spacing="10" Padding="10"

        toolkit:VisibleBoundsPadding.PaddingMask="All"

        Background="{ThemeResource

            ApplicationPageBackgroundThemeBrush}">

        <StackPanel x:Name="SignInForm" Spacing="10">

            <TextBox x:Name="UsernameInput"

                AutomationProperties.AutomationId=

                    "UsernameInput"

                TextChanged="Username_TextChanged"

                    Header="Username"/>

            <PasswordBox x:Name="PasswordInput"

                AutomationProperties.AutomationId=

                    "PasswordInput"

                PasswordChanged="Password_PasswordChanged"

                    Header="Password"/>

            <TextBlock x:Name=

                "SignInErrorMessageTextBlock"

                    AutomationProperties.AutomationId="

                        SignInErrorMessageTextBlock"

                Foreground="{ThemeResource

                    SystemErrorTextColor}"

                        Visibility="Collapsed"/>

            <Button x:Name="SignInButton"

                AutomationProperties.AutomationId=

                    "SignInButton"

                Click="SignInButton_Click"

                    Content="Sign in" IsEnabled="False"

                HorizontalAlignment="Center"

                    BorderThickness="1"/>

        </StackPanel>

        <TextBlock x:Name="SignedInLabel"

            AutomationProperties.AutomationId=

                "SignedInLabel"

            Text="Successfully signed in!"

                Visibility="Collapsed"/>

    </StackPanel>

  3. This is a simple sign-in interface that we will write tests for later in this chapter. It includes sign-in controls, a sign-in button, and a label that will be shown when signed in.
  4. Now, add the following two methods to the MainPage class:

    using UnoBookRail.Common.Auth;

    ...

    private void Username_TextChanged(object sender, TextChangedEventArgs e)

    {

        SignInButton.IsEnabled = UsernameInput.Text.Length

            > 0 && PasswordInput.Password.Length > 0;

    }

    private void Password_PasswordChanged(object sender, RoutedEventArgs e)

    {

        SignInButton.IsEnabled = UsernameInput.Text.Length

            > 0 && PasswordInput.Password.Length > 0;

    }

    private void SignInButton_Click(object sender, RoutedEventArgs args)

    {

        var signInResult = Authentication.SignIn(

            UsernameInput.Text, PasswordInput.Password);

        if(!signInResult.IsSuccessful &&

            signInResult.Messages.Count > 0)

        {

            SignInErrorMessageTextBlock.Text =

                signInResult.Messages[0];

            SignInErrorMessageTextBlock.Visibility =

                Visibility.Visible;

        }

        else

        {

            SignInErrorMessageTextBlock.Visibility =

                Visibility.Collapsed;

            SignInForm.Visibility = Visibility.Collapsed;

            SignedInLabel.Visibility = Visibility.Visible;

        }

    }

    This code adds handlers that allow us to enable the sign-in button as soon as the user has entered a username and password. Otherwise, the sign-in button will be disabled. In addition to this, we also handle the sign-in button click and update the UI accordingly, including showing the error message if the sign-in failed.

Now, if you start the UWP head of your app, you will see something like Figure 7.1:

Figure 7.1 – Screenshot of the running app with the sign-in form

Figure 7.1 – Screenshot of the running app with the sign-in form

Now that we have added a simple test app that we can test again, we can now add our Uno.UITest tests project:

  1. If you want to run tests for the WASM head of your app, make sure you have Google Chrome installed.
  2. First, you will need to update the project files for the Android, iOS, and WASM heads. For that, add the following entries before the last closing project tag of the .csproj files for those projects:

    <PropertyGroup Condition="'$(Configuration)'=='Debug' or '$(IsUiAutomationMappingEnabled)'=='True'">

        <IsUiAutomationMappingEnabled>

            True</IsUiAutomationMappingEnabled>

        <DefineConstants>$(DefineConstants);

            USE_UITESTS</DefineConstants>

    </PropertyGroup>

  3. For the iOS project, add a reference to the Xamarin.TestCloud.Agent NuGet package. Since, as of the time of writing, the latest stable version was 0.22.2, we will use that.

    Inside the OnLaunched method of your App.xaml.cs file, add the following at the beginning of the method:

    #if __IOS__ && USE_UITESTS

          // Launches Xamarin Test Cloud Agent

          Xamarin.Calabash.Start();

    #endif

    Since the Uno.UITest library uses Xamarin.UITest under the hood, for the iOS app, we need to add the preceding code. Otherwise Xamarin.UITest can't interact with the running iOS app and the tests won't work.

  4. Since the Uno.UITest project type is not included in the Uno Platform Visual Studio templates extension, make sure you have the Uno Platform dotnet new templates installed. You can find the instructions for this in Chapter 1, Introducing Uno Platform.
  5. Inside the UnoAutomatedTestsApp folder, create a folder named UnoAutomatedTestsApp.UITests.
  6. Inside the newly created folder, run the following command:

    dotnet new unoapp-uitest

    This will create a new Uno.UITest project inside the folder and also add the project to the solution file.

  7. Update the Android and iOS apps package names. For Android, replace the package entry inside the Properties/AndroidManifest.xml file of the Android project with package="UnoBook.UnoAutomatedTestsApp". To replace the iOS package name, open the Info.plist file inside the iOS project and replace Bundle Identifier with UnoBook.UnoAutomatedTestsApp.
  8. Now we need to update the Constants.cs file inside the Uno.UITests app project to point to the correct app. For this, replace lines 13, 14, and 15 with the following:

    public readonly static string iOSAppName = "UnoBook.UnoAutomatedTestsApp";

    public readonly static string AndroidAppName = "UnoBook.UnoAutomatedTestsApp";

    public readonly static string WebAssemblyDefaultUri = "http://localhost:[PORT]/";

    Since the port of the WASM app is generated randomly, replace [PORT] from the preceding code with the following information

    Note

    We need to update the Constants.cs file since Uno.UITest needs to be able to find the app through the app name or app URI in the case of WASM. To find out which URI your WASM head is running on, open Properties/launchSettings.json inside the WASM head. Inside there, depending on whether you will be using the IIS Express or the [ProjectName].Wasm target, either use applicationUrl from the iisSettings option or applicationUrl from the [Project name].Wasm profile to ascertain the port. In this chapter, we will be using IIS Target. The iOS app name is defined by the Bundle identifier inside the Info.plist file located inside the iOS project. For the Android app name, refer to the package property inside the Properties/AndroidManifest.xml file of the Android project.

Inside the UnoAutomatedTestsApp.UITests project, you will find three files:

  • Constants.cs: This contains the configuration to find the running app using the app package name or URL of the app, as explained earlier.
  • Given_MainPage.cs: This is a sample test file with a small test showing how to write a test.
  • TestBase.cs: This file contains all the bootstrap code that takes care of starting and tearing down the app and also exposes an IApp instance (more on this in the next section). This file also exports a TakeScreenshot function that you can use to take screenshots of the running app being tested.

Now that we have covered how to set up the Uno.UITest project and its structure, let's continue by writing our first Uno.UITest and learn how to run those tests.

Writing and running your first test

Before we start writing our first test, we will cover how you can use Uno.UITest to interact with your apps. For this, we will first start by covering the basics of the addressing elements using Uno.UITests query feature objects.

How Uno.UITest works

Since UI tests need to address UI elements of your app, every UI testing library needs to have a way to allow developers to address those elements. Uno.UITest does this using the IAppQuery interface to define queries and the IApp interface to run those queries and inject input.

The IApp interface provides you with the necessary APIs to interact with your app, including clicking elements, simulating scrolling, and injecting text input. As part of the creation of the Uno.UITest project, the TestBase class will provide you with an IApp instance. Since the IApp interface allows you to simulate input to your app and most interactions require a specific control to be the target of your interaction, most methods on the IApp interface require you to specify the AutomationID property of the control or by using the IAppQuery interface.

In the following example, we will use AutomationID to click the button, as defined by the following XAML:

<!-- Setting AutomationId to reference button from UI test -->

<Button AutomationProperties.AutomationId="SignInButton Content="Sign in"/>

When writing a Uno.UITest test, we can then press the button using the following code:

App.Tap("SignInButton");

In contrast to using x:Name/AutomationID of a control to specify the element, by using the IAppQuery interface, you can address controls based on other properties, for example, their type, or based on specific properties being set on a control. When working with IAppQuery, you will notice that the IApp interface does not expect to get an element of the IAppQuery type, but rather an element of the Func<IAppQuery,IAppQuery> type. Since the IApp interface relies heavily on this, you will often see the following using-alias statement:

using Query=System.Func<Uno.UITest.IAppQuery,Uno.UITest.IAppQuery>;

This allows developers to write queries more easily since you can simply use the Query type alias instead of having to write it out every time. For simplicity, in this chapter, we will also use this using statement and use the Query type.

If we take the XAML from before, pressing the button with the IAppQuery interface can be done as follows:

Query signInButton = q => q.Marked("SignInButton");

App.Tap(signInButton);

When we created the Uno.UITest project, you may have also noticed that a reference to the NUnit NuGet package was added. By default, Uno.UITest uses NUnit for assertions and their tests. Of course, this does not mean that you have to use NUnit for your tests. However, if you wish to use a different testing framework, you will need to update the TestBase.cs file since it uses NUnit attributes to hook into the setup and teardown of the tests.

Now that we covered the basics of how Uno.UITest works, we will now continue by writing tests for our sign-in interface.

Authoring your first test

We will start by writing our first tests for the sign-in interface we added at the start of this chapter. For simplicity, we will use NUnit since Uno.UITest uses this by default when creating a new Uno.UITest project, meaning that we don't have to update the TestBase class. We begin by creating a new file for our tests:

  1. First, remove the existing Given_MainPage.cs file.
  2. Create a new folder called Tests.
  3. Create a new class called SignInTests.cs inside the Tests folder.
  4. Update SignInTests.cs with the following code:

    using NUnit.Framework;

    using Query = System.Func<Uno.UITest.IAppQuery, Uno.UITest.IAppQuery>;

    namespace UnoAutomatedTestsApp.UITests.Tests

    {

        public class SignInTests : TestBase

        {

        }

    }

    We are inheriting from TestBase to access the IApp instance of the current test run and to be able to send input to our app. In addition to that, we are also adding a using statement for the NUnit library as we will use it later and add the named using statement we covered in the section How Uno.UITest works.

Now, let's add our first test. Let's start by simply checking whether the email and password input fields and the sign-in button exist. For the rest of this section, we will only be working inside the SignInTests.cs file since we are writing tests for the sign-in user interface:

  1. Start by adding a new public function, which will be our test case. We will name the function VerifySignInRenders.
  2. Add the Test attribute. This lets NUnit know that the function is a test.
  3. Now, add the following code inside the function:

    App.WaitForElement("UsernameInput");

    App.WaitForElement("PasswordInput");

    App.WaitForElement("SignInButton");

Your SignInTests class should now look something like this:

public class SignInTests : TestBase

{

    [Test]

    public void VerifySignInRenders()

    {

        App.WaitForElement("UsernameInput", "Username input

            wasn't found.");

        App.WaitForElement("PasswordInput", "Password input

            wasn't found.");

        App.WaitForElement("SignInButton", "Sign in button

            wasn't found.");

    }

}

Now what our test does is try to find the elements with the automation ID UserNameInput, PasswordInput, and SignInButton, and fail the test if it can't find any of those elements.

Now that we have written our first test, let's try it out! To do this, we'll first cover how to run those tests.

Running your tests on Android, iOS, and WASM

Running your Uno.UITest tests against the Android, iOS, and WASM head of your app is fairly simple, although the process is always slightly different depending on what platform you are trying to start.

Running tests against the WASM head

Let's start by running our test against the WASM head of our app:

  1. First, you will need to deploy the WASM head of the app. For this, select the UnoAutomatedTestsApp.Wasm start up project and select the IIS Express target, as shown in Figure 7.2. Then, press Ctrl + F5, which will deploy the project.
    Figure 7.2 – WASM project with IIS Express selected

    Figure 7.2 – WASM project with IIS Express selected

  2. Update Constants.cs and change Constants.CurrentPlatform to Platform.Browser. If you haven't updated the Constants.WebAssemblyDefaultUri property, do that as in the Getting started with Uno.UITest section.
  3. Open Test Explorer by clicking View in the menu bar and clicking on Test Explorer. Now, expand the tree and right-click the VerifySignInRenders test. Click the Run option from the popup. Now, the test will run against the app running in Chrome.

    Important note

    At the time of writing, due to a known bug with Uno.UITest, running the tests against the WASM head might not work as Chrome might fail to start. Unfortunately, no workaround is known yet. To learn more about the current state of this bug, refer to the following GitHub issue: https://github.com/unoplatform/Uno.UITest/issues/60.

Once the tests have started, Chrome will be started in headless mode and once the tests have finished, the test will be marked as passed in the Visual Studio Test Explorer.

Running tests against the Android version of your app

In addition to running your tests against the WASM head, you can also run the tests against the Android version of your app running on an emulator or running on an Android device. To do this, follow these steps:

  1. Ensure that Android Emulator is running and that the app has been deployed. To deploy the Android version of your app, select the Android project as the start up project and press Ctrl + F5. If you want to run the tests against the app running on your Android device, make sure the app is deployed on the device and that your device is connected to your computer.
  2. Update Constants.cs and change Constants.CurrentPlatform to Platform.Android. In case you haven't updated the Constants.AndroidAppName property, do that as in the Getting started with Uno.UITest section.
  3. As was the case with WASM, now right-click the test in Test Explorer and click on Run. The app will start inside the emulator or on your Android device and the tests will be running against the running Android app.

Running tests against the iOS version of your app

You can also run your UI tests against the iOS version of your app running on an emulator or on an iOS device. Note that macOS is required for this. To run the tests against the iOS head, follow these steps:

  1. Ensure that the iOS simulator is running and that the app has been deployed. To deploy the iOS version of your app, select the iOS project as the start up project and run the app. If you want to run the tests against the app running on your iOS device, make sure the app is deployed on the device and that it is connected to your computer.
  2. Update Constants.cs and change Constants.CurrentPlatform to Platform.iOS. Set iOSDeviceNameOrId to the name of the emulator or tethered device you wish to use.

    If using a tethered device, you may also need to change iOSAppName and the Bundle Identifier in info.plist so that it is compatible with your developer certificate.

  3. Now, right-click the test project in the Tests window and click on Run Test. The app will start and the tests will be run.

    Additional information

    Running the UI tests on a mac requires having compatible versions of the test libraries, tools, and OS versions. If you encounter errors when running the tests, ensure you have the latest versions of OS X, Xcode, Visual Studio for Mac, and the NuGet packages you are using in the test project. You may also need to ensure that the device or simulator you are running against is the latest iOS version (including any updates).

    Running the UI tests on a simulator can be resource-intensive. You may find it necessary to run the tests on a connected device if they don't start on the simulator.

    If testing on a physical device, UI automation must be enabled. Enable this at Settings > Developer > UI Automation.

    Hopefully, more documentation will be added that will make testing and debugging tests on a Mac easier. For progress on this, see https://github.com/unoplatform/Uno.UITest/issues/66.

Now that we have covered how to run your tests against the Android, iOS, and WASM versions of the app, we will dive deeper into writing tests by writing more UI tests for our sign-in interface.

Writing more complex tests

So far, we have only tested the very basic example of our sign-in interface rendering. However, we also want to make sure that our sign-in interface actually works and allows users to sign in. For this, we will write a new test that ensures that when a username and password are being provided, the sign-in button is clickable:

  1. Create a new function, VerifyButtonIsEnabledWithUsernameAndPassword, inside the SignInTests.cs file and add the Test attribute to it.
  2. Since we will use those queries more often, add the following Query objects to the SignInTests class:

    Query usernameInput = q => q.Marked("UsernameInput");

    Query passwordInput = q => q.Marked("PasswordInput");

    Query signInButton = q => q.Marked("SignInButton");

  3. Now, let's simulate entering text in the username and password fields by inserting the following code into the VerifyButtonIsEnabledWithUsernameAndPassword test:

    App.ClearText(usernameInput);

    App.EnterText(usernameInput, "test"); App.ClearText(passwordInput);

    App.EnterText(passwordInput, "test");

    Important note

    Due to a bug with Xamarin.UITest, the testing library Uno.UITest uses for Android and iOS, clearing and entering tests does not work on every Android device or emulator. You can find more information on this bug here: https://github.com/microsoft/appcenter/issues/1451. As a workaround, you can use an Android emulator with API version 28 or lower as those Android versions are not affected by this bug.

    This will simulate a user entering the text test into the username input field and the same text into the password input field. Note that in this and the following tests, we will always clear the text beforehand to ensure that the correct text has been entered.

    Note

    When running multiple tests as a group, for example, by selecting multiple tests or their root node in the Test Explorer, Uno.UITest will not reset the app between the individual tests. That means that you will need an initialization code for your tests if those rely on a specific initial app state.

  4. Now, let's verify that the sign-in button is enabled by using the following code:

    var signInButtonResult = App.WaitForElement(signInButton);

    Assert.IsTrue(signInButtonResult[0].Enabled, "Sign in button was not enabled.");

    For this, we ensure that the button exists and grab the IAppResult[] object for that query. We then check that the button is enabled through the IAppResult.Enabled property. Note that we added a message to the assert that will be displayed when the assert fails by providing a second parameter.

Now, if you run the test for Android, the app will start on your Android device or the emulator. Uno.UITest will then enter text inside the Username and Password input fields and you should see the sign-in button become clickable.

Let's now test whether invalid sign-in credentials provide a meaningful error message. For this, we will write a new test:

  1. Create a new function, VerifyInvalidCredentialsHaveErrorMessage, inside the SignInTests.cs file and add the Test attribute to it.
  2. Now, add a new query to the SignInTests class that we will use to access the error message label:

    Query errorMessageLabel = q => q.Marked("SignInErrorMessageTextBlock");

  3. Now, let's enter credentials that are definitely invalid and press the Sign in button using the following code:

    App.ClearText(usernameInput);

    App.EnterText(usernameInput, "invalid");

    App.ClearText(passwordInput);

    App.EnterText(passwordInput, "invalid");

    App.Tap(signInButton);

  4. Since we will be using Uno.UITest extensions methods and Linq inside our test, add the following using statements:

    using System.Linq;

    using Uno.UITest.Helpers.Queries;

  5. Lastly, we need to verify the error message using the following code. By that, we check that the error label is displaying the appropriate error message:

    var errorMessage = App.Query(q => errorMessageLabel (q).GetDependencyPropertyValue("Text").Value<string>()).First();

    Assert.AreEqual(errorMessage, "Username or password invalid or user does not exist.", "Error message not correct.");

  6. If you run this test now, you will see how the username "invalid" and the password "invalid" will be entered. After that, the test clicks on the sign-in button and you will see the error message Username or password invalid or user does not exist..

Lastly, we want to verify the fact that with valid credentials, users can sign in. For this, we will use the username demo and the password 1234, as these are known to the authentication code as a demo user:

  1. As with the previous tests, create a new function with the name VerifySigningInWorks inside the SignInTests.cs file and add the Test attribute to it.
  2. Since we will use the SignedInLabel to detect whether we are signed in, add the following query as we will use it later to detect whether the label is visible.
  3. Add the following code to enter the demo user credentials and sign in:

    App.ClearText(usernameInput);

    App.EnterText(usernameInput, "demo");

    App.ClearText(passwordInput);

    App.EnterText(passwordInput, "1234");

    App.Tap(signInButton);

  4. Lastly, check whether we are signed in by verifying that the signed-in label is visible and displaying the correct text using the following code:

    var signedInMessage = App.Query(q => signedInLabel(q).GetDependencyPropertyValue("Text").Value<string>()).First();

    Assert.AreEqual(signedInMessage, "Successfully signed in!", "Success message not correct.");

  5. If you run this test, you will see how the username demo and the password 1234 have been entered. After the sign-in button gets clicked by the test, the sign-in form will disappear and you will see the text Successfully signed in.

While we covered writing tests using Uno.UITest, of course, we didn't cover all the available APIs. Figure 7.3 shows a list of different APIs available as part of Uno.UITest and how you can use them:

Figure 7.3 – List of additional APIs available as part of Uno.UITest

Figure 7.3 – List of additional APIs available as part of Uno.UITest

Now that we have covered writing tests using Uno.UITest, let's look at tools you can use to write automated tests for your app, including using WinAppDriver to write UI tests for the UWP head of your app.

Test tools beside Uno.UITest

Uno.UITest is not the only tool you can use to write automated tests for your Uno Platform app. In this section, we will cover writing UI tests for the UWP head of your project using WinAppDriver and Selenium and writing unit tests for the UWP head of the project.

Testing the UWP head of your app with WinAppDriver

At the time of writing, Uno.UITest does not support running the tests against the UWP head of your app. However, you might also want to run UI tests against the UWP version of your app. Luckily, WinAppDriver and Appium allow us to achieve this. WinAppDriver is a tool developed by Microsoft that allows developers to simulate input to Windows apps, including UWP apps. While WinAppDriver allows you to interact with Windows apps, it does so by starting a web server locally and allows interaction with apps by communicating with WinAppDriver through a web-based protocol. To make the development process easier for us, we will use Appium.WebDriver as our library to write the UI tests. We will start by creating our test project and adding the necessary tests. Note that we will be creating a new project since we don't want Appium.WebDriver to interfere with Uno.UITest and we can't use Appium and WinAppDriver from inside a UWP project, meaning we can't reuse our UWP Unit test project:

  1. First, you will need to install WinAppDriver. For this, go to the releases page of WinAppDriver (https://github.com/Microsoft/WinAppDriver/releases) and download the latest MSI installer. At the time of writing, the latest stable release was version 1.2.1, and we will be using this version for this chapter. After downloading the MSI installer, run it to install WinAppDriver. Note that you will later need to start the WinAppDriver.exe file and if you install WinAppDriver in a different folder, you should make a note of the installation folder.
  2. Open the UnoAutomatedTestsApp solution and create a new Unit Test project. To do this, right-click on the solution node and click Add > New Project.
  3. In the dialog, search for Unit Test App and select the option highlighted in Figure 7.4:
    Figure 7.4 – Unit Test Project template in the new project dialog

    Figure 7.4 – Unit Test Project template in the new project dialog

  4. Press Next and enter the project name. We will name the project UnoAutomatedTestsApp.UWPUITests. Of course, you can name the project differently; however, we will assume that the project is named UnoAutomatedTestsApp.UWPUITests in this chapter. Then, press Next.
  5. Now, select the target framework; we will be using .NET 5.0. Now, click Create to create the project.
  6. Once the project is created, right-click the project in the solution view and click on Manage NuGet Packages…. Now, install the Appium.WebDriver package by searching for Appium.WebDriver in the Browse section and installing the package.

Now that we have created the unit test project, we can write our first UI test using Appium.Webdriver. We will only cover how to write your first test using Appium and WinAppDriver. You can find more information about WinAppDriver and writing tests in their official documentation:

  1. Before we write our first test, first rename the UnitTest1.cs file to SignInTests.cs. and also rename the UnitTest1 class to SignInTests.
  2. Open the Package.appxmanifest file located inside the UWP head of the app and change the package name located under Packaging to UnoAutomatedTestsApp. Now, deploy the UWP head of your app by selecting the UWP head and pressing Ctrl + F5. Since we have changed the package name, we want the test to start the app using the updated package name.
  3. Add the following using statements to the SignInTests class:

    using OpenQA.Selenium.Appium;

    using OpenQA.Selenium.Appium.Windows;

  4. Now, add the following code to the SignInTests class.

    private static WindowsDriver<WindowsElement> session;

    [AssemblyInitialize]

    public static void InitializeTests(TestContext _)

    {

        AppiumOptions appiumOptions = new AppiumOptions();

        appiumOptions.AddAdditionalCapability("app",

           "UnoAutomatedTestsApp_cdfyh0xbha7kw!App");

        appiumOptions.AddAdditionalCapability(

           "deviceName", "WindowsPC");

        session = new WindowsDriver<WindowsElement>(new

            Uri("http://127.0.0.1:4723"), appiumOptions);

    }

    This will start the app using the app's package identity and connect to the running WinAppDriver. Since we will use the created WindowsDriver object to interact with the app, we store a reference to it. Note that the highlighted section will be different for your app. To get the correct value, open the Package.appxmanifest file and open the Packaging tab. Then, replace the highlighted part with the Package family name value.

  5. Now, remove the existing TestMethod1 test and add the following test:

    [TestMethod]

    public void VerifyButtonIsEnabledWithUsernameAndPasswordUWP()

    {

        var usernameInput =

            session.FindElementByAccessibilityId(

                "usernameInput");

        usernameInput.SendKeys("test");

        var passwordInput =

            session.FindElementByAccessibilityId(

                "passwordInput");

        passwordInput.SendKeys("test");

        var signInButton =

            session.FindElementByAccessibilityId(

                "signInButton");

        Assert.IsTrue(signInButton.Enabled, "Sign in

            button should be enabled.");

    }

    Like the VerifyButtonIsEnabledWithUsernameAndPassword test we wrote in the Uno.UITest section, this test verifies that when a username and password have been entered, the sign-in button is enabled.

Now that we have written our first test, let's run it! To do this, you will first need to start WinAppDriver. If you have installed WinAppDriver in the default folder, you will find the WinAppDriver.exe file in the C:Program Files (x86)Windows Application Driver folder. If you have chosen a different installation folder earlier, open that folder and start the WinAppDriver.exe file inside there. Upon starting, you should see something as shown in Figure 7.5:

Figure 7.5 – Window running WinAppDriver

Figure 7.5 – Window running WinAppDriver

Now, you can start the test by right-clicking the VerifyButtonIsEnabledWithUsernameAndPasswordUWP test inside the test explorer and clicking on Run. The test will start the app, enter the text, and then check whether the sign-in button is enabled.

Automated accessibility testing with Axe.Windows

In addition to writing normal UI tests, you can also add Axe.Windows to your testing suite to automatically check your app for accessibility issues as part of the UI testing strategy. Axe.Windows is a library developed and maintained by Microsoft that aims to detect accessibility issues in apps. Adding Axe.Windows to your UI tests is simple:

  1. Add a reference to the Axe.Windows package in the UnoAutomatedTestsApp.UWPUITests project. To do this, right-click the project and click on Manage NuGet Packages…. Search for Axe.Windows and install the package.
  2. Now, add the following two using statements to the SignInTests.cs file:

    using Axe.Windows.Automation;

    using System.Diagnostics;

  3. Lastly, add the following test to the SignInTests class:

    [TestMethod]

    public void VerifySignInInterfaceIsAccessible()

    {

        var processes = Process.GetProcessesByName(

            "UnoAutomatedTestsApp");

        Assert.IsTrue(processes.Length > 0);

        var config = Config.Builder.ForProcessId(

            processes[0].Id).Build();

        var scanner = ScannerFactory.CreateScanner(

            config);

        Assert.IsTrue(scanner.Scan().ErrorCount == 0,

            "Accessibility issues found.");

    }

Since Axe.Windows needs to know the process ID, we first get the process ID of the running app using the System.Diagnostics.Process API. We then create a new Axe.Windows configuration using the process ID, which we then use to create a new Axe.Windows scanner. The Axe.Windows scanner allows us to scan our app for accessibility issues using the Scan function. Since Scan() returns a scan result object telling us that all accessibility issues have been found, we assert that we have found zero accessibility errors. When writing UI tests for more complex apps, you would scan the app more often to ensure that every scenario and view inside your app will be covered by this accessibility scan. For example, you could scan the app for accessibility issues every time you navigate to a different view. If you now run the test, the test app will start and after a few seconds, the test will be marked as Passed since our sign-in interface has no accessibility issues that can be found by Axe.Windows.

In this section, we have only scratched the surface in terms of testing with WinAppDriver and Axe.Windows and there is a lot more we could cover. If you would like to learn more about authoring tests with WinAppDriver, you can find more information in their authoring test scripts documentation (https://github.com/microsoft/WinAppDriver/blob/master/Docs/AuthoringTestScripts.md) or take a look at their sample code: https://github.com/microsoft/WinAppDriver/tree/master/Samples/C%23. If you wish to learn more about Axe.Windows, you can visit their GitHub repository: https://github.com/microsoft/axe-windows.

In the next section, we will cover how to write unit tests for your Uno Platform app, including the different approaches to it.

Writing unit tests for your Uno Platform app

As app complexity increases, ensuring that your app's logic is working becomes increasingly difficult to validate without tests. While you can use UI tests to validate the logic, you can only verify logic that gets exposed as part of the UI. Things such as network access or error handling, however, become very difficult to validate using UI tests as those things are generally exposed through the UI. In addition to that, UI tests are slower since they are mimicking user interaction and rely on the rendered UI to update.

This is where unit tests come in. Unit tests are small tests that verify single units of your code. Most commonly, classes or functions are treated as individual units and tests are grouped based on the class or function they are testing; in other words, for every class you want to test, there is a set of tests only targeting that class and not any other class. As the complexity of your app increases, unit tests allow you to verify that single classes are still working as expected.

Important note

Unit tests are no silver bullet! While unit tests allow you to verify the behavior of single pieces of functionality, larger and more complex apps also require more tests besides unit tests, in other words, UI tests to ensure that the app as a whole works as expected. Only because single classes work correctly in isolation, this does not mean that the whole construct works together as expected and is bug-free!

Since, at the time of writing, only creating unit tests against the UWP head is well supported, we will focus on this. We will now cover the different ways to create the unit test project.

Different approaches to adding a unit test project

Since most, if not all, of your app's logic sits inside a shared project, writing unit tests is a bit more complex. Since the shared project does not actually produce an assembly that you can reference, there are different ways to test your app's logic, which both come with their own benefits and drawbacks.

The first option is to create a project containing the unit tests for the platform you want to run the tests on and reference the shared project in that project. This is the easiest way to get started since you just need to create a new project and reference the shared project. One of the downsides is that since shared projects don't allow references such as NuGet packages to be added to them, any libraries you are using inside your shared project also need to be referenced by your test project. In addition to this, since the shared project does not create a binary but is compiled into the projects that are referencing it, changes made to the shared project will always result in the tests project recompiling.

The next option is to leave your code inside the shared project and reference the platforms head project inside the unit test projects; for example, create a UWP Unit test project and reference the UWP head of your app inside it. This option is better than the first option since you don't encounter the issues of library references needing to be added to the test project since the platform head references the libraries for us. We will use this approach in this chapter.

The last option is to move the code inside the shared project into a Cross Platform Library (Uno Platform) project and reference the library in the platform heads and unit test projects. This approach has the benefit that you can add library references to the library project on its own and don't have to manually add the reference to the individual projects. One of the downsides is that you have to switch to a cross-platform library project type instead of being able to use the existing shared project. This approach also has the downside that the cross-platform library will always be compiled for all platforms, thereby increasing the build time when only requiring specific platforms.

Let's now add a unit test to our app by using the second option previously discussed, that is, adding a reference to the platforms head project.

Adding your first unit test project

Since we will reference the UWP platform head, we need a UWP unit test app. For this, we first need to add a new project:

  1. Right-click the solution and click Add > New Project.
  2. In the dialog, search for the Unit Test App (Universal Windows) text and select the Unit Test App (Universal Windows) project type, as shown in Figure 7.6:
    Figure 7.6 – Unit Test App (Universal Windows) project type in a new project dialog

    Figure 7.6 – Unit Test App (Universal Windows) project type in a new project dialog

  3. Click Next and name the project UnoAutomatedTestsApp.UWPUnitTests. You can name the project differently, of course; however, in this and the following sections, we will assume that the project is named as mentioned previously.
  4. Select the minimum and target version. We will use 18362 for both since the UWP head of the app also uses those. Not using the same minimum and target version of the UWP head might result in build errors, so you should always aim to match the UWP head.
  5. Now, add a reference for the UWP head to the Unit Test App project. For this, right-click the UnoAutomatedTestsApp.UWPUnitTests project in the solution view, click Add > Reference… > Projects, check UnoAutomatedTestsApp.UWP, and then click OK.
  6. Since the reference to the UWP head will also copy the Properties/Default.rd.xml file into the build output folder, this will result in a build issue as there are two Default.rd.xml files that the compiler wants to copy into the same folder. Because of that, rename the Default.rd.xml file of the unit test app to TestsDefault.rd.xml. Then, also update the UnoAutomatedTestsApp.UWPUnitTests.csproj file to point to that file. If you are renaming the file from the Solution view, you just need to select the project and press Ctrl + S.
  7. In addition to that, we also need to rename the image assets of the unit test project. For this, prepend all images inside the Assets folder with UWPUnitTestApp-.

We are now able to write and run unit tests for everything included inside the UWP head, including classes included inside the shared project. For larger apps that also have platform conditional code, you will only be able to reference classes and code inside the UWP unit test project that are getting compiled for the UWP head. Now that we have created the project, let's write a small unit test. In contrast to the Uno.UITest tests project, the Unit Test App (Universal Windows) project type uses MSTest as the testing framework. Of course, you can change this, but for simplicity, we will stick with MSTest. Note that you can't use NUnit for UWP unit tests as it does not support UWP:

  1. Since we don't have many classes we can test now, let's add a new class to the shared project. To do this, create a new class named DemoUtils.
  2. Replace the code of the file with the following:

    namespace UnoAutomatedTestsApp

    {

        public class DemoUtils

        {

            public static bool IsEven(int number)

            {

                return number % 2 == 0;

            }

        }

    }

    We will just use this code so that we have something easy to write unit tests for.

  3. Now, rename the UnitTest.cs file inside the UnoAutomatedTestsApp.UWPUnitTests project to DemoUtilsTests.cs.
  4. Now, replace the content of the DemoUtilsTests.cs file with the following:

    using UnoAutomatedTestsApp;

    using Microsoft.VisualStudio.TestTools.UnitTesting;

    namespace UnoAutomatedTests.UWPUnitTests

    {

        [TestClass]

        public class DemoUtilsTests

        {

            [TestMethod]

            public void VerifyEvenNumberIsEven()

            {

                Assert.IsTrue(DemoUtils.IsEven(2),

                    "Number 2 should be even");

            }

        }

    }

    This is a small unit test to verify that our DemoUtils.IsEven function successfully determines that the number 2 is even.

We have now added our first unit test. As is the case with the UI test, you can run the test by opening the test explorer, expanding the tree, right-clicking the VerifyEvenNumberIsEven test, and clicking on Run. The test will then compile, deploy the unit test app, and start it. Your tests will be run and the unit test app will then close.

In the last section of this chapter, we will cover manual testing, why it is important, and how to approach testing accessibility manually using Accessibility Insights.

Performing manual testing and why it is important

While automated tests help to find bugs and issues, there are certain things they cannot cover that still require manual testing. When developing apps that make use of features such as a camera, Bluetooth, or other device capabilities, writing automated tests is hard and sometimes even impossible. In these scenarios, manual testing is necessary. This is especially important with connectivity features to see how your app handles unstable connections and whether your app still provides a good user experience, especially with varying connection quality. More importantly, testing using emulators makes it hard to verify how the app will feel on actual devices, especially when thinking about the user experience, such as elements being the right size and easily tappable on screens.

In addition to testing specific features that are hard to simulate as part of an automated test such as GPS or roaming data access, manual testing is also critical to ensure that your app is great usability-wise. While during development, running the app inside your emulator is fine, manual testing becomes more and more important as development progresses.

Besides manually testing your app by using the app on a device or emulator, another important aspect is manually testing your app for accessibility. Ensuring that your app is accessible by users is crucial when developing apps, and while automated tests, such as Axe.Windows tests, can help find issues, they are not perfect. Since people with all levels of ability might use your app, making your app not accessible makes your app harder or even impossible to use for those customers. Since everyone should be able to use your app regardless of their level of ability, there are different tools when testing your app for accessibility. In this section, however, we will focus on using assistive technology and using the Accessibility Insights scanning tool.

Accessibility insights is a tool that allows you to scan your app for accessibility issues manually, similar to what Axe.Windows does. In fact, Accessibility insights for Windows uses Axe.Windows under the hood. In contrast to Axe.Windows, Accessibility Insights also allows the testing of your web app and Android app for accessibility issues. In this chapter, you will learn how you can use Accessibility Insights for Windows. If you wish to learn more about Accessibility Insights, including using Accessibility Insights for Web and Accessibility Insights for Android, you can check out the official website: https://accessibilityinsights.io/.

Now, let's get started by using Accessibility Insights for Windows by using it on the UWP head of the UnoAutomatedTestsApp:

  1. To do this, first, you need to download Accessibility Insights for Windows from https://accessibilityinsights.io/docs/en/windows/overview/ by clicking on Download for Windows. If you have already installed Accessibility Insights for Windows, you can proceed with step 4.
  2. Once the download has finished, run the MSI Installer to install Accessibility Insights.
  3. Once the installation process has finished, Accessibility Insights for Windows should start and you will see something similar to that shown in Figure 7.7 after dismissing the telemetry dialog:
    Figure 7.7 – Screenshot of Accessibility Insights

    Figure 7.7 – Screenshot of Accessibility Insights

  4. Once you have closed the popups, start the UWP head of UnoAutomatedTestsApp.

    Now, if you hover over the app, you will notice that the area you are hovering over and the controls in that area will be surrounded by a dark blue area. In Accessibility Insights, you can see the different UI automation properties of the control, for example, the control's control type or whether they are keyboard-focusable. To scan a control, you can either select the control from Live Inspect tree or click on the scan button in the top-right corner of the blue rectangle, as shown in Figure 7.8:

Figure 7.8 – Highlighted scan icon on control

Figure 7.8 – Highlighted scan icon on control

While Accessibility Insights is a useful tool for finding accessibility issues, testing your app by using it with assistive technology is crucial to ensure that your app can be used by users with all levels of ability. For this, we will manually test the UWP head using Narrator. However, similar testing can be done on Android, iOS, and macOS. To learn how to start the assistive technology on different platforms, please refer to the Starting the screen reader on different platforms section in Chapter 5, Making Your App Ready for the Real World.

Let's walk through our app now using Narrator. To do this, start Narrator by pressing Windows logo key, Ctrl, and Enter at the same time and open UnoAutomatedTestsApp. Narrator should then announce UnoAutomatedTestsApp, Window. Using the Caps Lock key and the arrow keys, you can navigate through the app. As you navigate through the app, Narrator will then announce the header of the username input, the password input, and then the sign-in button. This also allows us to find potential accessibility issues that Axe.Windows and Accessibility Insights for Windows did not catch. For this, enter a username by navigating to the username input field, entering the text invalid, and repeating the process for the password field. Upon navigating to the sign-in button and hitting the space bar, you will notice that you are not being notified of any error messages. This is an accessibility issue as users relying on assistive technology will not be notified of the error message and will not know what happened.

For larger apps, navigating through the app will be more complicated. While our test app is small and all controls are accessible, for larger apps using this testing, you can find crucial accessibility issues, for example, controls that have an unhelpful or even misleading representation for assistive technology. Finding these issues early in the development process makes them easier to fix and prevents them from impairing users.

In this section, we scratched the surface of manual testing and why it is necessary. We also covered how to approach accessibility testing using Accessibility Insights and assistive technology.

Summary

In this chapter, we have learned how to write automated UI tests for your app using Uno.UITest and Selenium. We also then learned how to run those tests on different platforms, including running them on your app running on an emulator. After that, we covered how to write UI tests for the UWP head of the app using WinAppDriver and also write unit tests for the UWP head. Lastly, we covered manual testing and how to test for accessibility issues.

In the next chapter, we'll talk about deploying your app and how you can bring your Xamarin.Forms app to the web using Uno Platform. We will also cover how to build for other platforms and cover how you can join and even contribute to the Uno community.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.238.228.237