While unit tests ensure that a significant amount of code is tested, it is specifically focused on testing the logic within the app, leaving the user interface largely untested. This is where UI testing comes into play. UI testing allows us to automate specific actions within the app's user interface to assert that they function properly—just like regular unit tests assert that business logic delivers the desired results or functionality.
Xamarin provides a very rich set of tools for performing automated UI tests—both locally and in the Xamarin Test Cloud. UI tests can be written in C# and NUnit using Xamarin's UITest framework or in Ruby using Xamarin's Calabash framework. For the purposes of this book, we will only focus on using the C# approach using Xamarin.UITest.
The UITest framework enables you to automate interactions with an app using C# and the NUnit testing suite. All interactions take place through an instance of IApp
. The ConfigureApp
class is used to create iOS and Android IApp
instances. We will take a closer look at creating IApp
instances with ConfigureApp
in the next couple of sections.
At the time of writing this book, UITest only supports iOS and Android.
The UITest framework contains many APIs that can be used for interacting with an app's user interface. Here are some of the more commonly used methods and the ones we will specifically use to test the TripLog app in this chapter.
These methods are members of the IApp
interface:
Screenshot()
: This is used to take a screenshot of the current state of the appTap()
: This is used to send a tap interaction to a specific element on the app's current screenEnterText()
and ClearText()
: These are used to add and remove text from input elements such as Entry views in Xamarin.FormsQuery()
: This is used to find elements on the app's current screenRepl
: This is used to interact in real-time with the app through the terminal using the UITest APIWaitForElement()
: This is used to hold up the test until a specific element appears on the app's current screen within a specific timeout periodMany of the previous methods, including Query
and WaitForElement
, return an AppResult[]
object that can be used to determine the results of the call. For example, if a Query
call returns an empty result set, then we can assert that the element does not exist. We will see a couple uses of this in the next section when we start writing actual tests.
These methods are members of the AppQuery
class and are typically used in the AppQuery
Func
parameters of the IApp
methods mentioned previously:
Class()
: This is used to find elements on the app's current screen based on their class typeMarked()
: This is used to find elements on the app's current screen based on their text or identifierCss()
: This is used to perform CSS selector operations on the contents of a webview on the app's current screenOur UI tests will run side-by-side the normal unit tests in our solution:
Create a new Xamarin UITest project within the Tests
solution folder and name it TripLog.UITests
:
By default, we are supplied with an initial file named Tests.cs
. We will place our UI tests in this single Tests.cs
file for now. In some cases, it might make sense to break your UI tests up into separate files. In addition to the Tests.cs
class, we are also provided with a class named AppInitializer.cs
, which is used by the Tests.cs
class to create an IApp
instance and start the app under the test.
In order to start an app and interact with it via UITest, we must perform some initialization steps, which occur in the AppInitializer
class. The
AppInitializer
class has a static method called StartApp
, which is called by the Tests setup method to get an IApp
instance. It can be either an iOSApp
or an AndroidApp
using the ConfigureApp
class:
public class AppInitializer { public static IApp StartApp (Platform platform) { if (platform == Platform.Android) { return ConfigureApp .Android // TODO: Update this path ... //.ApkFile ("../../../Droid/bin/Debug/TripLog.apk") .StartApp (); } return ConfigureApp .iOS // TODO: Update this path ... //.AppBundle ("../../../iOS/bin/iPhoneSimulator/Debug/TripLog.iOS.app") .StartApp (); } }
Notice that the ApkFile
and AppBundle
methods are currently commented out. This is because it is not required to run the tests locally within the unit test pane in Xamarin Studio.
We will create tests for specific scenarios within the app to assert that screens and elements within them appear when expected and to capture screenshots of each step of each scenario for visual inspection.
When testing the UI, we will use a combination of the Xamarin.UITest and NUnit frameworks. UITest allows us to query and interact with elements on the app's user interface, whereas NUnit allows us to apply assertions to the results of the UITest operations resulting in either a pass or fail of the test—similar to typical unit tests.
The first scenario to test in our TripLog app is signing in. The main steps in our sign in scenario are as follows:
In order to create a UI test for the sign in scenario, perform the following steps:
Tests
file named SignIn
and decorate it with the [Test]
attribute:[Test] public void SignIn () { }
PerformFacebookSignInSteps
to the Tests
class that will handle the test steps specific to the Facebook sign in process. By placing these test steps into a separate method, they are easily reused in other tests that will also require automating the login process before carrying out other steps within the UI:void PerformFacebookSignInSteps (string email, string password) { // Wait for sign in with Facebook button to appear app.WaitForElement (x => x.Text ("Sign in with Facebook")); app.Tap (x => x.Text ("Sign in with Facebook")); // Wait for Login button within Facebook oAuth webview to appear app.WaitForElement (x => x.WebView().Css ("[name=login]")); app.Screenshot ("Facebook oAuth login screen"); // Enter text in the element within the webview with name="email" app.EnterText (x => x.WebView ().Css ("[name=email]"), email); // Enter text in the element within the webview with name="email" app.EnterText (x => x.WebView ().Css ("[name=pass]"), password); // Tap the button in the webview with name="login" app.Tap (x => x.WebView ().Css ("[name=login]")); }
SignIn
test method to call the PerformFacebookSignInSteps
method, and validate that the main screen appears after successfully signing in:[Test] public void SignIn () { app.Screenshot ("Sign in screen"); PerformFacebookSignInSteps ("<your-fb-email>", "<your-fb-password>"); var mainScreen = app.WaitForElement (x => x.Marked ("TripLog").Class ("UINavigationBar")); app.Screenshot ("Main screen"); // Assert main screen is shown Assert.IsTrue (mainScreen.Any (), "Main screen wasn't shown after signing in"); }
We can also write UI tests for alternative scenarios such as when the user taps the Cancel button on the OAuth web view authentication screen:
[Test] public void CancelSignIn () { app.Screenshot ("Sign in screen"); app.Tap (x => x.Text ("Sign in with Facebook")); // Wait for Login button within Facebook oAuth webview to appear app.WaitForElement (x => x.WebView().Css ("[name=login]")); // According to REPL there are 3 elements marked as Cancel - the one at the second 2nd index is a UIButtonLabel which we need var cancel = app.Query (x => x.Marked ("Cancel").Index (2)); // Assert there is a Cancel button the UINavigationBar Assert.IsTrue (cancel.Any ()); app.Screenshot ("Authentication screen with Cancel button"); app.Tap (x => x.Marked ("Cancel").Index (2)); // Wait for sign in screen to appear var mainScreen = app.WaitForElement (x => x.Marked ("Sign in with Facebook")); // Assert sign in screen is shown after auth webview screen is canceled Assert.IsTrue (mainScreen.Any ()); app.Screenshot ("Sign in screen after auth is canceled"); }
The second scenario that we can test is adding a new entry. The main steps in this scenario are as follows:
We will need to use the PerformFacebookSignInSteps
method again to advance through the sign in steps:
[Test] public void AddNewEntry () { app.Screenshot ("Sign in screen"); PerformFacebookSignInSteps ("<your-fb-email>", "<your-fb-password>"); // Wait for main screen to appear var mainScreen = app.WaitForElement (x => x.Marked ("TripLog").Class ("UINavigationBar")); app.Screenshot ("Main screen"); // Assert main screen is shown Assert.IsTrue (mainScreen.Any (), "Main screen wasn't shown after signing in"); app.Tap (x => x.Marked ("New")); // Wait for New Entry screen to appear var newEntryScreen = app.WaitForElement (x => x.Marked ("New Entry").Class ("UINavigationBar")); app.Screenshot ("New entry screen"); // Assert New Entry screen is shown Assert.IsTrue (newEntryScreen.Any (), "New Entry screen was not shown after tapping New on main screen"); // Enter title text into Title EntryCell (first EntryCell) app.EnterText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell").Index (0), "Test Entry"); // Tap into Date EntryCell (fourth EntryCell) app.Tap (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell").Index (3)); // Tap Done on the date picker app.Tap (x => x.Marked ("Done")); // Tap into Rating EntryCell (fifth EntryCell) and assert numeric keyboard app.Tap (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell").Index (4)); var numericKeyboard = app.Query (x => x.Class("UIKBKeyView")); // Assert numeric keyboard is shown for Rating EntryCell - numeric keyboard has 12 UIKBKeyViews Assert.IsTrue (numericKeyboard.Count () == 12, "Rating EntryCell doesn't use numeric keyboard"); // Clear default text in Rating EntryCell (fourth EntryCell) app.ClearText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell") .Index (4) .Descendant ("UITextField")); // Enter rating text into Rating EntryCell (fifth EntryCell) app.EnterText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell") .Index (4) .Descendant ("UITextField"), "3"); // Enter notes text into Notes EntryCell (sixth EntryCell) app.EnterText (x => x.Class ("Xamarin_Forms_Platform_iOS_EntryCellRenderer_EntryCellTableViewCell") .Index (5) .Descendant ("UITextField"), "Test entry notes."); // Tap Save to exit the new entry screen app.Tap (x => x.Marked ("Save")); // Wait for main screen to appear mainScreen = app.WaitForElement (x => x.Marked ("TripLog").Class ("UINavigationBar")); // Assert main screen is shown after saving a new entry Assert.IsTrue (mainScreen.Any (), "Main screen was not shown after saving a new entry"); }
Notice that a common pattern used to validate screens and elements is to use the WaitForElement
method in the UITest API along with the NUnit Assert.IsTrue
method.
There are a few references to iOS specifics throughout these tests. Use the private platform class variable to check the platform and handle accordingly for Android. Alternatively, if these tests will only be run for iOS, then remove the [TestFixture (Platform.Android)]
class attribute; the tests will be locked to iOS only.
In order to run UI tests, Xamarin Test Cloud Agent needs to be included in the app project. For Android, Test Cloud Agent is provided by the UITest framework, and so, it does not need to be manually added. For iOS, Test Cloud Agent needs to be manually added to the project either via NuGet or Xamarin component:
FinishedLaunching
method of the TripLog app AppDelegate
class to start Test Cloud Agent. Again, this step is only for iOS app projects:public override bool FinishedLaunching (UIApplication app, NSDictionary options) { global::Xamarin.Forms.Forms.Init (); Xamarin.FormsMaps.Init (); #if ENABLE_TEST_CLOUD Xamarin.Calabash.Start (); #endif LoadApplication (new App (new TripLogPlatformModule())); return base.FinishedLaunching (app, options); }
Notice the use of the ENABLE_TEST_CLOUD
compiler symbol in the #if
directive surrounding the Calabash.Start
method call. This ensures that Test Cloud Agent is only started under specific configurations. Be sure to include the ENABLE_TEST_CLOUD
symbol in the Debug configuration only and not in the Release configuration.
UI tests can be run from the Unit Tests pane within Xamarin Studio, just like normal unit tests. However, you will notice that there is a Test Apps node within the TripLog.UITests section.
You need to either add your app to the Test Apps node of the Unit Tests pane or specify the path to the app in the AppInitializer
class mentioned earlier; otherwise, your tests will fail to run.
To add your apps to the Tests Apps node within the Unit Tests pane, simply right-click on the Test Apps item and click on Add App Project. This will bring up a dialog that allows you to select one or many UITest compatible projects that can be added.
Select the project(s) that you want to add to the tests and click on OK. For the purpose of this chapter, we only need to select the TripLog.iOS app project since our tests are specific to iOS. You will now see the selected app listed under the Test Apps node in the Unit Tests pane.
If you don't see your iOS app project listed in the dialog when trying to add to the Test Apps node, ensure that you have added the Xamarin Test Cloud Agent NuGet package to your iOS project.
Once you have either added your app projects to the Unit Tests or referenced the app package within AppInitializer
, simply run the UI tests within the Unit Tests pane, just as you do for normal unit tests.
When your UI tests start to run, the UITest framework will automatically deploy your app to the simulator, run the app, and step through each of the steps specified in the test methods. The results of the tests will appear in the Test Results pane in Xamarin Studio.
By default, screenshots are not stored when UI tests are run locally. In order to store the screenshots captured during your tests, you must add the EnableLocalScreenshots
method to the ConfigureApp
method chain in the AppInitializer
class:
return ConfigureApp .iOS .EnableLocalScreenshots () .StartApp ();
Once enabled, the screenshot images will be stored in the bin
output folder where the tests are run from. Beware that these images will be overwritten the next time the tests run. So, copy them out to a different location if they need to be stored for each local test run.
Once you have run your UI tests locally and verified the steps within them, you can broaden the number of devices, screen sizes, and operating system versions that they run on by uploading them to Xamarin Test Cloud.
You can upload your UI tests to Xamarin Test Cloud directly from the Unit Tests pane in Xamarin Studio.
Xamarin Studio will build and upload your app and UI tests to Test Cloud. Once it has finished uploading the app and tests to Test Cloud, it will automatically launch the Test Cloud interface in your browser and allow you to select the parameters for the test run including specific devices and operating system versions. Finally, it will validate the tests and run them on the selected devices, providing the test results at completion.
18.226.185.196