Running Tests on the Command Line

One thing that may have occurred to you is that all the tests run up to this point have been performed in the Xcode GUI, either by clicking gutter buttons in the editor view, run buttons in the Test navigator, or with the menu item Product > Test (⌘U). That’s all well and good, but it’s obviously inadequate if you want your tests to be part of a continuous integration (CI) process, one which automatically runs tests for you, either periodically (like every night), or maybe every time new code is checked in.

Fortunately, it’s simple to have Xcode run tests from the command line or in a shell script. It’s actually a lot like automated building of your project, which you saw way back in ​Building on the Command Line​. In fact, it uses the same command, xcodebuild. The difference is that xcodebuild implicitly takes an action, which defaults to build. To run tests, you just need to use xcodebuild test.

Except… it’s not actually that simple. The test action doesn’t settle into sensible defaults, so just using xcodebuild test doesn’t work:

The hint is nice, as it tells you that xcodebuild -list will give you the name of all the schemes, one of which you’ll include with the command next time. As it turns out, that’s only half the story…

Right. So you need to not only provide a scheme, but also specify either an actual iOS device by name, or the identifier for one of the installed simulators. You can use the action showdestinations to show strings for all the available simulators, and then pick the one to run on:

Notice that you have to include which scheme you’re interested in, since you may have some simulators that are incapable of running a given scheme (maybe your app is iPhone- or iPad-only, or you have simulators older than the app’s minimum supported iOS version). The output of this command is a list of simulators, which are identified by id, OS version, and name. It’s a long list, so here’s just a clip:

The way to compose the -destination option is to provide a set of criteria that xcodebuild can match. You could use the id string, but it differs from computer to computer, so that approach won’t work for teams. Instead, you should compose your destination as a combination of platform, OS, and device name, like the following command (which has added a backslash line-continuation character to fit the book’s limited width):

This will run the tests on the specified simulator. If all of the tests pass, xcodebuild returns 0, like any other command-line process does when it ends normally. On the other hand, if any tests fail, the return value will be non-zero (usually 65), so you can watch for that in build scripts and CI systems.

What if you want to run your tests on a device? That’s when you use the id, since now you’re talking about one specific iOS device, as opposed to one instance of a simulator. You can get the device identifier by bringing up the Devices and Simulators window (either from the Window menu or with ⇧⌘2), as seen in the figure. Plug in your device via USB, select it from the Devices tab, and copy-and-paste the “Identifier” string:

With the device identifier in the clipboard, you can paste it into the Terminal as you enter the xcodebuild command, which will look like this:

Where We’re Going, We Don’t Need… Cords

One really cool feature is hiding here in the Devices and Simulators window: the check box marked “Connect via network”. Selecting it allows Xcode to run, test, and debug the app over Wi-Fi, without being connected via a USB cable. Once you enable Connect via network, the device can be unplugged and will still appear in the Devices and Simulators window and the scheme selector—as long as it’s within Wi-Fi range—and will have a little globe icon to indicate it is connected wirelessly.