Automating a non-web UI in Selenium WebDriver with Sikuli

Sikuli is another tool that can be used along with Selenium WebDriver to automate a non-web UI. It uses visual identification technology to automate and test graphical user interfaces (GUIs). The Sikuli script automates anything you see as a user on the screen rather than an API.

Sikuli is supported on Windows, Linux, and Mac OSX operating systems. Similar to Selenium IDE, Sikuli provides an IDE for script development and API that can be used within Java.

Sikuli works well for non-web UI. However, it also has certain limitations, as it is not supported by RemoteWebDriver. The Sikuli script might fail if it does not find a captured image due to overlapping windows at runtime.

In this recipe, we will explore the integration of the Sikuli API with Selenium WebDriver to test a non-web UI.

Getting ready

Download and install Sikuli from http://sikuli.org/.

How to do it...

We will use the HTTP authentication example from the previous recipe to automate the steps with Sikuli, as follows:

  1. Before we automate the steps on the Authentication Required dialog box, we need to capture images of the User Name: and Password: textboxes, and the Log In button:
    How to do it...
  2. Capture the screenshot of the Authentication Required dialog box and extract images, as follows, for each control:

    Control

    Screenshot

    User Name: textbox

    How to do it...

    Password: textbox

    How to do it...

    Log In button

    How to do it...
  3. Save these extracted images separately in PNG (Portable Network Graphics) format in a folder that can be accessed from tests easily.
  4. Add the Sikuli-script.jar file to the test project. You can get this file from the Sikuli installation folder.
  5. Create a new test, name it HttpAuthTest, and copy the following code to this test:
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.openqa.selenium.By;
    
    import org.sikuli.script.FindFailed;
    import org.sikuli.script.Screen;
    
    import org.junit.*;
    import static org.junit.Assert.fail;
    
    public class HttpAuthTest {
        private WebDriver driver;
        private StringBuffer verificationErrors = new StringBuffer();
    
        @Before
        public void setUp() {
            driver = new ChromeDriver();
            driver.get("http://www.httpwatch.com/httpgallery/authentication/");
        }
    
        @Test
        public void testHttpAuth() throws InterruptedException {
            driver.findElement(By.id("displayImage")).click();
    
            //Get the system screen.
            Screen s = new Screen();
    
            try {
    
                //Sikuli type command will use the image file //of the control
                //and text that needs to be entered in to the //control
                s.type("C:\UserName.png", "httpwatch");
                s.type("C:\Password.png","dhjhfj");
    
                //Sikuli click command will use the image //file of the control
                s.click("C:\Login.png");
    
            } catch (FindFailed e) {
                //Sikuli raises FindFailed exception it fails
                //to locate the image on to the screen
                e.printStackTrace();
            }
        }
    
        @After
        public void tearDown() {
            driver.close();
            String verificationErrorString = verificationErrors.toString();
            if (!"".equals(verificationErrorString)) {
                fail(verificationErrorString);
            }
        }
    }

How it works...

As Sikuli uses visual identification technology to identify and interact with windows and controls, it requires images of the controls to perform the action. During the execution, it locates the regions captured in the image on the screen and performs the specified action:

s.type("C:\UserName.png", "httpwatch");

In this example, it searches the screen for a region that matches the image captured in UserName.png, and performs the type () command. Sikuli replays these commands as if a real user is working on the screen.

Sikuli may fail to interact with the application if the region captured in the image is not displayed on the screen or is overlapped with other windows.

There's more…

By using Sikuli, you can automate RIA technologies such as Flash and Silverlight, along with the Selenium WebDriver.

See also

  • The Automating a non-web UI in Selenium WebDriver with AutoIt recipe
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.133.158.36