UIAutomation Template

The UIAutomation template is a robotic process whose functionality is to automate the Calculator application on Windows machines. The module and methods provide a way for robotic processes to access and interact with elements of the user interface. Microsoft UI Automation is typically applied to assistive technologies such as screen readers, but can be used in Appian RPA to detect certain attributes of elements on a screen and then act on them programmatically.

Workflow

The workflow in this template contains four actions: Start, Open calculator, Calc, and End. The Calc action performs calculations using the calculator application.

uiautomation-1.png

The sections below describe the methods associated with each action in the workflow.

Implementation

This section explains the RobotCalculator class included in the project. First, declare variables and constants used in later parts of the code. Next, we'll run through each of the methods that are associated with each action in the workflow.

Declaring variables and constants

For this robotic process, define the following:

  1. Constants to calculator app:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    
     /**
      * The Constant CALC_EXECUTABLE.
      */
     private static final String CALC_EXECUTABLE = "calc.exe";
        
     /**
      * The Constant CALC_TITLE_REGEX.
      */
     private static final String CALC_TITLE_REGEX = "Calculadora|Calculator";
    
  2. Appian RPA API objects to be used during the robotic process execution:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    
     /** The server. */
     private IJidokaServer<?> server;
        
     /**  The automation. */
     private UIAutomation automation;
        
     /** The calc window */
     private Window calcWindow;
        	
     /** The windows. */
     private IWindows windows;
        	
     /** The wait for.  */
     private IWaitFor waitFor;
    

startUp() method

The startUp() method initializes the Appian RPA modules the robotic process is going to use.

1
2
3
4
5
6
7
8
9
10
public boolean startUp() throws Exception {
	//Init the modules
	server = JidokaFactory.getServer();
	windows = IWindows.getFreshInstance(this);
	waitFor = windows.getWaitFor(this);

	automation = windows.uiAutomation();
	
	return IRobot.super.startUp();
}

start() method

In this template, we only use the start() method to set the number of items. It's a best practice to set the number of items, the current item the robotic process is working on, and the results. This may seem obvious when the robotic process is only working on one item, but it's included here as a recommendation.

1
2
3
4
public void start() {
	server.setNumberOfItems(1);		
	server.debug("Robot initialized");
}

startCalculator() method

The startCalculator() method launches the calculator app using the UIAutomation module.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public void startCalculator() {

	try {
			
		// Open the Calculator APP and get the Window
		Application calcApp = automation.launchOrAttach(CALC_EXECUTABLE);

		calcApp.waitForInputIdle(30);
		
		calcWindow = automation.getDesktopWindow(Pattern.compile(CALC_TITLE_REGEX));

		calcWindow.focus();
			
		server.sendScreen("calc");

	} catch (Exception ex) {
		throw new JidokaFatalException("Error initializing the calculator: " + ex.getMessage(), ex);
	}
}

applyFormulas() method

The applyFormulas() method manages the app using the automationIds defined and gets the result.

This method is defined so that it passes parameters from the workflow to the robotic process. In the workflow, you can configure the values for the parameters first, sign, and second, as shown in the image below.

uiautomation-2.png

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
/**
 * Apply formulas.
 *
 * @param first  the first operand
 * @param sign   the sign
 * @param second the second operand
 * @throws PatternNotFoundException the pattern not found exception
 * @throws AutomationException      the automation exception
 */
@JidokaMethod(name = "Calculate", description = "Formula to calculate")
public void applyFormulas(@JidokaParameter(defaultValue = "5", name = "first") String first,
		@JidokaParameter(defaultValue = "+", name = "sign") String sign,
		@JidokaParameter(defaultValue = "8", name = "second") String second)
		throws AutomationException {
	try {
		server.setCurrentItem(1, first + sign + second);

		for (String number : first.split("")) {
			clickOnControl(number);
		}
	
		clickOnControl(sign);

		for (String number : second.split("")) {
			clickOnControl(number);
		}
		
		clickOnControl("=");
		
		String result = "";
		
		// The property from which the value is obtained is different between win7 and win10
		if (SystemUtils.IS_OS_WINDOWS_7) {
			result = calcWindow.getTextBoxByAutomationId(ECalculatorControls.RESULT.getAutomationID()).getValue();
		} else {
			result = calcWindow.getTextBoxByAutomationId(ECalculatorControls.RESULT.getAutomationID()).getName();
		}
			
		server.setCurrentItemResultToOK(result);
	} catch (Exception e) {
		server.error("Exception calculating result", e);
		throw new JidokaItemException("Error calculating the result");
	}

}

To press each button, use the clickOnControl() method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
private void clickOnControl(String controlCode) throws AutomationException {

	String automationID = ECalculatorControls.getByCode(controlCode).getAutomationID();
	server.debug(String.format("Click on control [%s] with automationID [%s]", controlCode, automationID));
		
	// Wait element availability
	waitFor.wait("Waiting for : " + automationID, () -> {
			
		try {

			AutomationBase control = calcWindow.getControlByAutomationId(automationID);
				
			return control != null;
				
		} catch (AutomationException e) {
			return false;
		}
		
	});
		
	Button control = calcWindow.getButtonByAutomationId(automationID);
	control.click();
}

As you can see in the code, the template uses ECalculatorControls to define the controls for each operating system (Windows 7 and Windows 10). The robotic process is ready to run on each.

ECalculatorControls enum

This template uses the ECalculatorControls enum to define the Ids for each button in Windows 7 and Windows 10. In a real scenario, you won't need to do this, but if you want to build a library to manage an application, the following code sample could be useful.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
/**
 * The Enum ECalculatorControls.
 */
public enum ECalculatorControls {

/* The one */
ONE("1", "131", "num1Button"),

/* The two.*/
TWO("2", "132", "num2Button"),

/* The three. */
THREE("3", "133", "num3Button"),

/* The four. */
FOUR("4", "134", "num4Button"),

/* The five. */
FIVE("5", "135", "num5Button"),
....

/**
 * Gets the automation ID.
 *
 * @return the automation ID
 */
public String getAutomationID() {
	if (SystemUtils.IS_OS_WINDOWS_7 || SystemUtils.OS_NAME.equals(this.WS2019_OS_NAME)) {
		return w7automationID;
	}
	return w10automationID;
}

If you have installed the Inspector of Elements, you can check these values in your OS. This is an example for Windows 10

uiautomation-3.png

end() method

The end method is the last action called in the workflow. In this template, this method sends a message to the server.

1
2
3
4
5
6
/**
  Action to end the robot.
 */
public void end() {
	Server.info("End process");
}

cleanUp() method

Finally, the robotic process needs to close the application. To ensure that the process closes properly even if it encounters an exception, the template uses the cleanUp() method to close the application.

1
2
3
4
5
6
7
8
9
10
11
public String[] cleanUp() throws Exception {
	try {
		if(calcWindow != null) {
			calcWindow.close();
		}
	} catch (Exception e) {
		server.warn(e, e);
		windows.killProcess("Calculator.exe");
	}
	return new String[] {};
}
Open in Github

On This Page

FEEDBACK