Mouse Module

Moving the mouse and clicking elements are two primary ways a robotic process interacts with a user interface. Appian RPA includes a Mouse module to make this type of action easier to integrate into your process.

This page describes the methods you can use to integrate mouse actions in your robotic process. Developers who feel more comfortable working in the robotic process's code can use Java methods to configure mouse actions. These methods are contained in the Client module and can be found in the com.novayre.jidoka.client.api.IActionable interface.

Using the Mouse module

Low-code methods are associated with an action in the robotic process's workflow. When you build the robotic process workflow, you'll see the following methods available in the Mouse module:

Click

This method performs different types of click actions, either on the cursor's current location or a position you specify using coordinates. You can choose how many clicks to perform, and which button to use (left or right).

You can configure CONFIGURATION and AFTER COMPLETION parameters for this method.

Configuration

In the CONFIGURATION area of the method pane, configure the following parameters:

  • Click on…:
    • Current mouse position: When selected, the mouse clicks on its current position on the screen.
    • Provided mouse position:
      • X Coordinate: The X coordinate where the mouse should be positioned and then click. This field can be configured using the expression editor.
      • Y Coordinate: The Y coordinate where the mouse should be positioned and then click. This field can be configured using the expression editor.
  • Button: Choose to click the Left or Right button on the mouse. If you choose Right click, only Single click is supported for the Action, described below.
  • Action: Choose the click action to perform: Single click, Double click, Hold, Custom.
    • Clicks: If the Custom action is selected, specify the number of times the mouse should click.
    • Delay: If the Custom action is selected, specify how long the mouse should wait to click, in milliseconds.

After completion

In the AFTER COMPLETION area of the method pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Drag

This method asks the robotic process to click and drag the mouse. This method can be used to click and hold an element on the screen and move the cursor to another point. In this way, the Drag method can be used to drag and drop an item, as well as select text on the screen. This method can also select an area on the screen using a set of beginning coordinates and ending coordinates, highlighting everything in between.

You can configure CONFIGURATION and AFTER COMPLETION parameters for this method.

Configuration

In the CONFIGURATION area of the method pane, configure the following parameters:

  • Start from…: Choose where the mouse should begin dragging across the screen to select an area. You can choose the Current mouse position or Provided mouse position and specify coordinates:
    • X Coordinate (start): The X coordinate where the mouse should be positioned when it starts the drag operation. This option is visible only when Provided mouse position is selected for Start from…. This field can be configured using the expression editor.
    • Y Coordinate (start): The Y coordinate where the mouse should be positioned when it starts the drag operation. This option is visible only when Provided mouse position is selected for Start from…. This field can be configured using the expression editor.
  • Choose where the mouse should end dragging across the screen to select an area.
    • X Coordinate (end): The X coordinate where the mouse should be positioned when it ends the drag operation. This field can be configured using the expression editor.
    • Y Coordinate (end): The Y coordinate where the mouse should be positioned when it ends the drag operation. This field can be configured using the expression editor.

After completion

In the AFTER COMPLETION area of the method pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Get cursor properties

This method returns details about the cursor in the form of a custom data type called CursorProperties. This information can be used to populate additional options or actions. For example, if this method determines that the cursor is an hour glass, you can configure it to wait until the current screen is loaded to proceed in the workflow. CursorProperties contains the following properties:

Property Type Description
position Point X and Y coordinates marking the cursor's current position.
color String The color of the pixel the cursor is on, in hexadecimal format.
type String The cursor type. Values include ARROW, CLOCK, CROSS, HAND, HELP, and SIZE. This will only return a value if your resource is using Windows; it will return null for Mac and Linux.

You can configure OUTPUT and AFTER COMPLETION parameters for this method.

Output

This method returns the CursorProperties custom data type.

In the OUTPUT area of the method pane, configure the following parameters to store this result:

  • Operator: Select an option to store the result as a single-value variable (is stored as) or add the result as a multiple-value variable (is appended to).
  • Target: Enter the name of the robotic process variable where you want to store or append the result. If your process variable is type CDT, you can specify which field of a CDT to store the result into using dot notation.

Once the properties are stored in a robotic process variable, you can use dot notation in subsequent actions to return the values.

After completion

In the AFTER COMPLETION area of the method pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Move

This method enables the robotic process to move the cursor to a set of coordinates on the screen.

You can configure CONFIGURATION and AFTER COMPLETION parameters for this method.

Configuration

In the CONFIGURATION area of the method pane, configure the following parameters:

  • X Coordinate: The X coordinate where the mouse should be positioned. This field can be configured using the expression editor.
  • Y Coordinate: The Y coordinate where the mouse should be positioned. This field can be configured using the expression editor.

After completion

In the AFTER COMPLETION area of the method pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Release

This method releases the currently clicked button.

You can configure the following AFTER COMPLETION parameters for this method:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Example: Open the Windows Start menu

To demonstrate how to use the Mouse methods, we'll configure an action to open the Start menu on a Windows machine:

  1. Go to the Appian RPA Console and open the Robotic Processes tab.
  2. Open a robotic process configuration and navigate to an action in the workflow.
  3. Click the action and click the list icon 951644.png.
  4. In the Module tree, open the Mouse module and click Click.
  5. In the Click On area, choose Other position.
  6. Enter the X and Y coordinates of where the Start menu appears on the screen.
  7. In the Interaction area, configure the click behavior. Choose the Left mouse button to perform a Single click.
  8. Click OK to save your changes to the action.
  9. Click SAVE CHANGES in the page toolbar.

This action in the workflow simply clicks the area of the screen where the Start menu appears, which opens it. Configure additional actions in the workflow to move the mouse to a specific area of the menu and click those elements.


This version of the Appian RPA documentation was written for Appian 21.4, and does not represent the interfaces or functionality of other Appian versions.
Open in Github Built: Tue, Nov 23, 2021 (07:31:45 PM)

On This Page

FEEDBACK