Mouse Actions
This content applies solely to Appian RPA, which must be purchased separately from the Appian base platform.

Moving the mouse and clicking elements are two primary ways a robotic process interacts with a user interface. Appian RPA includes Mouse actions to make this type of action easier to integrate into your process.

This page describes the methods you can use to integrate mouse actions in your robotic process. If you need to extend Appian RPA capabilities, you can use Java to create workflow libraries for robotic processes.

Using the Mouse actions

When you build the robotic process, you'll see the following Mouse actions:

Click

This action performs different types of click actions, either on the cursor's current location or a position you specify using coordinates. You can choose how many clicks to perform, and which button to use (left or right).

You can configure CONFIGURATION and AFTER COMPLETION parameters for this action.

Configuration

In the CONFIGURATION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • Click on…:
    • Current mouse position: When selected, the mouse clicks on its current position on the screen.
    • Provided mouse position:
      • X Coordinate: The X coordinate where the mouse should be positioned and then click. This field can be configured using the expression editor.
      • Y Coordinate: The Y coordinate where the mouse should be positioned and then click. This field can be configured using the expression editor.
  • Button: Choose to click the Left or Right button on the mouse. If you choose Right click, only Single click is supported for the Action, described below.
  • Action: Choose the click action to perform: Single click, Double click, Hold, Custom.
    • Clicks: If the Custom action is selected, specify the number of times the mouse should click.
    • Delay: If the Custom action is selected, specify how long the mouse should wait to click, in milliseconds.

After completion

In the AFTER COMPLETION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Drag

This action asks the robotic process to click and drag the mouse. This action can be used to click and hold an element on the screen and move the cursor to another point. In this way, the Drag action can be used to drag and drop an item, as well as select text on the screen. This action can also select an area on the screen using a set of beginning coordinates and ending coordinates, highlighting everything in between.

You can configure CONFIGURATION and AFTER COMPLETION parameters for this action.

Configuration

In the CONFIGURATION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • Start from…: Choose where the mouse should begin dragging across the screen to select an area. You can choose the Current mouse position or Provided mouse position and specify coordinates:
    • X Coordinate (start): The X coordinate where the mouse should be positioned when it starts the drag operation. This option is visible only when Provided mouse position is selected for Start from…. This field can be configured using the expression editor.
    • Y Coordinate (start): The Y coordinate where the mouse should be positioned when it starts the drag operation. This option is visible only when Provided mouse position is selected for Start from…. This field can be configured using the expression editor.
  • Choose where the mouse should end dragging across the screen to select an area.
    • X Coordinate (end): The X coordinate where the mouse should be positioned when it ends the drag operation. This field can be configured using the expression editor.
    • Y Coordinate (end): The Y coordinate where the mouse should be positioned when it ends the drag operation. This field can be configured using the expression editor.

After completion

In the AFTER COMPLETION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Get cursor properties

This action returns details about the cursor in the form of a custom data type called CursorProperties. This information can be used to populate additional options or actions. For example, if this action determines that the cursor is an hour glass, you can configure it to wait until the current screen is loaded to proceed in the workflow. CursorProperties contains the following properties:

Property Type Description
position Point X and Y coordinates marking the cursor's current position.
color String The color of the pixel the cursor is on, in hexadecimal format.
type String The cursor type. Values include ARROW, CLOCK, CROSS, HAND, HELP, and SIZE. This will only return a value if your resource is using Windows; it will return null for Mac and Linux.

You can configure OUTPUTS and AFTER COMPLETION parameters for this action.

Output

This action returns the CursorProperties custom data type.

In the OUTPUTS area of the ACTION CONFIGURATION pane, configure the following parameters to store this result:

  • Operator: Select an option to store the result as a single-value variable (is stored as) or add the result as a multiple-value variable (is appended to).
  • Target: Enter the name of the robotic process variable where you want to store or append the result. If your process variable is type CDT, you can specify which field of a CDT to store the result into using dot notation.

Once the properties are stored in a robotic process variable, you can use dot notation in subsequent actions to return the values.

After completion

In the AFTER COMPLETION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Move

This action enables the robotic process to move the cursor to a set of coordinates on the screen.

You can configure CONFIGURATION and AFTER COMPLETION parameters for this action.

Configuration

In the CONFIGURATION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • X Coordinate: The X coordinate where the mouse should be positioned. This field can be configured using the expression editor.
  • Y Coordinate: The Y coordinate where the mouse should be positioned. This field can be configured using the expression editor.

After completion

In the AFTER COMPLETION area of the ACTION CONFIGURATION pane, configure the following parameters:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Release

This action releases the currently clicked button.

You can configure the following AFTER COMPLETION parameters for this action:

  • Wait before executing next action: Enter the number of seconds the robotic process waits before it continues to the next action in the workflow.
  • Take a screenshot: If you select this option, the robotic process takes a screenshot of the resource immediately after the action completes successfully.

Example: Open the Windows Start menu

To demonstrate how to use the Mouse actions, we'll configure an action to open the Start menu on a Windows machine:

  1. Go to the Appian RPA Console and open the Robotic Processes tab.
  2. Open a robotic process configuration and navigate to the Task tab.
  3. Click and hold the CLICK action in the Actions Palette.
  4. Drag and drop it to the section or action group where you want to add the action.
  5. In the Click On area of the ACTION CONFIGURATION pane, choose Other position.
  6. Enter the X and Y coordinates of where the Start menu appears on the screen.
  7. In the Interaction area, configure the click behavior. Choose the Left mouse button to perform a Single click.
  8. Click SAVE CHANGES in the page toolbar.

This action in the workflow simply clicks the area of the screen where the Start menu appears, which opens it. Configure additional actions in the workflow to move the mouse to a specific area of the menu and click those elements.

Open in Github Built: Wed, Aug 17, 2022 (01:05:05 PM)

On This Page

FEEDBACK