Falcon Robot Template

If your robotic process will need to capture images or search for existing images on a screen, you will use the Falcon module to configure your process. To get started, the Falcon template provides beginner level steps on using the Falcon module. This template details how to search and click on an image defined in your support files.

When you use a template to create a robotic process, you are provided with a zip file containing your source code and a pre-built workflow in the Robotic process configuration page. Within the source code of this template, the robotic process references a sample image. For the robotic process to locate this image, you will need to use the support image creation tool from the Appian RPA agent to capture the sample image.

The following page explains the contents of the Falcon template workflow and the source code generated by the template.

Prerequisite

For the robotic process to locate the sample image referenced in this template, you will need to use the support image creation tool to capture a screenshot of an image on your resource.

The template comes with the image test.png attached, which is an image of the Firefox icon. Before making changes to the template code, you can capture your own image instead of using the Firefox icon.

To create a support image on your resource:

  1. Right-click the agent icon and click Create support image.
  2. Click and drag your mouse over an area of the screen to take a partial screenshot.
  3. In the side panel, under Save image, select the dropdown and choose to save your image to your robotic process.
  4. Enter test as the name of your image.
  5. Click Save image. The image will be sent to the console and available as a robot support file.

falcon-template-2.png

Workflow

falcon-workflow2.png

The Falcon template's workflow includes three actions: Init, Search, and End.

Similar to an Appian process model, each workflow has a start and end action. This is represented by the Init and End actions. The Search action is where most of the configuration will occur.

If you click the list icon on any action, you'll see it is associated with a method. These methods are in the source code generated from the template. Within the source code, Appian has provided an outline of how you can configure each method.

Source code

Within the downloaded ZIP file, you will see two files: JidokaImageSupport.java and RobotFalconTemplate.java.

The JidokaImageSupport class allows you to obtain an IFalconImage instance of the image test.png that is stored in your support files. The RobotFalconTemplate class contains the main methods used to configure the Init and Search actions within your robotic process.

JidokaImageSupport

The JidokaImageSupport class allows you to obtain supporting code for your images. Within the template, the getTestPng() method is a method specifically created for the test.png image saved as a support file. This method will return an object of type IFalconImage, an image abstraction used by the Falcon module. Images must be of type IFalconImage to work directly with the Falcon module.

Within the source code, you'll see the following:

  • IFalconImage (line 102) is used to instantiate the getTestPng() method.
  • getTestPng() (line 102) is the method specific to calling the test.png image that is stored as a support file. This method will return an object of type IFalconImage.
  • getImage (line 103) is responsible for returning the image specified.
  • Paths.get(server.getCurrentDir(), "test.png") (line 103) is the file path to the support file containing the test.png image.
97
98
99
100
101
102
103
104
/**
 * Returns a falcon image referencing image resource test.png
 * @return
 * @throws IOException
 */
public IFalconImage getTestPng() throws IOException {
	return getImage(Paths.get(server.getCurrentDir(), "test.png"));
}

The JidokaImageSupport class can be automatically created during the Maven build process using the Appian RPA Maven plugin. It's recommended to use this plugin when using the Falcon module in your robotic process. To use the plugin, you will need to configure the plugin in your robotic process's pom.xml. See the Falcon module for steps to configure the plugin.

RobotFalconTemplate

The RobotFalconTemplate class implements the IRobot interface and is annotated by @Robot. Every Appian RPA robotic process must have the IRobot interface and @Robot annotation to allow the Appian RPA API to identify the class as a robot.

RobotFalconTemplate includes the methods listed below. Each of these actions correlate to the workflow actions listed in the Robotic process configuration page.

Method Description Associated Action
start() Provides initial information to the process. Init
searchImage() Where the robotic process determines the image to locate, searches for the image on the screen, and then determines what the process should do if the image is found. Search
end() Provides any final information to the process. End

Attributes

The first few lines in the template's source code reference the attributes related to the robotic process:

  • IJidokaServer (line 36) is responsible for communicating with the Appian RPA Server. The server is what communicates information to the console.
  • IFalcon (line 39) is the core interface of the Falcon module. This interface provides methods to take full or partial screenshots and search for images.
  • IClient (line 42) is responsible for making low-level operating system API calls. This object allows the robotic process to perform actions on a resource, regardless of its operating system.
  • JidokaImageSupport (line 44) refers to the JidokaImageSupport class that contains the supporting code for your images.
35
36
37
38
39
40
41
42
43
44
45
	
	/** The server. */	
	private IJidokaServer< ? > server;	

	/** The falcon. */	
	private IFalcon falcon;	

	/** The client. */	
	private IClient client;	

	private JidokaImageSupport images;	

start() method

The start() method is responsible for defining the initial actions of the workflow. This is where you should initialize the objects that will be used by the robotic process, and communicate to the server how many items your process will be iterating through.

  • server = JidokaFactory.getServer(); (line 52) initializes IJidokaServer.
  • client = IClient.getInstance(this); (line 53) initializes IClient, where this refers to the class that implements the IRobot interface. From the template, this would be the class RobotFalconTemplate.
  • falcon = IFalcon.getInstance(this, client); (line 54) initializes IFalcon, where this refers to the class that implements the IRobot interface, and client refers to the initialized IClient interface.
  • images = JidokaImageSupport.getInstance(this); (line 55) initializes the JidokaImageSupport class, where this refers to the class that implements the IRobot interface.

Once these objects are initialized, the server needs to know the total number of items to process and which item is currently being processed. This is communicated using the following attributes:

  • server.setNumberOfItems(1); (line 57) tells the server that there is only 1 item to process.
  • server.setCurrentItem(1, images.getTestPng().getDescription()); (line 58) tells the server which item is the current item being processed. This requires two parameters: currentItemIndex, which is set to 1, and the currentItem description, which in this case is identified by the initialized JidokaImageSupport class.
46
47
48
49
50
51
52
53
54
55
56
57
58
59
/**
 * Initialize the modules
 * @throws IOException 
 */
public void start() throws IOException {
		
	server = JidokaFactory.getServer();
	client = IClient.getInstance(this);
	falcon = IFalcon.getInstance(this, client);
	images = JidokaImageSupport.getInstance(this);
		
	server.setNumberOfItems(1);
	server.setCurrentItem(1, images.getTestPng().getDescription());
}

searchImage() method

The searchImage() method is where the robotic process will determine the image to locate, search for the image on the screen, and then interact with the image when it's found.

First the process will find and load the image to look for:

  • IFalconImage testImage = images.getTestPng().search(); (line 68) loads an IFalconImage instance of the test.png image.
  • falcon.sendImage(testImage.getImage(), "Test image"); (line 77) uses the Falcon module to send the test.png image to your log.
  • server.info (lines 74 and 79) send a descriptive log message of type info to the server.
  • server.sendScreen (line 80) sends a screenshot of the current resource to the server. A description is accompanied with the image.

Next, the process will search for the image on the screen. Using an if statement, the process will perform the following actions if the image is found:

  • drawRectangle(testImage); (line 88) click and drag the mouse in a rectangle over the image to mark the place where it was found. The drawRectangle() method is responsible for taking a screenshot and then saving the image.
  • testImage.clickOnCenter(); (line 91) click the middle of the image.
  • server.setCurrentItemResultToOK(); (line 93) set the current item result to OK. This accepts a string parameter where you can add a description of the processing result.

If the image is not found:

  • server.warn (line 97) send a descriptive log message of type Warn to the server.
  • server.setCurrentItemResultToWarn(); (line 98) set the current item result to Warn. This accepts a string parameter where you can add a description of the processing result.
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
public void searchImage() throws IOException, AWTException {
		
	IFalconImage testImage = images.getTestPng().search();		
	server.info("Searching the image: " + testImage.getDescription());
		
	// Sends the image to the trace
	falcon.sendImage(testImage.getImage(), "Test image");
		
	server.info("Desktop capture");
	server.sendScreen("Current desktop");
		
	// Image search on the desktop
	if (testImage.search().found()) {
			
		server.info("Image found at: " + testImage.getPointsWhereFound().get(0));

		// Draw a rectangle where the image was found
		drawRectangle(testImage);
			
		// Single left click over the image
		testImage.clickOnCenter();
			
		server.setCurrentItemResultToOK();	
	} else {		
		server.warn("Image not found");
		server.setCurrentItemResultToWarn();
	}
}
drawRectangle() method

The drawRectangle() method is called from the searchImage() method when the image is found. This method is responsible for taking a screenshot of the entire screen, drawing a red rectangle over the located image, and then saving the image.

Although this method is not required to use this template or the Falcon module, you can leverage this method to perform some advanced actions on an image identified by the process.

First the template will take a screenshot of the entire screen and save it to your support files.

  • Path screenshot = Paths.get(server.getCurrentDir(), "screenshot.png"); (line 113) defines the file path of the attribute screenshot. This path indicates the image screenshot.png will be saved in the support files.
  • ImageIO.write(server.getScreen(), "png", screenshot.toFile()); (line 114) takes a screenshot of the entire screen and writes the image to the file path listed in the screenshot attribute.

Next, the robotic process will add a red rectangle around the coordinates where the Firefox icon is located.

  • BufferedImage img = ImageIO.read(screenshot.toFile()); (line 116) decodes the screenshot and returns a BufferedImage, which provides image data, and stores it in the img attribute.
  • Graphics2D g2d = img.createGraphics(); (line 117) creates a Graphics2D object, which allows you to draw on the image stored in the img attribute. The following code snippets refer to the g2d attribute to draw the rectangle:
    • g2d.setStroke(new BasicStroke(3)); (line 121) determines the rectangle's line width.
    • g2d.setColor(Color.RED); (line 122) sets the color to red.
    • g2d.drawRect(p.x, p.y, imageFound.getRectangle().width, imageFound.getRectangle().height); (line 123) draws a rectangle on the coordinates of the found Firefox image.
    • g2d.dispose(); (line 124) indicates the end of the image modifications and releases the system resources using the graphic's context.
  • Point p = imageFound.getPointsWhereFound().get(0); (line 119) stores the Firefox image's coordinates in the p attribute.

Finally, the image is saved and the Falcon module will send the modified screenshot to your log.

  • String output = addSuffix(screenshot.toFile().getAbsolutePath(), "_mod"); (line 126) names the newly modified image screenshot_mod.png.
  • File f = new File(output); (line 128) creates a new File instance of the modified image using the output attribute.
  • ImageIO.write(img, "png", f); (line 130) writes the newly modified image as a new file.
  • server.info("Save image modified: " + output); (line 131) send a descriptive log message of type info to the server indicating the name of the newly modified image.
  • falcon.sendImage(ImageIO.read(new File(output)), output); (line 133) uses the Falcon module to send the modified screenshot to your log.
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
private String drawRectangle(IFalconImage imageFound) throws AWTException {

		try {
			
			Path screenshot = Paths.get(server.getCurrentDir(), "screenshot.png");
			ImageIO.write(server.getScreen(), "png", screenshot.toFile());
			
			BufferedImage img = ImageIO.read(screenshot.toFile());
			Graphics2D g2d = img.createGraphics();

			Point p = imageFound.getPointsWhereFound().get(0);

			g2d.setStroke(new BasicStroke(3));
			g2d.setColor(Color.RED);
			g2d.drawRect(p.x, p.y, imageFound.getRectangle().width, imageFound.getRectangle().height);
			g2d.dispose();

			String output = addSuffix(screenshot.toFile().getAbsolutePath(), "_mod");

			File f = new File(output);

			ImageIO.write(img, "png", f);
			server.info("Save image modified: " + output);
			
			falcon.sendImage(ImageIO.read(new File(output)), output);

			return output;
		} catch (IOException e) {
			return null;
		}

	}

end() method

After the searchImage() method, the process will move to the end() method (line 163) where the execution will finish. In your own robotic process, this is a good place to add final log messages that indicate the process has completed.

160
161
162
163
164
165
    /**
	 * End.
	 */
	public void end() {
		
	}
Open in Github

On This Page

FEEDBACK