Monitoring and Fixing Autoscaled Processes

Overview

As your autoscaled processes move through the different stages of work, you will want to keep an eye on them to ensure they are performing as expected. Being proactive in monitoring your autoscaled processes is important to catch spikes in errors and remediate long process wait times. On this page, you'll learn to use monitoring features to identify and investigate these issues. You will also learn how to fix processes with errors so you can resume work quickly.

View autoscaled process activity

The Autoscaled Process Activity tab, accessed from the Monitor view, is the central place to view information about the autoscaled processes that ran in a selected time period. High-level KPIs offer a view of how your process models are performing collectively. This data can be explored in more detail with the Process Models grid which breaks down these KPIs for each process model.

Since autoscaled processes generate large amounts of data, it is best to open the Monitor view inside the application you are working on. This will automatically limit the process models to those associated with the app, making it easier to find the processes you are interested in. You can also open the Monitor view from the Applications view, to see the process models across all applications in your environment.

Understand the available metrics

This annotated screenshot and the following descriptions define the buttons and grid columns appearing on this tab.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

1. Time period: The time period you want to explore with the KPIs and grid data. You can select from preset date ranges or choose a custom range of up to seven days. The custom range can be used to find process models with errors (up to 14 days old). See Process data retention for more information about data storage.
   All process models associated with the application are shown in the grid, regardless of activity in the selected time period. Process models without activity will display zero values. Learn more about how the time period impacts the metrics on this tab.
2. Errors KPI: The number of errors that have occurred in processes during the selected time period. This number is not restricted to processes that started running in that period; it includes all errors that occurred even if the process started before the selected period.
3. Started Processes KPI: The number of processes that were taken from the queue and started running during the selected time period. Processes that were only queued are not included.
4. Search by process UUID: Opens a dialog that lets you search for a process by its unique identifier, view information about when the process ran, and cancel or delete the process.
5. Process model search: Allows you to search for a process model by name.
6. Process Model: Lists all process models associated with the application. Click the link to see a list of processes run from this process model.
7. Occurred Errors: The number of errors that occurred in all processes during the selected time period. This number does not change if processes are deleted.
8. Queued Processes: The total number of processes added to the queue during the selected time period. This number does not change when processes start or complete.
9. Started Processes: The number of processes that started running during the selected time period. This number does not change when processes end.
10. Completed Processes: The number of processes that ended during the selected time period. This includes processes that completed or completed with errors. This count will not change if you delete a process.
11. Avg. Completion Time: The average amount of time it took for individual processes to run during the selected period. This calculation includes all completed processes (with and without errors).

Understand the time periods for process monitoring

Because autoscale is built to run large volumes of processes, the metrics on this page aggregate your environment's process data as a snapshot in time. The data is static, so deleting or canceling processes will not have an effect on the KPIs or values shown in the grid.

By default, the data shown on the tab is the last hour of process activity. In the following image, this hour-long snapshot is the clear area between the gray ends of the timeline.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Events occurring outside the snapshot's period—like the five queued events that happened before the last hour—are not included in the metrics.

View metrics for processes

When you click a process model name in the Process Models grid, you will see a list of recent processes for that process model.

This annotated screenshot and the following descriptions define the buttons and grid columns appearing on this tab.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

1. Search: Allows you to search to find a specific process by name. The search is case sensitive.
2. Status: The status of the processes you want to see in the grid. You can select stopped, active, completed, or canceled processes.
3. Error State: Whether the processes shown in the grid have errors.
4. Start Time: The start time of processes you want to see in the grid.
5. Show process details: Toggles the Process UUID and Queued Time for each process.

The grid has a number of columns with details about your processes.

Column	Description
Select (checkbox)	Selects the process to cancel or delete.
State (icon)	The current state of the process.
Process Name	The name of the process. Click the name to open a process in the process modeler's monitoring mode.
Model Version	The version of the process model.
Errors	The number of errors that the process encountered.
Process UUID	The unique identifier of the process. Only shown when Show process details is selected. This value is accessible using the pp!instanceUuid process property.
Started By	The user or other actor that started the process.
Queued Time	The date and time the process was added to the queue. Only shown when Show process details is selected.
Time in Queue	The amount of time the process waited in the queue.
Start Time	The date and time the process's work was started. You can sort the grid by earliest or most recent start time by clicking the header cell. This value is also accessible using the pp!startTime process property.
End Time	The date and time the process reached all end nodes or a terminate event. If the process is Active with Errors or otherwise incomplete, no data is shown.
Process Duration	The amount of time between the process start time and end time.

Deleting processes

Appian automatically deletes older autoscaled processes, but you can manually delete processes if needed. You can specify the processes you want to delete using the filters for Status, Error and Start Time.

To delete a process or group of processes:

1. Select the processes in the grid (up to 25 for each page of results).
2. Click Delete.

Canceling active processes

When running processes at high scale, you may need or want to prevent a set of active processes from completing. For example, if an infinite loop was inadvertently created in one of your process models, it would be important to cancel any active instances of this broken process. Or you may want to cancel processes that cannot progress because a synchronous child process failed.

You can use the grid to select each process you want to cancel, or you can let Appian select all processes matching your filter selections.

To cancel a specific process or group of active processes:

1. Choose the processes you want to delete by selecting their checkboxes (up to 25 for each page of results).
2. Click Cancel.
3. Click Cancel Process.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

You can also cancel all of the active processes matching your filter selections in a single operation. Your selection can include processes in any status, but only the active processes will be canceled.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

To cancel all processes all matching your filters:

1. Use the Name, Status, Error, andStart Time filters to find the processes you want to cancel.
2. Select all 25 processes on the first page of results.
3. In the banner, click Select all processes that match current filter criteria.
4. Click Cancel.
5. On the confirmation dialog, verify the filters are correct, and then do one of the following:
   1. To cancel just the processes matching the filters, click CANCEL PROCESSES.
   2. To cancel the matching processes and any matching processes in the queue, select Cancel all processes for this model that have not yet started. Then, click CANCEL AND CLEAR QUEUE.

Cancellation details can also be viewed in the process history.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Search for processes

Running processes at high scale results in a large amount of monitoring data, so Appian gives you different ways to search and filter this critical information resource. Depending on what you need to know or view, you can use one of these approaches: searching by name or searching by UUID.

Search by name

When viewing the grid of recent processes for a process model, you can use the search and filters to narrow down the list of processes by their names.

There are a few important strategies to keep in mind for the most efficient searches across this large dataset. When searching, keep these tips in mind:

• Constrain the data to be searched.

Start by selecting a status and narrowing the time range as much as possible. This will shrink the group of processes that Appian needs to search once you enter the search term.

• Multi-word search terms are always treated as the whole phrase and must match exactly.

For example, if you search for silver sedan, only processes with that string would be returned. If the name includes only silver or sedan, the process would not appear in the results.

• Search is case sensitive.

Let's say the process names include the customer's first and last name (such as Clark Simon). If you searched for either clark or simon, Appian would not find matching processes for that customer.

Search by UUID

You can also search for a process using its unique identifier (UUID). This search method is most helpful when you're trying to troubleshoot a problem in a specific process.

Appian does not automatically send alerts or notifications for autoscaled process errors. Instead, we recommend using the process's UUID—available as the process property pp!instanceUuid—to design a recordkeeping or alerting solution that works best for your application.

For example, you may want to include the UUID in the process name so it's highly visible. You could also write the value to a data source or set up an error flow in the process model to send an email with the UUID.

However you make the UUID available, finding the process is straightforward.

To search for a process by its UUID:

1. On the Autoscaled Process Activity tab, click SEARCH BY PROCESS UUID.
2. In the Search by Process UUID field, enter a UUID.

On the Search for Process dialog, you can see basic information about the process.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Monitor a process

You can explore and debug a specific process using the data available in the process modeler's Monitor mode. This mode lets you examine the flow of the process, view the process's variables, and check any errors that occurred.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

The process's nodes and flow arrows are shown with different colored borders depending on their status.

Format	Status
Blue	Completed
Green	Active
Red with a slash (Ø) through the object	One or more errors occurred. If the process spawns multiple instances of a node, the red slash will appear if any instance encounters an error.
Gray with a slash (Ø) through the object	The node was skipped.
No color	Not yet activated

View process details

To see more information about the variables and any errors, click Process Details on the process modeler toolbar.

The Variables tab shows the final value of each process variable. You can change the Format Value from the default of Formatted to Raw or Expression.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

The History tab shows the status of the whole process above the grid and the status of each node in its own row. The Node column shows the display name and the type of each node.

If one of the nodes starts a separate autoscaled process, click Open Monitoring to open the other process in monitoring mode.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Changes to process variables are recorded at each node where a change is made. Click the name of a variable to see its value as it was at the end of that node.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

The Errors tab lists any nodes that encountered an error and the details of the problem.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

View step-by-step playback

When investigating a problem in production or debugging a process, it's important to see how the process ran and how its data changed at each step. The step-by-step playback tool lets you click through each node, examine the variables that were updated, and check that the duration is within the expected timeframe.

Click Step-by-Step Playback on the process modeler toolbar to open the playback pane.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

The start node is shown by default. Click NEXT NODE to highlight the next node in the flow. For a process with many nodes, click SKIP 10 NODES to jump ahead to a later point in the process.

The NODE DETAILS tab shows the node's name and its execution time data. If the node is a Start Process node, a link is shown so you can open the other process for monitoring.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

If the current node did not run successfully, the Errors section provides the error message so you can understand what went wrong.

The VARIABLES CHANGED tab lists any variables whose value changed when the node ran. Expand each variable to see the data type and final value of the changed variables. You can also how the variables are displayed from the default of Formatted to Raw or Expression.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Error dashboard

The error dashboard provides a dedicated interface for investigating errors in your autoscaled processes. This dashboard helps operations teams quickly assess the range of errors occurring in process models to determine if errors can be resolved with a single remediation action or if fixing the problem requires multiple steps.

To access the error dashboard:

1. Navigate to the Autoscaled Process Activity tab.
2. For the process model you want to view, click the number in the Occurred Errors column to open the error dashboard.

View error data

The error dashboard provides two complementary panes for error analysis: the node errors pane (1) and the error instances pane (2). The whole dashboard can be filtered by choosing a value from the Time Period dropdown (3).

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

The node errors pane aggregates node errors by the types of issues the node encountered.

This view helps you with the following tasks:

• Identifying which nodes are experiencing errors.
• Understanding the error types occurring at each node.
• Assessing total error counts to prioritize remediation efforts.

The error instances pane contains a filterable grid that shows individual error occurrences and detailed error messages containing runtime context. Clicking an error in the node errors pane switches the content of the grid to your selection.

Column	Description
Node Instance UUID	The unique identifier of the node instance where the error occurred.
Process Name	The name of the process instance that encountered the error.
Version	The version of the process model when the error occurred.
Started By	The user or other actor that started the process instance.
Error Message	Specific error messages with runtime information.
Occurred	The date and time when the error occurred.
Resolved	The date and time the error was addressed and who resolved the error.

You can filter errors by any of the following factors:

• Time period when errors occurred.
• Node instance UUID.
• Process name.
• Process model version.
• Error message text.
• Process status (active, stopped, completed).
• User that started the process.
• Error resolution state.

This filtering helps you focus your investigation and identify patterns in error occurrences.

Fix process errors

Any process can run into unexpected issues, so Appian lets you take remedial action on errored processes and get your workflow back on track quickly.

Fixing an errored process can require one or more actions:

• Changing process variables.
• Restarting the node.
• Skipping the node.
• Bulk remediation for multiple errors with the same root cause.

To start, you need to open the process in monitoring mode. This can be done from the Autoscaled Process Activity tab or the error dashboard.

Changing process variables

It's possible to change the value of variables in an errored process so you can get the process back on track.

To change process variables:

1. From the toolbar, click Process Model Properties to view the process details.
2. In the variables grid, click Edit for the variable you want to change.
3. In the dialog that appears, enter the new value and click OK. The new value is shown in the Pending Expression column.
   

   Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

4. When you are done changing variables, click SAVE AND CLOSE.

Restart a node with errors

If the errored node needs to run for the process to complete, right-click the node and select Restart to launch a new instance of the node. If the node has multiple instances with errors, you can restart all of the instances by selecting Restart number instances.

When deciding to restart an errored node, keep the following in mind:

• The process will change state to either Active (if the restarted node was the only one with errors) or Active with Errors (if there are more nodes with errors in the same process).
• Any process variables you've changed will use their updated values if referenced in the node inputs.
• A restarted node runs using the same security context as if the process is running for the first time.
• If the process is stopped, completed, or canceled and has an end time, the end time and scheduled deletion will be cleared when you restart. After restart, the process is retained an additional 7 or 14 days from the time it stops.

Skip a node with errors

Some processes can continue without the output of the errored node. In that case, right-click the node and select Skip to continue the process on any outgoing flows. If the node has multiple instances with errors, you can skip all of the instances by selecting Skip number instances.

When deciding to skip an errored node, keep the following in mind:

• None of the results or custom outputs defined on the node's Outputs tab will be saved to process variables when the node is skipped.
• Skipping a node does not roll back any effects that have already occurred (such as writing a record or calling an integration).
• Gateway nodes (such as XOR) can only be restarted, since they determine which outgoing path or paths to take.

Bulk remediation

When you have multiple errors with the same root cause, you can remediate them in one operation instead of addressing each error individually. This is done from the error dashboard.

To perform bulk remediation:

1. In the error instances grid, select the checkboxes for the errors you want to fix:
   • Select individual errors by checking their row checkboxes.
   • Select all errors on the current page by checking the header checkbox.
   • To select all errors across all pages, first select all on the current page, then click Select all unresolved errors that match current filter criteria in the banner that appears.
2. Click RESTART FAILED NODE or SKIP NODE AND CONTINUE.
3. Review the confirmation details showing the number of errors and processes that will be affected.
4. To confirm, click RESTART FAILED NODES or SKIP AND CONTINUE.

The bulk operation is processed in the background. You can see the result by refreshing the error dashboard. All individual remediation actions are recorded in the process history for each process.

View resolved errors

Once you've made changes to your process and resolved the issue by skipping or restarting nodes, you can see this reflected in the process history.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

In monitoring mode, you can filter the error dashboard by All or Resolved to see which errors have been fixed, who fixed them, and when.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Email error alerts

When an autoscaled process encounters an error, an email alert is automatically sent to the process's administrators. This email provides basic information about the error and a link to the Autoscaled Process Activity page so you can investigate further. If your team uses an incident management tool like PagerDuty or OpsGenie, you can use these emails to start an incident response.

How email alerts work

Let's say you have an autoscaled process that writes to a data store. If there is a network issue and Appian can't connect to the database, your process model could see a large number of errors in a brief period.

To inform you of the problem quickly, an email will be sent when a process model begins to encounter errors. After that, if errors continue to occur for that process model, Appian sends a digest of these every 5 minutes until the errors stop occurring.

To see all the instances of an error, you can either filter the list of processes to show those with the Errored status, or use the error dashboard to view detailed error information aggregated by node and error type.

Email alert permissions

To access the process model from the link in the email, you must have the Designer user role assigned, which gives access to Appian Designer.

To receive email error alerts for an autoscaled process model, make sure:

• Outbound emails are turned on for the environment.
• You have Administrator permissions to the process model object in that environment.

Process states

When you click on a specific process model, you will see a grid of all processes included in the selected time period. In this grid, each process can have one of the following states:

Icon	State	Description
	Active	The process is currently running
	Active with Errors	The process is currently running but one or more errors have occurred
	Canceled	The process was manually canceled
	Canceled with Errors	The process was manually canceled and one or more errors occurred
	Completed	The process ran and reached one end event with a terminate trigger or reached all end events defined in the process model
	Completed with Errors	An error occurred, but the process ran and reached one end event with a terminate trigger or reached all end events defined in the process model
	Stopped	The process did not reach all end events nor an end event with a terminate trigger, and Appian cannot make further progress in the flow. This can occur when a process was canceled or completed via a terminate event, but the process is later reactivated by restarting or skipping a node.
	Stopped with Errors	Appian encountered an error and cannot make further progress in the flow

Process data retention

Autoscaled processes create large volumes of data to power your troubleshooting with the process activity and process history. Appian automatically removes older data at regular intervals to ensure your site remains performant.

The following data retention rules are in place for autoscaled processes:

• Data for processes that ended without errors is automatically deleted after seven days.
• Data for processes that ended with errors is automatically deleted after 14 days.
• Data for active processes is automatically deleted after 90 days.
• Aggregated data for the Autoscaled Process Activity tab is automatically deleted after 30 days.

When viewing process model activity, you can use a custom range to select any seven-day time period. This lets you view metrics for older processes with errors.

Logging

When you run autoscaled processes, Appian logs data for each runner in its own folder of the shared-logs/actor-executors directory. The logs available in each appian-actor-executor folder are a subset of those available for the main Appian application.

• audit/decryption.csv
• data-server/client.log
• perflogs/
• perflogs/epex_chain_trace.csv
• perflogs/epex_chain_details.csv
• perflogs/epex_chain_summary.csv
• search-server/search-server-replication.log
• app-server-jwq-monitor.log
• data-server-client-errors-detail.log
• data-type-locks.log
• deployment.log
• engine-client.log
• object-template.log
• tomcat-stdOut.log

Feedback