Memory Circuit Breaker: Design Guidance and Troubleshooting

Overview

Expressions that use a large amount of memory can cause performance problems, both for that particular expression and possibly even the system as a whole. To prevent this, the Memory Circuit Breaker will automatically detect expressions that use too much memory and abort those evaluations to prevent the application server from running out of resources. Only the offending expression fails when the circuit breaker is tripped, allowing all other evaluations to complete successfully.

The threshold for the Memory Circuit Breaker is set at 100,000 AMUs, though that may change in the future. Based on our analysis of actual expression evaluations, this would abort less than 0.01% of all expression evaluations. This threshold is constant regardless of the max heap space configurations for your site, so the only way to avoid the circuit breaker is to modify your design.

It is important to design expressions to limit their memory footprint to ensure they will always evaluate successfully. All functions will use some amount of memory during evaluation, but some design patterns will use more memory than others or retain that memory for a longer period of time.

This page provides guidance on how to design memory-efficient expressions and diagnosing memory issues.

Designing memory-efficient expressions

When an expression uses too much memory, it is typically doing one of the following:

• storing too much data in local variables
• looping over too many items
• returning too many components or too much data

This section provides guidance on how to avoid these design patterns.

Page and filter query results

Large queries are one of the easiest ways to store too much data in local variables. When you query a lot of data and store it in a variable, that data is stored in memory for as long as that variable can be used. For local variables configured to refreshAlways, these values are stored for the entire evaluation of the a!localVariables() function. For local variables with any of the other refresh configurations, the values are stored across all evaluations on that interface. See the Interface Evaluation Lifecycle page for more information on how local variables are evaluated.

To avoid memory issues with storing too much data in local variables, always do at least one of the following:

1. Page your query results using a reasonable batch size. Unbounded queries (those that use a batch size of -1) can return an unpredictable amount of data, especially since your production environment will have a lot more data than your development environment where you initially build the query.
   • a!queryEntity, a!queryProcessAnalytics, a!queryRecordType, and query rules all support paging using a!pagingInfo
   • If you're using an integration, check whether the API supports paging. If it does not support paging, you will need to filter the results instead.
2. If you can't use paging, always apply filters to the data to limit the number of results. Be sure you understand how many results would be returned by a particular combination of filters, particularly if it varies based on the user's selection or group membership.
   • a!queryEntity, a!queryProcessAnalytics, and a!queryRecordType support filtering using a!queryFilter and a!queryLogicalExpression
   • Many integrations also support filtering by configuring HTTP query parameters. Check what parameters are supported by the API.

Limit looping iterations

Looping over a large array of data can use a large amount of memory. While a looping function is being evaluated, the results of each iteration are accumulated so they can be returned at the end of the function evaluation. Even if each iteration is only returning a very small value, the overall result can use a lot of memory if the number of iterations is very large.

This problem can be compounded if you have nested looping functions. Memory will be allocated for each looping function iteration across all nested loops and won't be reclaimed until the parent loop is completed.

To avoid memory issues with looping over too many items:

• Check that you actually need to use a looping function. Some functions can also work on an array of data, such as text() or if(). If you can easily achieve the same behavior without using a looping function, you shouldn't use one.
• Limit the number of items in your array by filtering or paging the data. In general, avoid looping over more than 500 items.
• Don't use deeply nested looping functions. In general, avoid nesting beyond 2 levels.

Avoid large interfaces

In reality, the result of your expression is just a piece of data, and interfaces are no exception. Interfaces that contain a large number of components or components that contain a large amount of data can cause memory issues when being processed by the system. This is usually more of a problem when the interface contains a dynamic number of components based on a large data set, such as a grid with a dynamic number of rows.

To avoid memory issues with too many components on an interface:

• Avoid dynamically generating components (including grid rows or chart series) based on an unbounded or large data set. Always limit the amount of data by paging or filtering before displaying on an interface.

Diagnosing memory issues

When Appian aborts an expression that is using too much memory, the error message will typically include information about the last function that was evaluating when the error occurred and its line number. This gives you a good starting point for diagnosing the issue.

However, because memory is accumulated throughout the course of the evaluation, it's possible that the function mentioned was just the tipping point, not the actual root cause. Here are some tips for diagnosing the issue:

1. When a user reports an error, use the evaluation ID in the error message to find the corresponding log entry in the Design Errors Log.
2. Use the object information provided in the Design Errors log to find the object that encountered the error. Open that object in your development environment.
   • If runtime information (such as record-related information or process instance information) is also provided, use that to look up the current state of that object in the environment that encountered the error. The current state of the data may be useful in debugging the issue.
3. If the error message contains references a particular function and line number:
   • Check the function that errored to see if it is using one of the problematic design patterns mentioned on this page (such as looping over too many items or storing too much data in a local variable)
   • If that function seems unlikely to have caused the issue on its own, check any local variables or looping functions that called the function that errored out. Because they're still evaluating, they're still accumulating memory.
   • If the parent functions also seem unlikely culprits, it's possible that the expression was evaluating in parallel and the issue was caused by a different function in the expression. Check other instances of local variables or looping functions throughout the expression for one of the problematic design patterns listed above.
4. If the error message does not reference a particular function or line number, check the overall result of your expression to see if it returns a large amount of data, a large number of components, or components with a large amount of data within them

Remember that local variables with a refresh configuration other than refreshAlways are persisted across evaluations of interfaces. Even if the initial value as configured in the expression is small, it could cause issues if a large value is saved into it on a subsequent reevaluation.

Feedback