This document provides general guidance and best practices for testing expression rules with the test cases feature.
For information on automated testing strategies and testing rules in bulk, see: Automated Testing for Expression Rules
Test cases aren't just useful for teams that have large testing infrastructures or use automated testing, they can help improve the maintenance of individual rules as well.
Save Ad-Hoc Tests
Inline Documentation
Test-Driven Design
Use Cases
Edge Cases
Regression Testing
The following list of guidelines can help direct your testing efforts and avoid common mistakes. These guidelines have been generalized; for examples on how they are specifically applied, see the Common Testing Scenarios section below.
Test only your logic. Make sure your test cases are testing the logic of your rule and not testing Appian built-in logic. You can rely on Appian to function as expected. For example, don’t test that the sum() function can add numbers. If your rule is taking numerical values from multiple sources and summing them together, you may want to create tests that ensure the inputs are of the correct type or that it can handle nulls, but you should not test that 2 + 2 = 4.
Make tests as specific as possible. Isolate one part of the expression rule when creating a test case, so the result of the test execution is more specific. For example, if your rule can have three different results, instead of having one test that checks three parts of a rule, have three separate tests per expected result.
Make sure the test isn’t testing itself. If the assertion is the same as the expression rule definition, it’s what we call a "1=1 test," and we don’t learn anything from it. This is a surprisingly easy mistake to make.
Write reliable tests. Tests that incorporate values from an external system (either via a query, or using a date/time function, etc.) in their definitions are considered "fragile," as their results can be affected by changes to those external sources. For example, a test that queries a table for the values in select rows could fail when the values in those rows are changed by another process; in this case the test failure would not reflect a failing of the rule's logic.
Test appropriate expression rules. Not all expression rules can be effectively tested. There are some rules, such as rules that query transactional (non-reference) data, that do not need test cases, because the data is different on each environment. To learn more about when to write test cases, see the Playbook article on Expression Rule Test Cases.
Writing and maintaining unit tests is an integral part of ensuring the long term success of an application. The Manage Test Cases dialog can assist you with this maintenance. For more information on writing test cases for expression rules, see Expression Rule Test Cases.
Manage Test Cases allows you to manage the test cases for all of the expression rules in your application. From this view you can identify rules without any test coverage (rules without test cases), run the test cases for any number of rules in your application, and review the results of the previous run in order to address any failures.
Throughout development you should ensure that you have comprehensive test coverage for your expression rules, and that all of test cases for those rules are passing. Appian recommends running the test cases for all rules in your package at least once directly prior to deployment.
To view test cases, open the settings menu , then select Manage Test Cases.
When you open the Manage Test Cases dialog, you will see the following:
Tip: Manage Test Cases only displays previous test case results for test cases that were run from inside of Manage Test Cases, the Start All Rule Tests smart service, or the Start Application Rule Tests smart service. Test case runs from within an expression rule’s test pane do not have their results persisted and their results will not be reflected within the Manage Test Cases dialog.
You can also manage and execute your test cases when comparing and deploying your packages. During direct deployments you will be warned if any packaged rules have outdated test data, failing test cases, or are missing test coverage.
Once your tests have finished executing you can select an expression rule from the rules grid on the left hand side of the dialog and see its test case results in more detail in the rule results grid on the right hand side.
Clicking on a test case's name from the rule results grid provides you with more information about that test case’s execution including its status, execution time, rule inputs, and results.
The following scenarios describe common expression rule uses and how they can be tested. Each of these scenarios reflects different guidance and practices related to testing expression rules.
Testing a rule that returns an array of years based on the current year (uses the today() function). This scenario reflects guidance on:
Testing a rule that takes a CDT array and returns a filtered subset of that CDT array. This scenario reflects guidance on:
Testing a rule that validates whether the provided text input is a valid email address. This scenario reflects guidance on:
In this scenario, we have a rule that takes a single year as an input and returns an array of the years between the input year and the current year.
We use this rule to populate a dropdown for an application that allows employees to retrieve a copy of their W2 from a select year. In this case we pass the employee’s start year to the rule.
If that year was 2008, we would get a list of values as seen in the dropdown in the image below.
startYear (type: Integer)
1
2
3
4
5
a!localVariables(
local!numYearsSinceStartYear: year(today()) - ri!startYear,
local!arrayOfIndices: enumerate(local!numYearsSinceStartYear + 1),
reverse(local!arrayOfIndices + ri!startYear)
)
Because this rule uses the today() function, the same input will result in a different output depending on the year that it is run. If we create a test case with 2008 as the input, and we run it in 2016, the output will be an array of 9 values:
{2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016}
If we set that array as the asserted output, the test case will pass in 2016, but fail when run in 2017 because the same test case will return 10 values:
{2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017}
This is a clear sign that we should avoid asserting the value of this rule directly — any tests that assert a value from something that is dynamic or context dependent like this is likely fragile.
Since we can’t assert the literal results in this case, we might try to write an expression to calculate the array to compare. So, maybe we would write an expression that would calculate the array of values, like this:
1
2
3
4
5
a!localVariables(
local!numYearsSinceStartYear: year(today()) - 2008,
local!arrayOfIndices: enumerate(local!numYearsSinceStartYear + 1),
reverse(local!arrayOfIndices + 2008)
)
But, in doing so, we’ve basically rewritten the original expression rule, which doesn’t actually test anything; since we’re rewriting the logic of the expression rule, it’ll just output the rule we’re seeking to test. We call this a "1=1 test."
So, how do we actually test our rule in this situation? Well, instead of testing that the result of one test case matches the result of running the entirety of the expression rule, we’ll break down the result of the rule into testable components.
Here we assert that when the start year is the current year, we should always expect the rule to output only a single year. We don't need to assert that the year is 2016, 2017, or 2034 because we are confident that year(today()) is always going to return the right current year in the expression rule. Our tests don’t need to test Appian functionality.
Here we assert that we'll always have a certain number of values in the output array given the number of years ago the start date is. A common error in rule logic is that values are sometimes off by one index. Testing for 1 and for 6 years is enough coverage that if they both pass, we can safely assume all intervening and exceeding years will work as well because the logic is straight-forward, and we are still relying on Appian functions to work as intended.
When writing test cases, we should also keep in mind how other application objects use this rule, and what assumptions that are making about what the rule returns. For example, another rule might take the output of this rule and use it as the list parameter for the apply function. If this sometimes returned a scalar, then the apply function would break. We should protect & document these types of assumptions in test cases.
Above we are asserting that even if we only return a single year that it’s an array type. This test case can serve as documentation for other developers to understand that this is the expected behavior; if they make changes that affect the output data type, this test will fail.
Weeks later it turns out there is a bug where the rule breaks when a future year is passed into it, so we add some additional logic to handle this case. To fix this bug, we'll add logic to check whether the number of years since the start date is negative; if so, we'll return an empty integer array to handle this case.
1
2
3
4
5
6
7
8
9
a!localVariables(
local!numYearsSinceStartYear: year(today()) - ri!startYear,
local!numYearsIsNegative: local!numYearsSinceStartYear < -1,
if(
local!numYearsIsNegative,
tointeger({}),
reverse(enumerate(local!numYearsSinceStartYear + 1) + ri!startYear)
)
)
We'll add another test case that asserts the output value is an empty integer array for any input values that are future start years.
In this scenario, we have a rule that takes a set of purchase requests (a CDT array) and a type (like stationary, or electronic equipment, or office furniture), and returns only the requests that match the type. This is used by multiple parts of an application for ordering office supplies. One process model queries for all new purchase requests and sends them to this rule to be sorted by type and sent to the appropriate department.
purchaseRequests (type: Purchase Request [Array])
type (type: Text)
1
2
3
4
5
6
a!localVariables(
/* Find the indices of the purchase requests of a given type, then */
/* use those indices to only return those items from the array */
local!filteredIndices: wherecontains(ri!type,index(ri!purchaseRequests,"type",{})),
index(ri!purchaseRequests, local!filteredIndices,{})
)
Because the input to this rule will be different depending on the day (since we are querying a database that is constantly being updated), we can’t create a test output that will consistently be the same. We don’t want to run the same query on the purchase requests table and use its result as the input value here, because it’ll be different from moment to moment. So, we need to define a static set of purchase requests to test against to safeguard against making the test fragile.
Since our rule only filters a set of data, that’s the only part we really need to test. We don't need to test what the entire CDT definition of a purchase request data type looks like, and we don't need to test the values of all the fields in the purchase request because they aren't relevant here. So, we need to create our data set with purchase requests of different types and check to make sure that the result of running the rule only contains requests of a single type.
We came up with the following test cases to ensure our rule is working as intended.
Since we know exactly what fields of the purchaseRequest input the rule needs, we can hardcode some example data by just adding a dictionary with the fields of the CDT.
Since the rule input is already of type array of PurchaseRequest, any values we pass into that rule input will be cast to the appropriate type. We don't need to query on real data, and we can hardcode our types to avoid any brittleness without sacrificing test coverage. We can specify the test input as the expression below:
1
2
3
4
5
6
{
{type:"OFFICE"},
{type: "ELECTRONICS"},
{type:"PAPER"},
{type: "ELECTRONICS"}
}
Notice how we didn't add any of the other fields to the CDT, we know we don't care about these for this specific rule, so don't bother!
This tests that a set of Purchase Requests with varying purchase types will filter out all requests that don’t match the specified type. We check that the remaining purchase requests don’t have any with a different type than the one specified. We've effectively tested the logic of this rule without having to rely on any actual data from an external source.
Another aspect we might want to verify is that the expected number of items was returned. If we expect 7 results, we can check to make sure we get 7 unique results.
We can use the same test inputs for this case, so let's duplicate the test case (select it in the test cases grid and click the duplicate button) and assert the length of the test output.
Though the test cases in this scenario have relatively simple assertions you should always make separate test cases for them. This will assist you when troubleshooting the rule - if the assertions were all in one test case then you wouldn't immediately know if the output was wrong because it is returning the wrong status values or it's not returning all the items with a given status value.
In this scenario we have a rule that takes a text input, performs a lot of different logic to ensure it is in the format of a valid email address, and returns a boolean corresponding to whether or not it is a valid email address format.
We use this rule to validate a user's input on an email address field on a customer onboarding form and block the user from submitting the form until they provide a valid email address.
address (type: Text)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
if(
or(
len(trim(ri!address)) > 255,
length(split(trim(ri!address), " ")) > 1,
count(split(ri!address, "@")) <> 2
),
false(),
a!localVariables(
local!localPart: split(trim(ri!address),"@")[1],
local!domainPart: split(trim(ri!address),"@")[2],
if(
or(
length(split(local!domainPart, ".")) < 2,
contains(split(local!localPart, "."), ""),
contains(split(local!domainPart, "."), ""),
not(isnull(stripwith(lower(local!domainPart), "abcdefghijklmnopqrstuvwxyz1234567890-."))),
not(isnull(stripwith(lower(local!localPart), "abcdefghijklmnopqrstuvwxyz1234567890-._+'&%")))
),
false(),
true()
)
)
)
When first identifying that you need to create this rule, it is likely that you are thinking about examples of valid and invalid email addresses to understand what rule's behavior will need to be. This rule actually has a lot of separate pieces of logic working together here - this is a good opportunity to use Test-Driven Development when writing this rule.
In Test-Driven Development you'd start creating a test case for each example email address strings you need to validate and whether or not they'd return true or false. Then once you were finished, you would start writing the expression rule itself and continuously test it via the test cases grid to start watching tests pass as you implement more and more of the necessary logic. This would help ensure that none of the relevant examples we are aiming to support are missing.
This rule has a lot of separate pieces of logic working together and it will be up to you to determine how much of it you'll want to document and test. In the example below we are testing that all the logic pertaining to the @ symbol works properly:
Since this rule will be used by many other rules, we'll also want to make sure that we document what the output type will be in most occasions. The next two test cases below ensure that even in edge cases, the rule still returns a scalar boolean value.
Rules that have a lot of unrelated logic in them (like this one) should have test cases that describe each of these parts of logic separately. Like we mentioned in the CDT rule example, even though these are simple assertions and you could verify many of them together by just using and() in your assertion expression, separating your assertions to multiple test cases will make debugging your rule easier. Having multiple test cases is a good way to define the expected behavior of the rule, which isn't possible to decipher from a single test case with a complicated assertion.
Expression Rule Testing with Appian
