Sync Records Smart Service

Overview

The Sync Records smart service allows you to sync a record or set of records in the specified record type. This smart service is supported for all record types with data sync enabled. If you want to sync data in a service-backed record type, you must first configure a sync expression.

Permissions needed

The user executing this activity must have Viewer permission on the selected record type in order to trigger a sync.

Properties

• Category: Data Services
• Icon:
• Assignment Options: Unattended

Configuration options

This section contains tab configuration details specific to this smart service. For more information about common configurations, see the Process Node Properties page.

Data tab

Node inputs

See also: Record Type

Node outputs

Usage considerations

Syncing data in a service-backed record type

To use this smart service with a service-backed record type, you must first configure a Sync Expression on your record type.

A Sync Expression is similar to a Record Data Source expression; the only difference is that you pass individual record identifiers to your Sync Expression to fetch and return a row or set of rows from the web service. This enables Appian to sync specific rows of data from the web service when you use the Sync Records smart service.

Learn how to generate a Sync Expression on your record type.

Syncing more than one record type in a single process

This smart service can only sync data for a single record type.

To sync data for more than one record type, consider creating a process model that uses multiple instances of the node (one for each record type).

Syncing data deletions

In addition to syncing new or changed data, the Sync Records smart service can also sync data deletions from the source system.

If you pass a primary key value into the smart service, and that value is not in the source system, then that data will also be deleted in Appian. Any deleted records will not appear in the List of IDs returned in the smart service output.

For example, imagine you have a Employee record type that syncs inventory information from an external system. Appian is not used to make changes to any of this data, but staffing statistics are displayed in charts alongside other business metrics in reports. When employees leave the company, this data is updated in the external system, and any deleted records should be removed from Appian to provide an accurate report.

If the external system sends a webhook message for these events, you can configure a web API with Sync Records to sync the deletion.

If webhooks aren't available, you can sync additions, updates, and deletions by enumerating the list of record IDs and looping to sync the changes.

Syncing more than 1,000 records

For large datasets, the 1,000-records-per-request limit means you may need to run multiple instances of the Sync Records smart service when that data changes. One way to do this is to add a loop to your process model to repeat the Sync Records operation.

Each iteration of the loop can sync up to 1,000 unique records. To get the next batch of unique records, you can use a script task to increment a counter to get the next set of records to sync. See this sync example for a full explanation.

Sync Records and the record type row limit

Each record type with data sync enabled can sync up to 4 million rows of data.

If your record type exceeds the synced row limit after the Sync Records smart service executes, the write will be successful and the sync will succeed as long as:

• The record type has Keep data available at high volumes enabled.
• The record type is within 10% of the synced row limit.

Otherwise, the write will succeed but the sync will fail and your record data will be unavailable.

Using the onSuccess parameter

When setting the onSuccess parameter, you can use fv!recordsUpdated to reference the updated records in the response. The record type's fields are accessed using the record type domain (recordType!).

For example, to return the names for a Customer record type backed by a web service, write an expression similar to this example:

1
2
3
4
5
6
7
8
9
10
11
a!syncRecords(
  recordType: recordType!Customer,
  identifiers: ri!identifiers,
  onSuccess: a!httpResponse(
    statusCode: 200,
    headers: {
      a!httpHeader(name: "Content-Type", value: "application/json")
    },
    body: "Customers updated: " & `fv!recordsUpdated[recordType!Customer.fields.fullName]`
  )
)

If you are displaying this Customer record type in an interface, you can use a!save() in the onSuccess parameter to update a local variable with the latest data available in fv!recordsUpdated.

1
2
3
4
5
6
7
a!syncRecords(
  recordType: recordType!Customer,
  identifiers: ri!identifiers,
  onSuccess: a!save(
    local!records, fv!recordsUpdated    
  )
)

Automatic refresh on record views

When you use this smart service on a record view, the view will automatically refresh with the latest record data.

This means that when your record view interface has a Record Action component that triggers the Sync Records smart service, or a button component that triggers the a!syncRecords() function, the record view data will automatically refresh after the sync occurs.

This refresh behavior only applies to data supplied by rv!record. If your record view definition uses rv!identifier or a local variable to supply the record data, the data will not automatically refresh; instead, you will need to configure additional refresh behavior.

a!syncRecords()

The Sync Records smart service is available as an expression function that can be executed inside a saveInto on a Interface Component or as part of a web API.

Syntax

a!syncRecords( recordType, identifiers, onSuccess, onError )

Parameters

Possible errors

Appian will return an error when there is a problem syncing data for a record type. This section explains some of the common reasons that errors occur, and which errors will trigger an automatic retry of your sync.

Security and permissions

By default, the smart service is run as whoever starts the process. If that user does not have Viewer permission for the record type being synced, the sync is not triggered. The node will fail and pause the process with an exception.

Data type mismatch

The data type of the items listed in the identifiers parameter must match the type of the source's primary key field. If a mismatch occurs, the node will fail and pause the process with an exception.

Missing sync expression

If you use this smart service to sync data in a service-backed record type, you must configure a Sync Expression.

Without this expression, Appian cannot request the correct data from the web service. If the expression is missing, the record type will not be able to sync the specified records and the node will fail. The record type will continue to be available after this error is returned.

Integration errors

The Sync Expression uses an integration to communicate with the source system.

If a sync fails due to an issue with the integration, the error will be reported through the expression and appear on the interface where the expression is called. The record data may be unavailable until the issue is corrected and a successful sync occurs.

Automatic retries

In the following cases, Appian will automatically retry the sync operation when the smart service fails:

Appian will retry the sync operation up to three times (with two seconds between each retry). After three attempts, the sync will fail and the record data will be unavailable.

See Troubleshooting Data Sync for more information on the different sync errors.

Examples

You can use this smart service in a variety of ways to keep your data fresh.

Trigger a sync from a web API

Let’s say you have a Case record type that uses Salesforce as the source. When cases are logged in Salesforce, you need to immediately sync those changes in Appian.

To sync the new or changed cases, you can generate a web API in Appian that includes the a!syncRecords() function so you can specify the record type to sync when a change occurs in the source. Then, you can add the web API URL in Salesforce so it can notify Appian of any changes and trigger the sync.

Learn how to generate a web API to sync records.

Trigger a sync from an interface

But perhaps you want to re-sync specific records on demand. For example, if you have a report that shows all critical cases and their status, you may want to refresh this report every so often to ensure you see the latest information about those critical cases.

To refresh (or re-sync) specific cases, you can add a button component to your report and configure the a!syncRecords() function as the saveInto value. This way, anytime a user clicks the button, the specified list of records is re-synced from the source.

Trigger a sync from a process model

Alternatively, you may have an existing process model that executes a stored procedure in order to update your source data.

To sync changes created by your stored procedure, you can add the Sync Records smart service to your process model. This way, any data changed by the stored procedure is fetched from the source and synced in Appian.

Sync more than 1,000 records

Imagine you have a database-backed Item record type with thousands of records. The record type syncs inventory information from an external system. Appian is not used to make changes to any of this data, but inventory statistics are displayed in charts alongside other business metrics in reports. Users of the reports need updated information throughout the day, but the external system does not provide webhook notifications when data changes.

To meet this need, you can build a looping process model with a Sync Records node. This process will sync all data from the source:

To store information used at different points in the process, three variables are needed:

• pv!pageNumber is a number that is incremented with each iteration of the loop. Its initial value is 1.
• pv!recordIds is a list of primary key values (numbers) passed into the Identifiers node input.
• pv!updatedRecords is a Boolean used by the XOR node to decide when to end the process.

Once you have your process variables, configure an expression in the output of a Script Task node.

1
enumerate(1000) + (pv!pageNumber - 1) * 1000 + 1

The expression takes the current value of pv!pageNumber and returns a list of IDs stored as pv!recordIds. The first page of data will return a list from 1 to 1000, and a second page will be 1001 to 2000.

Next, the Sync Records node is run using the list of identifiers in pv!recordIds. A custom output is used to check if any values are returned in the Records Updated output. When any records are synced, pv!updatedRecords is set to true; otherwise, pv!updatedRecords is set to false.

After the first set of records is synced, the XOR node checks if pv!updatedRecords is true. If it is, a second script task increments pv!pageNumber by 1 and the process repeats.

Say the Item record type has 1,890 records. After the first 1,000 records are synced, 890 remain. During the second iteration of the loop, the expression will return the next 1,000 identifiers, and the corresponding records will be synced.

Because 890 records were synced, the process continues, and pv!pageNumber will be set to 3. During the third loop, pv!recordIds will be set to a list from 2001 to 3000. The Sync Records node will not update any records, pv!updatedRecords will be set to false, and the XOR node will route the process to the end node.

Feature compatibility

Old versions

There are older versions of this smart service. You can identify older versions by looking at the name to see if there is a version suffix. If you are using an old version, be sure to refer to the corresponding documentation from the list below.

To use the latest version of the smart service node:

1. Drag in a new Sync Records Smart Service node from the palette, and place it on the same connector as the current node.
2. Configure the new node with settings matching your current, older node.
3. Delete the old Sync Records Smart Service node.

To learn more about how Appian handles this kind of versioning, see the Function and Component Versions page.

Feedback