This page explains what it means to enable data sync in your record type. This page also provides guidance on when to enable sync, and how to monitor and keep your synced data up-to-date.
If you encounter an error while enabling sync, see Troubleshooting Syncs.
Data sync allows you to operate with the benefits of onsite data, while leaving your source data where it is. When you enable data sync, you are caching your source data in Appian. With a cache of your data, this means Appian will only have to execute queries from the cached data instead of the external source whenever you view or interact with the record data.
Think of data sync like a cache on your web browser. When you cache local data on your browser, your site content loads faster and improves the speed of your browsing. The same applies when you sync your source data—your record data will load faster for more performant queries and reports.
While your synced data allows for faster queries, your source data is still what backs the record type. This means that any updates or changes to your source data will need to be re-queried (or re-synced) in Appian.
To keep your synced data up-to-date, any changes to your source data written by Appian will be immediately synced and reflected in the record type. You can also schedule a daily sync to sync any updates written to your source by an external system once a day. Using immediate and daily syncs, you get the performance benefits of data sync while continuously working with fresh data.
In addition to faster queries and reports, data sync also unlocks additional functionality that can speed up your development time. Learn more about these sync-enabled features below so you can build your applications faster:
Appian leverages the proximity to your cached data to provide additional features that are more powerful and flexible than we could automatically provide for external data. These features make it easier to change your data model, access the data you need, and even configure parts of your application automatically.
For example, additional data modeling capabilities like record type relationships and custom record fields provide a low-code alternative to manually building database views. Instead of creating database views to relate and transform data, you can use relationships to relate data from a variety of sources, and create custom record fields to perform data transformations—all without complex SQL.
The following features are only available when you sync your data in Appian.
Establish relationships between record types to easily reference related record data throughout your applications. Relationships can be established between record types with data sync enabled, which allows you to relate data from different source types, like Salesforce and a database table—going beyond traditional relationships you may establish in a database view.
Using relationships, you can easily build advanced records and reports that display data from a variety of sources. Simply reference the relationship in your interface or query to return related record data. Learn more about referencing related record data.
Custom record fields allow you to extend your data by transforming and manipulating existing record data into new custom record fields. Using predefined templates or expression mode, you can perform operations that calculate new values, group your data, manipulate text, and more, then display those values as new fields in the record type.
Not only does data sync make it faster and easier to configure your data model, it makes configuring the entire record type faster. When you enable data sync, Appian can automatically configure parts of your record type for you:
Auto-generated user filters: A user filter is automatically generated on the record type whenever you define a many-to-one relationship. The user filter will leverage related data to filter your list of records, and you can modify the user filter at any time.
Generate a Summary view interface: Generate a Summary view interface with all your record data. Once you generate the interface, the interface expression will automatically populate in the Summary view.
Generate record actions: Allow Appian to configure common record actions for you. Choose whether your record action should create, update, or delete records, then Appian will configure the record action using existing objects or creating new design objects for you.
Suggested relationships: Any time you add a relationship to a record type, the inverse relationship will be suggested on the related record type. For example, if the Customer record type has a one-to-many relationship with the Order record type, a Suggested Relationship will appear on the Order record type to add a many-to-one relationship with the Customer record type. With a click of a button, you can configure that relationship.
There are some data structures and business scenarios that are better fit for data sync than others. Consider the following requirements before enabling data sync in your record type:
This requirement only applies to customers that are on Advanced or Enterprise Support.
When you enable data sync, a record type can sync up to 1,000,000 rows of data from your selected source. Data sources that exceed or are expected to exceed the row limit may not be a good fit for data sync if you need access to all rows. However, if you only need a portion of a data source that meets or exceeds the row limit, you can configure source filters to limit the number of rows synced in Appian.
Source filters allow you to determine which rows of data from the source are synced in Appian based on the conditions you specify. When a source filter is applied, only the data that meets the specified criteria will be available in the Appian system and visible to end users. You can add source filters when defining the source of the record type, or anytime after by navigating to the Data Model page.
If your source is within 10% of the limit (900,000 to 1,000,000 rows), the sync will successfully complete but a warning will appear in the Status column on the Record Sync Status page to let you know the source is approaching or has reached the row limit. After the data source exceeds 1,000,000 rows, the record type will become unavailable. If the record type reaches the row limit after an immediate sync, the write operation will be successful, but the sync will fail and the record type will become unavailable.
If you expect the source data to exceed the row limit, consider applying multiple source filters. If your record type continues to exceed the row limit, data sync may not be the best option for your business scenario. See Troubleshooting Syncs for steps to disable data sync.
Any field values with more than 4,000 characters will be truncated in Appian. If your source contains fields with more than 4,000 characters, the sync will still complete successfully and you can use the truncated data throughout your applications. However, it's not recommend to use fields with more than 4,000 characters since it will result in a mismatch between the synced data and the source data. To ignore fields that exceed the character limit, deselect them from the field list when configuring the source of the synced record type.
When your record type uses a Salesforce object as the source, syncing 1,000 rows of data requires a single API call. If your record type has the maximum number of rows (1,000,000), the API limit on your Salesforce instance must be at least 1,000 per day. If you have multiple record types that use a Salesforce object with sync enabled, ensure your API limit is large enough to support all your record types.
For example, say you have five record types that total to 400,000 rows, and each record type has a schedule daily sync. To successfully sync each record type, you would need at least 120,000 API calls per month. If the API limit is not high enough, the sync will fail and the record type will be unavailable.
You can enable data sync when you configure the source of a record type to be a database, Salesforce, or other web service.
Appian recommends creating new record types with data sync enabled. If you enable data sync on an existing record type, new record fields with new UUIDs will be created. As a result, any existing references to your record fields will break, and you'll need to update them with the new record field references.
Instead, if you create a new record type with sync enabled, you can ensure that any existing record field references continue to work. Then, you can incrementally update references in your applications to point to your new record type with sync enabled. This way, you can continuously make changes to your record type without impacting your applications.
To enable data sync on a new record type, see:
In some cases, you may decide that converting your existing record type to use sync is more efficient than creating a new record type. For example, if your existing record type isn't widely used, you won't need to update many references in your application if you convert your record type to use data sync.
To enable data sync on an existing record type:
Once you change the source of your existing record type to use data sync, you will need to update all references to the original record fields throughout your applications. This includes references within the record type as well, such as record field references in the record list.
To find and update record field references, it's best to start by reviewing the record type's dependent objects. To review these objects, select the record type and click DEPENDENTS. Replace any existing record field references in these objects with the new record field references.
If you want to find specific field references in your environment:
/* and */.In the OBJECTS View, paste the string into the search box under DESIGN OBJECTS.
You can search for multiple strings at once by separating each string with a space.
Every object that references the string will appear in your search results. Once you have located all the references in your objects, update the field references. For updating field references in interfaces, we recommend updating the references in Expression Mode.
If you disable data sync on a record type, you will also need to review the objects in your applications and update any record field references associated with the record type. Any sync-enabled features, like relationships and custom record fields, will also be removed and made unavailable. For steps, see Troubleshooting Sync.
There are two ways to keep your synced data up-to-date with the source:
You can also trigger manual syncs at any time. However, manual syncs should only be used in development environments.
To view detailed information about all syncs that occurred for a specific record type, see the Sync History.
An immediate sync will occur anytime Appian writes to the database table or Salesforce object that backs your record type. This means that whenever data is inserted, updated, or deleted from the source using a smart service or Salesforce integration, the data in the associated record type is also automatically synced to reflect the changes.
An immediate sync will only refresh the data that was changed by Appian. If more than 1,000 rows of data are changed by Appian, the smart service or integration will successfully write to the data source, but the immediate sync will fail. To prevent an immediate sync from failing, it's recommended to apply a limit to the associated smart service or integration so it does not update more than 1,000 rows at a time.
If Appian is the only system writing to your database table or to Salesforce, immediate sync allows you to be confident that you are always working with fresh data.
Other than a Salesforce object, Appian does not support immediate syncs for record types that use other web services as the data source.
In a process model, you can configure Data Service smart services to write updates to a database table that’s connected to a data store entity. Once the database table is updated, Appian will trigger a sync to update the data in any synced records that use the database table as the source.
Without any additional configuration, you can use the following smart services to update the source of your synced data and trigger an immediate sync:
To use Data Service smart services, you will need to create a custom data type (CDT). Regardless of how your table data is structured in the CDT, immediate sync will occur when any table data that backs the record type is updated. The CDT is only necessary to write to the source.
To configure immediate syncs for your synced record:
Appian will not sync updates to synced records when the source updates are made using other smart services, plugins, or external systems. For example, the following services and tools will not trigger an immediate sync:
Data sync is particularly helpful when using an external system, like Salesforce, as the source of your record type. To keep your synced data up-to-date with the source data in Salesforce, Appian triggers an immediate sync whenever you use a Salesforce integration to write to the Salesforce connected system.
You can update your source data and trigger an immediate sync by calling the Salesforce integration in a process model, a related action configured on the record type, or a save on an interface.
If the Salesforce integration fails to write to the source, the immediate sync will not occur. Likewise, if Appian does not receive a notification that the write to the source was successful, the sync will not occur.
After you enable data sync, it's recommended that you create a scheduled sync so that your data outside of Appian will automatically sync with your record type once a day.
For faster syncs, Appian recommends scheduling full syncs outside of peak traffic or working hours. Additionally, you should avoid scheduling multiple record types to sync at the same time. Instead, schedule a full sync on each record type at varying intervals, like 15 or 30 minutes apart. You can use the Sync History to determine how long each record type takes to sync and use that time to determine the correct intervals in your production environment.
To schedule a sync:
You can edit your sync schedule here or on the Data Model page by opening the page's properties menu and selecting Change Sync Schedule.
Manual syncs allow you to sync your external data with the record type whenever you need it. However, manual syncs should only be used in development environments to test your application data.
To trigger a manual sync, use one of the following methods:
Whenever your record type is deployed to a new environment, a sync will automatically occur. If you deploy additional packages that don't include the record type, but impact the synced data, you'll need to trigger a manual sync.
To trigger a sync when the additional packages are deployed, add recordType.<UUID of a Record Type>.forceSync=true to an import customization file. You can reference this property multiple times to trigger a sync on different record types.
To sync your source data in Appian, the source schema and the record type schema must match. The source schema includes the names, types, and uniqueness constraints of all the fields in your source. When you enable data sync for your record type and configure your record fields, Appian automatically creates a mapping between the source schema and the record type schema.
If a change is made to the source schema without making a corresponding change to the record type, the record type will fail to sync. For example, if you try to refresh your synced data using an immediate sync, the smart service will write to the data source, but the sync will not occur.
When a record type fails to sync, it becomes unavailable until you resolve the schema mismatch. In addition, any calls to a!queryRecordType() for the synced record will fail.
For information on resolving a schema mismatch, see Troubleshooting Sync.
To ensure that your data is syncing successfully, you can check the history and status of all syncs associated with your record type.
In the Monitoring View of Appian Designer, the Record Sync Status tab displays the statuses of all record types with sync enabled that you have access to view. The grid displays all record types, source types, sync statuses, and the time of the last sync. The tab also has a refresh button and a conditionally visible START SYNC button.
From this page you are not only able to check the sync status, but you can also access any errors or warnings that may arise during the course of your syncs. Learn more about the Record Sync Status tab.
You can view detailed information about all syncs that occurred for a specific record type. From either the Monitoring View of Appian Designer, or from the Data Sync tab of your selected record type, you can view a record type’s Sync History.
The record type’s sync history will display information on the past and currently running syncs, including the times of the syncs and any errors that may have occurred.
Error messages will provide information on the cause of the error or possible solutions. When monitoring a record type that uses a web service as the source, you can even view errors from the web service request to troubleshoot issues with the web service. For more information on problems with your sync and fixing sync errors, see Monitoring View and Troubleshooting Syncs.
When checking the time of your scheduled sync, you may notice that it didn't occur at the exact time at which you scheduled it. There are a variety of possible causes:
Limit for syncs running simultaneously: If you have multiple record types scheduled to sync at the same time, not all of them can occur at exactly the scheduled time. Appian limits the number of simultaneous syncs running to 2 per application server. For example, if multiple record types are set to sync at 3:00 AM, one of the scheduled syncs may not complete until 3:02 or 3:10 AM. If you have other processes that have significant resource use scheduled to occur at this time, this may also cause your syncs to complete later than expected.
Failed & re-trying: It is possible that something caused your sync to fail. In some cases, the system will try again twice before ultimately failing or successfully completing the sync. The system will only retry the sync in situations where it's likely to succeed. For example, if your sync failed because of a connection issue, Appian will try to sync again; however, if the sync failed because of a schema mismatch, the sync will not retry. If a completed sync had to retry multiple times, it may be a few minutes behind schedule.
Check your timezone: Make sure that you selected the right timezone when you set up your scheduled sync. If not, you can change this by editing the sync schedule settings in Data Sync.
Daylight savings time: If your timezone or region observes daylight savings and your sync is scheduled for one of the hours in which daylight savings time changes, your sync may not run when the time changes. To avoid this, choose a time after 2 AM.
