This topic outlines the backup process for Appian documents and data.
There are four primary data components that should be backed up:
Each of these components is backed up separately, yet all backups must occur simultaneously. When scheduling a script to backup the Appian engines, external databases must be backed up at the same time. This ensures that during recovery the system's data snapshot is synchronized and consistent. Otherwise, you might encounter synchronization issues.
Appian engines are in-memory databases containing application information, including process information, and metadata for groups, users, pages, and documents. The checkpointing process for these databases writes a snapshot of the most recent data in memory to an image file on disk (the application automatically performs this operation). These image files are numbered .kdb files and must be backed up regularly. The Appian .kdb file locations are listed in the following table, with * representing the number of the file. Additional pairs of execution engines (03 - 31) and analytics engines (0003 - 0031) may be enabled in your instance.
|Appian Engine Name||File Name||Location|
|process-execution ## (03 - 31)||pe*.kdb|
|process-analytics 00## (0003 - 0031)||pa*.kdb|
Application data comprises files stored on the server that are referenced by Appian engines, including documents stored in the collaboration center, archived processes, search indices, the keystore file, the patch contents in the data server, and web content channel (portlet) files. The folders containing this application data should be backed up at the same time as the .kdb files are backed up. The locations of application data components are listed in the following table.
When running in a system with multiple application servers, some of these directories are shared between servers. In those cases, the data only needs to be backed up once. For those directories that are not shared between servers, the data needs to be backed up from each of the servers. See High Availability and Distributed Installations for a list of directories that are shared when configuring a multiple server setup.
|Component Name||Folder Name||Default Folder Location|
|Search (shared content search index)||search|
|Search (Search Server indices)||data|
|Data Server (Historical Store)||hs|
|Data Server (Snapshot)||ss|
|Web Content Channel||mini|
Note: After performing an upgrade or restoration, it is particularly important to ensure your keystore and Appian data source data are present and in the correct state. If there is a problem with one of the two, the system will not start up and will log an ERROR message to the application server log.
Strictly speaking, backing up the web application is not required as part of the scheduled data backup when the application is live. These files never change unless there is an upgrade. Before performing an upgrade (such as a patch or service pack) we recommend backing up the web application components.
The Kafka logs contain the historical data required to replay transactions to the data server. These logs must be backed up to ensure that all of the data in the data server is backed up. ZooKeeper is the configuration management service that runs Kafka, and its directory must also be backed up in order to run Kafka. The Kafka logs can be found in the
<APPIAN_HOME>/services/data/kafka-logs directory and the ZooKeeper logs can be found in the
All external data sources used with Appian must be backed up separately and simultaneously. Use your preferred backup mechanism for the specific RDBMS used in the Appian environment.
This section outlines the steps to be followed to backup all data of an Appian installation at an application level. It is not meant to replace or eliminate the need to backup the server at a drive and operating system level. Application data backup procedures must be accompanied by some form of tape or similar backup for a production environment.
Before a product upgrade or patch, the following steps must be taken to backup the Appian application in preparation for the upgrade:
pscommand for a UNIX environment, or your preferred system monitoring tool
For normal maintenance of an Appian application, regularly scheduled backups of the system data are strongly recommended. The frequency of the backup varies depending on the usage of the system. Daily to weekly backup intervals are recommended, depending on the resource requirements of your data and your recovery scenarios.
To perform a scheduled backup of Appian, follow the instructions below. Please note that all such scheduled backups should be configured to run during off-peak hours.
The time it takes to restore from a backup will depend on how long after a checkpoint the backup was taken. To minimize the time it takes to restore, schedule the <APPIAN_HOME>/services/bin/checkpoint.sh (.bat) script to execute approximately 15 minutes after the data cleanup script. This script checkpoints the Appian databases to disk, creating a new .kdb file for each database, which contains the latest image. It can be configured as a scheduled task. As before, ensure that the account used to execute the task is the same service account that is running the Appian application.
For more information regarding application checkpointing, see: Configuring Application Checkpointing
Schedule any external data source snapshots or full backups to execute at the same time as the checkpoint script. This ensures data synchronization during recovery.
Schedule the data cleanup script located at <APPIAN_HOME>/services/bin/cleanupArchives.sh (.bat). This script can be configured as a scheduled task. We recommend the following parameters:
1 services/bin/cleanupArchives.sh (.bat) --keep 3
These parameters retain the three most-recent .kdb database images and delete all older ones. Ensure that the account used to execute the script is the same service account that is running the Appian application. For more information, see: CleanupArchives Script.
Schedule a data backup script to execute approximately 15 minutes after the checkpoint script. The data backup script can also be configured as a scheduled task. As before, ensure that the account used to execute the task is the same service account that is running the Appian application.
The data backup scripts referenced above are located in the following folder.
These scripts utilize Apache Ant as the building technology for the scripts. There is no single batch file included with the application since the parameters used for the backup can differ depending on the site. Some examples are provided in the /examples subfolder.
For a UNIX system, to backup the Appian .kdb data as well as Appian application data, use the script provided below. Save this code as a shell script (.sh) in the <APPIAN_HOME>/_admin/_scripts/tools/datamaintenance/ folder. Schedule it to run as a cron job.
1 2 3 4 #!/bin/bash source ../../../../server/_scripts/exports.sh INSTALLDIR=$AE_HOME/_admin/_scripts/tools/datamaintenance/antScripts ant -f $INSTALLDIR/aecopyfiles.xml -Dae.source.location=$AE_HOME -Dae.dest.location="/data/aebackups" -Dae.datadelete="n" -Dae.copycontents="y" -Dkdb.num="1" -Dkdb.invertselect="y" -Dbatchmode="y"
For a Windows-based system, to backup the Appian .kdb data as well as Appian application data, using the script provided below. Save this code as a batch file (.bat ) in the <APPIAN_HOME>/_admin/_scripts/tools/datamaintenance/ folder. Configure it to run as a Windows Scheduled Task.
1 2 3 4 5 echo off call ../../../../server/_scripts/exports.bat set INSTALLDIR=%AE_HOME%/_admin/_scripts/tools/datamaintenance/antScripts ant -f %INSTALLDIR%/aecopyfiles.xml -Dae.source.location=%AE_HOME% -Dae.dest.location="c:/ae_backups" -Dae.datadelete="n" -Dae.copycontents="y" -Dkdb.num="1" -Dkdb.invertselect="y" -Dbatchmode="y" echo on
The location for the data backup in the script above is set to
c:/ae_backups for the Windows version or
/data/aebackups for the UNIX version, by default. You can change this to any desired location.
This section outlines the steps to be followed to restore all data of an Appian installation at an application level.
Stop all Appian services. Backup the Appian application data into a temporary folder by copying the contents of the
<APPIAN_HOME> directory into a temporary backup location.
Delete the contents of the
<APPIAN_HOME>/services/data/ directory. Copy the
/services/data/kafka-logs and the
/services/data/zookeeper/ directories from the backup location into the empty
Delete any remaining .kdb files from the
<APPIAN_HOME>/server/<ENGINE_NAME>/ directories to avoid conflicts. Copy your KDB files from the backup location (the highest numbered .kdb file for each engine on production) into the corresponding folder in the Appian installation directory. Refer to the Appian Engine Data table earlier in this document for the relevant directories. Ensure that the new .kdb file is the highest numbered .kdb in the target folder. If it is not, rename it to be the highest numbered .kdb file.
Delete the contents of the Appian application data directories to avoid file naming/id conflicts. Refer to the Appian Application Data table earlier in this document for file locations. Copy the contents of the application data folders from the backup location into the corresponding folder in the Appian installation directory.
When restoring to a multiple application server environment, restore the files to the single shared directory for those backed up directories that are shared between servers. Restore the the files to the local directories for the those backed up directories that are not shared between servers.
Restore the latest backup or snapshot of any external data sources so that the data is synchronized.