Configuring Backup and Restoration

This topic outlines the backup process for Appian documents and data.

There are primarily three data components to be backed up at an application level:

  • Appian engines
  • Appian application data (documents, archived processes, and other files)
  • Data Stores and external data (such as external RDBMS data)

Each of these components is backed up separately, yet all backups must occur simultaneously. When scheduling a script to backup the Appian engines, external databases must be backed up at the same time. This ensures that during recovery the system's data snapshot is synchronized and consistent. Otherwise, you might encounter synchronization issues.

Appian Engines

Appian engines are in-memory databases containing application information, including process information, and metadata for groups, users, pages, and documents. The checkpointing process for these databases writes a snapshot of the most recent data in memory to an image file on disk. (The application automatically performs this operation.) These image files are numbered .kdb files and must be backed up regularly. The Appian .kdb file locations are listed in the following table, with * representing the number of the file. Additional pairs of execution engines (03 - 31) and analytics engines (0003 - 0031) may be enabled in your instance.

Appian Engine Name File Name Location
channels channels*.kdb /server/channels/gw1
portal ap*.kdb /server/portal/gw1
collaboration (main) dc*.kdb /server/collaboration/gw1
collaboration (statistics) stat*.kdb /server/collaboration/gw1
forums af*.kdb /server/forums/gw1
personalization ag*.kdb /server/personalization/gw1
notification (email) s*.kdb /server/notifications/gw1
notification (main) n*.kdb /server/notifications/gw1
process-design pd*.kdb /server/process/design/gw1
process-analytics (0000) pa*.kdb /server/process/analytics/0000/gw1
process-execution (00) pe*.kdb /server/process/exec/00/gw1
process-execution (01) pe*.kdb /server/process/exec/01/gw1
process-analytics (0001) pa*.kdb /server/process/analytics/0001/gw1
process-execution (02) pe*.kdb /server/process/exec/02/gw1
process-analytics (0002) pa*.kdb /server/process/analytics/0002/gw1
process-execution ## (03 - 31) pe*.kdb /server/process/exec/##/gw1
process-analytics 00## (0003 - 0031) pa*.kdb /server/process/analytics/00##/gw1
engine transaction logs * /services/data/kafka-logs

Appian Application Data

Application data comprises files stored on the server that are referenced by Appian engines, including documents stored in the collaboration center, archived processes, search indices, the keystore file, the patch contents in the data server, and web content channel (portlet) files. The folders containing this application data should be backed up at the same time as the .kdb files are backed up. The locations of application data components are listed in the following table.

When running in a system with multiple application servers, some of these directories are shared between servers. In those cases, the data only needs to be backed up once. For those directories that are not shared between servers, the data needs to be backed up from each of the servers. See High Availability and Distributed Installations for a list of directories that are shared when configuring a multiple server setup.

Component Name Folder Name Default Folder Location
Collaboration accdocs1 /_admin/
Collaboration accdocs2 /_admin/
Collaboration accdocs3 /_admin/
Process archived-process /server/
Search (shared content search index) search /_admin/
Search (Search Server indices) search-local /_admin/
Data Server (Historical Store) hs /data-server/data/
Data Server (Snapshot) ss /data-server/data/
Forums msg /server/
Web Content Channel mini /_admin/
Process models /_admin/
Process process_notes /_admin/
Keystore shared /_admin/

Note: After performing an update or restoration, it is particularly important to ensure your keystore and Appian data source data are present and in the correct state. If there is a problem with one of the two, the system will not start up and will log an ERROR message to the application server log.

Strictly speaking, backing up the web application is not required as part of the scheduled data backup when the application is live. These files never change unless there is an upgrade. Before performing an upgrade (such as a patch or service pack) we recommend backing up the web application components.

Appian Data Source, Data Stores, and External Data

All external data sources used with Appian must be backed up separately and simultaneously. Use your preferred backup mechanism for the specific RDBMS used in the Appian environment.

Backup Procedures

This section outlines the steps to be followed to backup all data of an Appian installation at an application level. It is not meant to replace or eliminate the need to backup the server at a drive and operating system level. Application data backup procedures must be accompanied by some form of tape or similar backup for a production environment.

Pre-Upgrade Backup Procedures

Before a product upgrade or patch, the following steps must be taken to backup the Appian application in preparation for the upgrade:

  • Stop all Appian and application server services.
  • Ensure all Appian and application server processes are stopped using the Task Manager for a Windows environment, the ps command for a UNIX environment, or your preferred system monitoring tool.
  • Copy the entire Appian installation directory to a backup location.
  • If your application data is stored on a SAN or other high-availability drive, ensure that all file system content on this drive is copied to the backup location.
  • Take a snapshot or data dump of the Appian data source your data stores and any external RDBMS.
  • Proceed with the product upgrade.

Scheduled Backup Procedures

For normal maintenance of an Appian application, regularly scheduled backups of the system data are strongly recommended. The frequency of the backup varies depending on the usage of the system. Daily to weekly backup intervals are recommended, depending on the resource requirements of your data and your recovery scenarios.

To perform a scheduled backup of Appian, follow the instructions below. Please note that all such scheduled backups should be configured to run during off-peak hours.

The time it takes to restore from a backup will depend on how long after a checkpoint the backup was taken. To minimize the time it takes to restore, schedule the <APPIAN_HOME>/services/bin/checkpoint.sh (.bat) script to execute approximately 15 minutes after the data cleanup script. This script checkpoints the Appian databases to disk, creating a new .kdb file for each database, which contains the latest image. It can be configured as a scheduled task. As before, ensure that the account used to execute the task is the same service account that is running the Appian application.

For more information regarding application checkpointing, see: Configuring Application Checkpointing

Schedule any external data source snapshots or full backups to execute at the same time as the checkpoint script. This ensures data synchronization during recovery.

Schedule the data cleanup script located at <APPIAN_HOME>/services/bin/cleanupArchives.sh (.bat). This script can be configured as a scheduled task. We recommend the following parameters:

services/bin/cleanupArchives.sh (.bat) --keep 3

These parameters retain the three most-recent .kdb database images and delete all older ones. Ensure that the account used to execute the script is the same service account that is running the Appian application. For more information, see: CleanupArchives Script.

Schedule a data backup script to execute approximately 15 minutes after the checkpoint script. The data backup script can also be configured as a scheduled task. As before, ensure that the account used to execute the task is the same service account that is running the Appian application.

The data backup scripts referenced above are located in the following folder.

<APPIAN_HOME>/_admin/_scripts/tools/datamaintenance/

These scripts utilize Apache Ant as the building technology for the scripts. There is no single batch file included with the application since the parameters used for the backup can differ depending on the site. Some examples are provided in the /examples subfolder.

For a UNIX system, to backup the Appian .kdb data as well as Appian application data, use the script provided below. Save this code as a shell script (.sh) in the <APPIAN_HOME>/_admin/_scripts/tools/datamaintenance/ folder. Schedule it to run as a cron job.

#!/bin/bash
source ../../../../server/_scripts/exports.sh
INSTALLDIR=$AE_HOME/_admin/_scripts/tools/datamaintenance/antScripts
ant -f $INSTALLDIR/aecopyfiles.xml -Dae.source.location=$AE_HOME -Dae.dest.location="/data/aebackups" -Dae.datadelete="n"   -Dae.copycontents="y" -Dkdb.num="1" -Dkdb.invertselect="y" -Dbatchmode="y"

For a Windows-based system, to backup the Appian .kdb data as well as Appian application data, using the script provided below. Save this code as a batch file (.bat ) in the <APPIAN_HOME>/_admin/_scripts/tools/datamaintenance/ folder. Configure it to run as a Windows Scheduled Task.

echo off
call ../../../../server/_scripts/exports.bat
set INSTALLDIR=%AE_HOME%/_admin/_scripts/tools/datamaintenance/antScripts
ant -f %INSTALLDIR%/aecopyfiles.xml -Dae.source.location=%AE_HOME% -Dae.dest.location="c:/ae_backups" -Dae.datadelete="n" -Dae.copycontents="y" -Dkdb.num="1" -Dkdb.invertselect="y" -Dbatchmode="y"
echo on

The location for the data backup in the script above is set to c:/ae_backups for the Windows version or /data/aebackups for the UNIX version, by default. You can change this to any desired location.

Restore Procedures

This section outlines the steps to be followed to restore all data of an Appian installation at an application level. Before restoring, ensure that all Appian and application server services are stopped.

Before restoring the data

Copy/Backup all Appian Application data and Appian KDB data into a temporary folder, if you might need to restore it later.

Delete any existing kdb files out of the <APPIAN_HOME>/server/<ENGINE_NAME>/gw*/ directories to avoid conflicts.

Delete the contents of the services/data/ directory.

Delete the content of Appian application data to avoid file naming/id conflicts. Refer to the Application Data Table for file locations.

Restoring Appian Engine Data

Copy your KDB files from the backup location (the highest numbered .kdb file for each engine on production) into the corresponding folder in the Appian installation directory. Refer to the table in the first section of this document for KDB locations. Ensure that the new KDB is the highest numbered KDB in the target folder. If it is not, rename it to be the highest numbered KDB.

Start the Appian engines and execute the status script to ensure that the engines are operating properly.

Restoring Appian Application Data

Delete all application data folders from the Appian installation (see the application data table earlier in this document).

Copy the application data folders from the backup location into their respective locations on the production server.

When restoring to a multiple application server environment, restore the files to the single shared directory for those backed up directories that are shared between servers. Restore the the files to the local directories for the those backed up directories that are not shared between servers.

Restoration of External Data

Restore the latest backup or snapshot of any external data sources so that the data is synchronized.

Start all Appian services and your application server.

FEEDBACK