Using the Appian on Kubernetes Migration Tool

This page outlines the steps and information needed for using the Appian on Kubernetes Migration Tool for self-managed customers.

Why use the Migration Tool?

The Migration Tool makes it easier to transition an existing machine-based Appian deployment to an Appian on Kubernetes deployment by packaging up your data and configurations.

Before starting

• Shut Appian down and make a full backup of your original Appian deployment and associated databases before proceeding.
• Download the Migration Tool binaries that are compatible with the operating systems of the machines used to execute the migration. This includes the machines that host your existing Appian site, as well as the machine that will interact with your destination Kubernetes cluster.
• Ensure that the Migration Tool binary is present on all servers that run your Appian site. If you run your Appian site in a highly available or distributed topology, the Migration Tool must be present on each server.
• Set up the Appian operator on the destination Kubernetes cluster, and keep the following in mind:
  • Set up registries with the Appian product images. The Appian images need to be the same version as their current deployment.
  • Load the licenses into the cluster.
  • Install the Appian operator Helm chart.
  • Don't create the Appian custom resource or go any further, as the Migration Tool will handle those tasks.
• Your site's database connection needs to be accessible by your destination Kubernetes cluster. The Migration Tool does not support migrating your database data.

The Migration Tool commands each have -h flags that explain supporting flags and which parameters are required or optional. You can see the full list of commands available by running migrate --help or migrate -h.

Want to get a feel for your Appian on Kubernetes deployment without having to migrate your production data yet? See Drafting your Appian Custom Resource.

Step 1: Exporting your Appian data

To begin your migration, you will run the Migration Tool on the servers of your machine-based Appian site. You will use the export command, pointing to your Appian installation directory with the -i flag and defining a directory for output with the -o flag.

The tool will package your Appian data into a ZIP, including shared data for HA or distributed Appian deployments, and prepare it to be deployed in Kubernetes.

As an example, if your Appian installation is located at C:\appian-prod\appian on a Windows server and you have created a new directory on that server C:\appian-migration for storing the exported data, your command would look like this:

1
.\migrate.exe export -i C:\appian-prod\appian -o C:\appian-migration

For Unix/Linux systems the command would look similar to this:

1
./migrate export -i /usr/local/appian -o /appian-migration

If you have a HA/distributed deployment, you will need to run the export command on each server. Follow through all the prompts to ensure that your site and its data are exported properly. You may also be able to follow KB-2356 to migrate your HA/distributed shared data in place, decreasing the amount of data the migration tool needs to process and reducing the overall migration time.

Excluding data

The export command supports excluding different types of data to enable manual migration and other advanced use cases. First and foremost, export supports excluding logs via the --no-logs flag. Other types of data can be excluded via the --exclude-data (-x) flag. The flag supports the following values and can be passed multiple times or with comma-separated values:

• zookeeper
• kafka
• search-server
• data-server
• engine-checkpoints
• content-documents
• archived-processes

Step 2: Preparing and merging your Appian data

Once all your site data has been exported, the merge command is needed to generate the Appian custom resource from your data and configurations. This command must be executed regardless of whether you have a single, HA, or distributed topology.

The merge command must have access to all of the export ZIPs to properly execute. If you have an HA/distributed site, you will need to colocate the export ZIPs from each server prior to running the merge command.

Here is an example of executing the merge command for a Unix/Linux system of a distributed site:

1
./migrate merge -z export.site-node-1.zip -z export.site-node-2.zip -z export.site-node-3.zip -o .

The merge action will package your data and configurations into a single ZIP (merge.zip) and a base Appian operator Custom Resource (appian.yaml) for your Kubernetes deployment. These files will both be used for the import action.

Step 3: Importing your Appian data into Kubernetes

The import command will need to be executed on an environment that has direct access to your Kubernetes cluster. If you need to move the appian.yaml and merge.zip to a different machine to perform the import make sure you have downloaded the appropriate Migration tool for that machine's OS.

import will validate your data, create PVCs (Kubernetes Persistent Volume Claims), and copy the zipped data from merge into those PVCs.

Before running the import command, you will need to update the appian.yaml custom resource file based on your cluster requirements. Below are fields which will need to be modified at a minimum:

• metadata.name
• spec.appianLicConfigMapName
• spec.zookeeper.image.repository
• spec.kafka.image.repository
• spec.searchServer.image.repository
• spec.dataServer.image.repository
• spec.serviceManager.image.repository
• spec.webapp.image.repository
• spec.webapp.dataSources.primary.passwordSecretKeyRef
• You'll need configure how to expose your Appian site in Kubernetes so the front end is accessible.

Any HA or multi-replica definitions of Webapp or Service Manager will also require the haExistingClaim to be defined for those services. See High Availability for this set up.

It is strongly recommended to update the Resource limits for the Appian services based on your sites's needs. Refer to Resource Requests and Limits for guidelines on resource updates based on your cluster and site's needs.

Here is an example of executing the import command for a Unix/Linux system of a distributed site:

1
./migrate import -z merge.zip -a appian.yaml -n <site namespace>

Once this process is completed, the tool will start up your new Appian deployment. If you have not setup how to expose Appian the front end will not be accessible. However you should check that all the Appian services were able to come up and verify that the number of pods match your site's defined replica count. At this point, you may want to visit the Tasks, Configuration, and Reference sections of the Appian on Kubernetes documentation for instructions on upkeep, upgrading, and additional configurations.

Manually migrating data

During import, the Migration Tool gives you the opportunity to manually migrate data that was excluded during export. If the import command detects that there was excluded data, it will ask whether the data has been manually migrated. If you answer "Yes", the import command will proceed to copy the data managed by the Migration Tool and start your site. Otherwise, it will ask whether you'd like it to start your site with sleep init containers or abort. Starting your site with sleep init containers enables you to copy excluded data to your site's underlying persistent volumes using kubectl cp - just like the import command normally would:

1
2
# Must be run from the directory containing the data to be copied
kubectl cp . <POD>:<DIRECTORY_IN_POD> -c sleep

The table below details where each type of excludable data is stored both on the server and in Kubernetes:

If you use the sleep init container option to migrate your excluded data, you will need to manually delete the Appian custom resource once you have finished your data transfer. You will also need to remove the finalizers on the Appian custom resource after calling kubectl delete in order to successfully delete it.

1
2
kubectl -n <appian-namespace> delete appian <appian-name> --wait=false
kubectl -n <appian-namespace> patch appian <appian-name> --type='json' -p='[{"op": "remove", "path": "/metadata/finalizers"}]'

Once all of the pods have terminated, you can rerun the import command to continue with the migration process.

Step 4: Update paths to Appian on Kubernetes based deployment

Once migration has been completed and you've verified your Appian site is up, ensure your internal paths point to the current Appian on Kubernetes based paths. To get started with understanding how to interact with your new AoK deployment, see our kubectl Cheat Sheet.

Find the Webapp pod

Changing paths for your deployment will be executed on the Webapp pod and propagated to the other pods. For Highly Available (HA) sites, there will be multiple Webapp pods, these steps can be executed on any one of them.

1. List all of the pods in your namespace, and locate the Webapp pod(s).

   1
   kubectl -n <NAMESPACE> get pods
   

2. If migrating to an Appian version < 24.4, copy your <APPIAN_HOME>/_admin/_scripts/classpath.jar from your machine-based deployment to the Appian on Kubernetes Webapp pod. If your Appian version is >= 24.4 you may skip this step, the classpath.jar will be located at /usr/local/appian/ae/_admin/_scripts/classpath.jar by default.

   1
   kubectl -n <NAMESPACE> cp <APPIAN_HOME>/_admin/_scripts/classpath.jar <WEBAPP_POD>:/tmp/classpath.jar
   

3. Exec into the Webapp pod.

   1
   kubectl exec -it <WEBAPP POD NAME> -- bash
   

Change your internal paths for your new deployment

You will be executing the ChangePaths tool java -cp classpath.jar com.appiancorp.tools.ChangePaths for each data directory that your installation of AoK utilizes. This tool supports the following options:

1. Go to /usr/local/appian/ae/_admin/_scripts/ or /tmp/ depending on where classpath.jar is located at on your Webapp pod.

2. Specify the data directory you wish to update the paths for, followed by the Absolute paths from your old directory path to your new Appian on Kubernetes based directory path, for example:

   1
   java -cp classpath.jar com.appiancorp.tools.ChangePaths ARCHIVED <OLD_APPIAN_HOME>/server/archived-process /usr/local/appian/ae/server/archived-process
   

Drafting your Appian Custom Resource

The migration tool supports generating a sample Appian custom resource yaml file from your current Linux or Windows installation, without having to complete the above steps of Exporting and Merging. This is useful for those who want to get a feel for Appian on Kubernetes, without having to worry about the data migration steps yet.

Use the draft-yaml command of the Migration Tool to draft your sample Appian Custom Resource. On a Linux server, if your Appian installation is located at /usr/local/appian and you created a new directory on that server /appian-migration for storing the generated yaml file, your command would look like this:

1
./migrate draft-yaml --install-dir /usr/local/appian --output-dir /appian-migration-draft

On Windows, that would look like:

1
.\migrate.exe draft-yaml --install-dir C:\appian-prod\appian --output-dir C:\appian-migration-draft

There will be a few questions to determine more details about your current Appian configuration. The draft-yaml command only needs to be run on a single server of your Appian installation. Any server can be used, assuming your configuration files are the same across servers. The draft-yaml command reads the following configuration files:

• <APPIAN_HOME>/conf/appian-topology.xml
• <APPIAN_HOME>/conf/build.info
• <APPIAN_HOME>/conf/custom.properties
• <APPIAN_HOME>/conf/tomcatResources.xml
• <APPIAN_HOME>/server/_conf/partitions.xml
• <APPIAN_HOME>/search-server/conf/custom.properties (optional)
• <APPIAN_HOME>/data-server/conf/custom.properties (optional)

Once the draft-yaml command completes, you should now see an appian.yaml file located in the /appian-migration-draft directory. This Appian custom resource configuration file can be further updated as needed, and then used for a sandbox Appian site.

Migrating custom properties

Keep the following in mind when migrating custom properties to the Appian CR configuration file.

Supported custom properties

Most properties from the legacy custom.properties file are supported as .spec.customProperties fields in the Appian CR configuration file.

For a list of supported properties, see Custom Properties.

Unsupported custom properties

In Kubernetes, various Appian pods run in their own containers where Appian is installed at /usr/local/appian/ae in those container images. In Appian on Kubernetes, the directories are fixed to the configured locations and cannot be changed.

If you are migrating to Appian on Kubernetes, the following custom properties from earlier self-managed environments are no longer supported:

• conf.suite.AE_LOGS
• conf.suite.BASE_PATH
• resources.appian.process.application.PM_STORE_ROOT_PATH

Legacy custom properties now managed as webapp properties

Certain custom properties were previously configured in the legacy custom.properties file, but are now internally managed and calculated by the Appian operator based on the .spec.webapp properties in the Appian CR configuration file.

The table below maps the legacy properties to the appropriate custom resource fields:

Note:

• These custom properties will show up under conf.* in the config map containing custom properties created by the operator.
• If any of these custom properties are configured in the Appian CR configuration file under .spec.customProperties, they will be removed by the Appian operator.

Configuring sleep init containers

The import and sleep commands start the Appian installation with all component pods initialized with sleep init containers. You can configure these sleep init containers' corev1.Container fields by defining them in the Appian CR YAML. The tool will merge its generated fields into your definitions according to the following rules:

Setting Resources

1
2
3
4
5
6
7
8
9
10
11
spec:
  zookeeper:
    initContainers:
      - name: sleep
        resources:
          requests:
            cpu: "1"
            memory: 4Gi
          limits:
            cpu: "2"
            memory: 8Gi

Setting Security Context

By default, the tool sets the following security context on the sleep init containers:

1
2
3
securityContext:
  runAsNonRoot: true
  runAsUser: <component's runAsUser, or 500 (the Appian container user)>

If you define a securityContext on the init container in the CR, it is preserved as-is and the default is not applied.

1
2
3
4
5
6
7
spec:
  zookeeper:
    initContainers:
      - name: sleep
        securityContext:
          runAsUser: 1000
          runAsNonRoot: true

Feedback