Search Server

On This Page

Appian runs Elasticsearch to provide search and data retrieval capabilities. In the product and documentation, this is referred to as the "search server."

Configuring the search server

The search server can be configured as a single instance or in a cluster for data redundancy and high availability.

Search Server Instances	Data Redundancy	Automatic Failover	Example CR entry
1	No	No	`searchServer: replicas: 1`
2	Yes	No	`searchServer: replicas: 2 port: 9400 httpPort: 9500 podDisruptionBudget: minAvailable: 1`
3	Yes	Yes	`searchServer: replicas: 3 port: 9300 httpPort: 9200 podDisruptionBudget: minAvailable: 2`

The above examples from the Appian CR configuration file also demonstrate using the port field to change the port used by each search server node from the default of 9300 to a different value. The port can be set to be different for each search server instance, as in the second example, or set to be the same for each server in the cluster as shown in the third example.

The port specified in the port field (default 9300) will be used by the search server for communication with search servers on the other nodes in your highly available Appian installation. For communication with the application server, the search server will use the port specified by an optional httpPort field. When not specified, it defaults to the value of 9200. Thus, customers should have the 9200 port open for the search server. If you want to use different ports instead of the defaults, you can specify the port and httpPort fields for the search server in the Appian CR configuration file. For example, if 9400 and 9500 ports are used for search server communication, the search server configuration will look like this:

spec:
  searchServer:
    replicas: 3
    port: 9400
    httpPort: 9500
    podDisruptionBudget:
      minAvailable: 2

In Kubernetes deployments, the search server pods use internal cluster networking and service discovery, so hostname resolution is handled automatically by the Kubernetes cluster.

Index data

Location

The index data for the search server is located in <APPIAN_HOME>/search-server/data/.

Backup and restoration

You cannot perform a live backup of search server data by simply copying the data directory <APPIAN_HOME>/search-server/data/. Instead, you must use the Elasticsearch snapshot and restore APIs. You can automate snapshots by configuring an Elasticsearch snapshot lifecycle management policy.

Sizing

The search server's use of disk and memory resources will scale with the amount of design objects, user activity, doc extraction mappings, and rule executions. The search server stores at most six entries per rule per minute, per application server. Although limited, rule execution metrics are the largest factor so we will use them for the sizing estimation. Additionally, the rule metrics are only stored for 30 days so we will calculate the maximum disk usage based on the number of minutes in 30 days using the following equation:

Max Disk Space =
Number of Unique Rules in the System
x Number of Application Servers
x 43,200 (the number of minutes in 30 days)
x 6 (the maximum number of rule metric entries per minute)
x 1 Kb (the approximate size of a rule metric entry)

For example, if your system has a sustained rate of ten unique rule executions every ten seconds for 30 days, you would expect to use ~2.6GB of disk for a single application server and ~7.8GB of disk for a three application server system.

To determine the number of rules in the system, review the "Rules and Constants" column in the content.csv file in the <APPIAN_HOME>/logs/data-metrics directory. To determine the number of unique rule executions that occur on your site in a given time period, review the expressions_details.csv file in the <APPIAN_HOME>/logs/perflogs directory.

Search server kernel setting

In order for Elasticsearch to run, it requires the kernel's vm.max_map_count setting to be at least 262144. If this is set to anything less, Elasticserach will error out with a message like:

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

The side effect of setting this parameter is that it could potentially increase the memory consumption of all applications running on the host. Specifically, only processes which actually use a large amount of memory maps are affected. This parameter allows those processes to use significantly more maps, which increases memory consumption.

Directly configuring the host

The first option is to use the following command on the host (using sudo, or while logged in as root):

sysctl -w vm.max_map_count=262144

However, this option won't persist the setting on the host, and will be reset to original value upon a host restart. To make the setting persistent, you can add vm.max_map_count=262144 to the /etc/sysctl.conf file. After adding this configuration, running the sysctl -p command or a subsequent restart of the host will reload the parameters in the /etc/sysctl.conf file.

The benefit of using this command is that it is very simple, and is transparent to the users when configuring Appian. Of course to do this change on the host, one would need to have admin access to the node. It is also important to note that this setting needs to be set on all nodes in the cluster, since the Search Server pod is not tied to any specific node and could move around. For this reason, it is best if it is encoded in the host image.

Using an init container

Another option is to use an init container for the search server to make sure the value is set on the kernel before the search server pod is started. Since changing this setting requires admin privileges, the init container needs to run in the privileged mode. This command needs to run as root. Therefore, either the container image that is used should run as root, or runAsUser: 0 should be included in the pod definition.

To do this, the definition of the search-server in the CR needs to be modified to include the init container as follows:

spec:
  searchServer:
    initContainers:
    - name: set-vm-max-map-count
      image: busybox
      imagePullPolicy: IfNotPresent
      securityContext:
        privileged: true
        runAsUser: 0
      command: ['sysctl', '-w', 'vm.max_map_count=262144']

Note that setting the sysctl parameter in the init container still sets the parameter at the host level, and affects all applications running on the host.

Starting and stopping

See Editing, Stopping, and Restarting Appian.

Monitoring and recovery

Logs for the search server are located in the <APPIAN_HOME>/logs/search-server/ directory. Log levels can be controlled by editing <APPIAN_HOME>/search-server/conf/log4j.properties.

Cluster health information is printed every five minutes to the <APPIAN_HOME>/logs/data-metrics/search_server_cluster.csv file. Description of the columns in this file can be found in the search server cluster metrics Log document.

The table below describes the meaning of the various cluster status levels and recovery procedures, if applicable.

Status	Meaning	Recovery
GREEN	All configured search server nodes are part of the cluster and operational	N/A
YELLOW	At least one search server node is down, but a majority are still available. The cluster remains operational, accepting both reads and writes.	Recover the down node(s) by scaling the search server deployment or replacing failed pods. In Kubernetes, this is typically handled automatically, but you can manually intervene by: Checking the search server pod status and logs to identify the issue. If needed, updating the Appian CR configuration file to adjust the search server configuration and apply the changes to the cluster. Kubernetes will automatically handle pod replacement and service discovery for the new search server instances.
RED	Less than a majority of search server nodes are available. The cluster is only partially operational, accepting only reads. Writes are rejected.	Recover the down node(s) by following the steps in the previous row. If this occurs with a two-replica cluster, an option to return the cluster to a functional status is to scale down to a single replica. This will return the cluster to GREEN status and allow writes again, but eliminates data redundancy. Update the Appian CR configuration file to set `searchServer.replicas: 1` Apply the updated CR configuration to the cluster, which will scale down the search server deployment to a single pod.

Other error scenarios

Should all of the nodes in the search server cluster fail during operation, both read and write calls will be rejected. Here are the features that will be affected:

Current User Activity - unavailable
- If there is a limit on the number of concurrent sessions enforced per account, no users will be able to log in
Precedents and Dependents views in Appian Designer - unavailable
- A red error box will appear on attempting to view either
- Dependents on Custom Data Types in /designer will still be available, but it will be slow
Missing Precedents view - unavailable
- A red error box will appear on attempting to view if there are any missing precedents
Historical interface performance trends - unavailable
Appian document extraction auto-mapping - unavailable
- The a!docExtractionResult() function will return an empty output
- Reconcile Doc Extraction Smart Service - the reconciliation task interface will load without any auto-mapped values. A red error box will appear on task submission.

If the search server pods are unreachable, the application server will log the following error message on startup:

The search server cannot be reached. Failed to connect to server at [host:port]. Check that the search server is started. If running multiple application servers, check that the Appian CR configuration file is properly configured with the search server details. (APNX-1-4274-001)

Feedback

Was this page helpful?