The data server is a storage layer designed for the Appian platform. It currently tracks the design objects added to the application patches for export and stores user-saved filters. In future versions of Appian, the data server will be used as a data persistence layer for an increasing number of functionalities and capabilities. It provides better performance, more reliability, and increased security for the application data.
Following are the major components of the data server:
The historical store is a single kdb+ database optimized for data writes. The historical store (also referred to as hs) and the historical store gateway (also referred to as hs-gw) form the writer in the data server. Every write made to the data server goes to the historical store.
The real-time store consists of kdb+ databases and serves all query requests. It consists of "snapshot tables" which are read-optimized tables that serve queries (reads) in conjunction with in-memory, real-time tables. The real-time store (also referred to as rts) and the real-time store gateway (also referred to as rts-gw) form the reader in the data server. For on-premises installation, only a single real-time store is currently supported for the data server.
Watchdog is responsible for starting all of the data server components and monitoring the health of each component. The watchdog also heals (restarts) a component in case of an isolated failure.
A valid license (k4.lic) is required to run the data server. Please refer Requesting and Installing a License page to obtain and install the license.
The configuration for the data server has to be specified in data-server-cluster element in the appian-topology.xml file. For example, for a single instance of data server:
1
2
3
4
5
6
<topology>
...
<data-server-cluster>
<data-server host="machine1.example.com" port="5400" rts-count="1"/>
</data-server-cluster>
</topology>
For a high availability configuration, specify three instances of data server on different machines:
1
2
3
4
5
6
7
8
<topology>
...
<data-server-cluster>
<data-server host="machine1.example.com" port="5400" rts-count="1"/>
<data-server host="machine2.example.com" port="5400" rts-count="1"/>
<data-server host="machine3.example.com" port="5400" rts-count="1"/>
</data-server-cluster>
</topology>
If data-server-cluster configuration is not specified in appian-topology.xml, it will run on the first node of Kafka listed in the kafkaCluster configuration. The appian-topology.xml file must be copied from <APPIAN_HOME>/conf/ into <APPIAN_HOME>/data-server/conf/ before starting the data server. The two copies of the topology file in both the locations must always be in sync, irrespective of the configurations specified.
This attribute in data-server element represents the name of the machine hosting the data server. If host is not provided, the default is localhost. However, this behavior has been deprecated and will be removed in the future. In future versions of Appian, host needs to be explicitly specified for the data server. If the host is not specified, the following warning is printed in watchdog.log:
WARN com.appian.data.TopologyFactory - DataServer host is empty, defaulting to 'localhost'. This behavior is deprecated and will be removed in the future, and this topology will be considered invalid. You should not leave the host blank.
Port element can be defined on both, the data-server-cluster and the data-server elements. If defined on both, the data-server element takes precedence. If not supplied, the default port number is 5400.
The rts-count parameter determines the number of real-time stores in the data server. Multiple real-time stores will provide horizontal read scalability in future. Currently, the rts-count should only be set to 1 for on-premises installation.
Changes to data-server-cluster configuration in appian-topology.xml are needed for reasons like adding or removing data server nodes, changing the host name, or changing the port. In such scenarios, take the following steps:
<APPIAN_HOME>/data-server/bin/stop.sh on Linux or <APPIAN_HOME>\data-server\bin\stop.bat on Windows<APPIAN_HOME>/ae/data-server/node/election directory from all the servers. This directory contains runtime data that needs to be deleted when certain topology changes are made.<APPIAN_HOME>/data-server/bin/start.sh on Linux or <APPIAN_HOME>\data-server\bin\start.bat on WindowsRequests to the data server are secured with a security token unique to every customer environment. For cloud customers, this token is generated during the cloud deployment. For on-premise customers, this token is generated by the configure script. The data server will not start if the token has not been set properly, and consequently the application server will not start. See the Data Server Connection Restrictions page for more information.
The data files are located in <APPIAN_HOME>/data-server/data/. The hs folder contains the historical store data and the ss folder contains the real-time store (snapshot) data written on the disk. The <APPIAN_HOME>/data-server/data/ directory should be backed up as a part of your regular backup procedures.
See also:
The Java Virtual Machine (JVM) and Go processes executed by the data server utilize approximately 200MB of RAM without any application data. On top of that, there is a negligible usage of RAM for application data by the data server. The background processes run every hour (or less) and flush application data from memory to disk. Application patches and user-saved filters are the only functional objects stored in the data server in the current version.
The data server may take up to 40MB of disk space by default after it is started for the first time in your environment. In addition to that, disk space will be occupied based on the number of application patches created on the site and the number of user-saved filters. We do not expect these objects to utilize high amount of disk.
To start or stop the data server, please refer Starting and Stopping Appian page.
Note that when logging out of Windows, the data server process started by the user using the script will stop. Instead, the data server can be installed as a Windows service and started and stopped using the Windows service management console. For instructions on controlling the data server as a Windows service see Installing data server as a Windows Service.
To check the status or health of the data server, execute <APPIAN_HOME>/data-server/bin/health.sh (health.bat on Windows).
The following information is contained in the output after executing the health script:
For the data server cluster,
For each node in the data server cluster,
Watchdog continuously monitors each component of the data server and ensures that system is up again in case of an isolated failure.
Logs generated by the data server reside in <APPIAN_HOME>/logs/data-server/ folder. Logs are generated for each process in the data server.
Other Error Scenarios
If the data server is unreachable when the application starts up or if the application is started before the data server is running, the application server will not start.
If the data server cannot start and the watchdog.log indicates an issue with the security token, refer to the Data Server Connection Restrictions page for troubleshooting.
If the data server stops running while the application server is running, it will not be possible to create, update or export application patches. Similarly, user-saved filters will not be accessible.
