The data server is a storage layer designed for the Appian platform. It tracks the design objects added to the application patches for export. In future versions of Appian, the data server will be used as a data persistence layer for an increasing number of functionalities and capabilities. It provides better performance, more reliability, and increased security for the application data.
Following are the major components of the data server:
The historical store is a single kdb+ database optimized for data writes. The historical store (also referred to as
hs) and the historical store gateway (also referred to as
hs-gw) form the writer in the data server. Every write made to the data server goes to the historical store.
The real-time store consists of kdb+ databases and serves all query requests. It consists of "snapshot tables" which are read-optimized tables that serve queries (reads) in conjunction with in-memory, real-time tables. The real-time store (also referred to as
rts) and the real-time store gateway (also referred to as
rts-gw) form the reader in the data server. For on-premises installation, only a single real-time store is currently supported for the data server.
Watchdog is responsible for starting all of the data server components and monitoring the health of each component. The watchdog also heals (restarts) a component in case of an isolated failure.
A valid license (
k4.lic) is required to run the data server. Please refer Requesting and Installing a License page to obtain and install the license.
The configuration for the data server has to be specified in
data-server-cluster element in the
appian-topology.xml file. For example, for a single instance of data server:
1 2 3 4 5 6 <topology> ... <data-server-cluster> <data-server host="machine1.example.com" port="5400" rts-count="1"/> </data-server-cluster> </topology>
For a high availability configuration, specify three instances of data server on different machines:
1 2 3 4 5 6 7 8 <topology> ... <data-server-cluster> <data-server host="machine1.example.com" port="5400" rts-count="1"/> <data-server host="machine2.example.com" port="5400" rts-count="1"/> <data-server host="machine3.example.com" port="5400" rts-count="1"/> </data-server-cluster> </topology>
data-server-cluster configuration is not specified in
appian-topology.xml, it will run on the first node of Kafka listed in the
kafkaCluster configuration. The
appian-topology.xml file must be copied from
<APPIAN_HOME>/data-server/conf/ before starting the data server. The two copies of the topology file in both the locations must always be in sync, irrespective of the configurations specified.
This attribute in
data-server element represents the name of the machine hosting the data server. If host is not provided, the default is localhost. However, this behavior has been deprecated and will be removed in the future. In future versions of Appian, host needs to be explicitly specified for the data server. If the host is not specified, the following warning is printed in
WARN com.appian.data.TopologyFactory - DataServer host is empty, defaulting to 'localhost'. This behavior is deprecated and will be removed in the future, and this topology will be considered invalid. You should not leave the host blank.
Port element can be defined on both, the
data-server-cluster and the
data-server elements. If defined on both, the
data-server element takes precedence. If not supplied, the default port number is 5400.
rts-count parameter determines the number of real-time stores in the data server. Multiple real-time stores will provide horizontal read scalability in future. Currently, the
rts-count should only be set to 1 for on-premises installation.
data-server-cluster configuration in
appian-topology.xml are needed for reasons like adding or removing data server nodes, changing the host name, or changing the port. In such scenarios, take the following steps:
<APPIAN_HOME>/data-server/bin/stop.shon Linux or
<APPIAN_HOME>/ae/data-server/node/electiondirectory from all the servers. This directory contains runtime data that needs to be deleted when certain topology changes are made.
<APPIAN_HOME>/data-server/bin/start.shon Linux or
The data files are located in
hs folder contains the historical store data and the
ss folder contains the real-time store (snapshot) data written on the disk. The
<APPIAN_HOME>/data-server/data/ directory should be backed up as a part of your regular backup procedures.
The Java Virtual Machine (JVM) and Go processes executed by the data server utilize approximately 200MB of RAM without any application data. On top of that, there is a negligible usage of RAM for application data by the data server. The background engine runs every hour and writes application data from memory to disk. Application patches and patch contents are the only functional objects stored in the data server in the current version.
The data server may take up to 40MB of disk space by default after the application is started. In addition to that, disk space will be occupied based on the number of application patches created on the site. Following table gives an estimation of the amount of disk space utilized by patches on the data server:
O = number of objects per patch; P = number of patches created in the environment; S = size on disk per object
|O x P x S||Disk space used by patches (approximate)|
|25 x 1,000 x ~1KB||25MB|
|25 x 100,000 x ~1KB||2.5GB|
Patches continue to reside on disk for audit purposes even after they are cleared.
To start or stop the data server, please refer Starting and Stopping Appian page.
Note that when logging out of Windows, the data server process started by the user using the script will stop. Instead, the data server can be installed as a Windows service and started and stopped using the Windows service management console. For instructions on controlling the data server as a Windows service see Installing data server as a Windows Service.
To check the status or health of the data server, execute
health.bat on Windows).
The following information is contained in the output after executing the health script:
For the data server cluster,
For each node in the data server cluster,
Watchdog continuously monitors each component of the data server and ensures that system is up again in case of an isolated failure.
Logs generated by the data server reside in
<APPIAN_HOME>/logs/data-server/ folder. Logs are generated for each process in the data server.
Other Error Scenarios
If the data server is unreachable when the application starts up or if the application is started before the data server is running, the application server will come up successfully and the following error message will be logged in the application-server log:
Failed to connect to data server. The ability to store and retrieve application patches will not be available until the connection is reestablished. Check the status of the data server.
Creating Application patches is the only functionality that will be affected if the data server is not available. The user will not be able to add objects to a patch if the data server is down.