The data server is a storage layer designed for the Appian platform. It tracks the design objects added to the application patches for export. In future versions of Appian, the data server will be used as a data persistence layer for an increasing number of functionalities and capabilities. It provides better performance, more reliability, and increased security for the application data.
Following are the major components of the data server:
The historical store is a single kdb+ database optimized for data writes. The historical store (also referred to as
hs) and the historical store gateway (also referred to as
hs-gw) form the writer in the data server. Every write made to the data server goes to the historical store.
The real-time store consists of kdb+ databases and serves all query requests. In the current version of Appian, there can only be one real-time store. It consists of "snapshot tables" which are read-optimized tables that serve queries (reads) in conjunction with in-memory, real-time tables. The real-time store (also referred to as
rts) and the real-time store gateway (also referred to as
rts-gw) form the reader in the data server.
Watchdog is responsible for starting all of the data server components and monitoring the health of each component. The watchdog also heals (restarts) a component in case of an isolated failure.
The background engine (
bg) is responsible for producing the optimized snapshot views that are used by the real-time store.
A valid license (
k4.lic) is required to run the data server. Please refer Requesting and Installing a License page to obtain and install the license.
Currently, the data server can only be configured to run on a single node (machine). The configuration has to be specified in
data-server-cluster element in the
appian-topology.xml file. For example,
1 2 3 4 5 6 <topology> ... <data-server-cluster> <data-server host="machine1.domain.tld" port="5400" rts-count="1"/> </data-server-cluster> </topology>
This attribute in
data-server element represents the name of the machine hosting the data server. If host is not provided, the default is localhost.
Port element can be defined on both, the
data-server-cluster and the
data-server elements. If defined on both, the
data-server element takes precedence. If not supplied, the default port number is 5400.
rts-count parameter determines the number of real-time stores in the data server. Multiple real-time stores provide horizontal read scalability. The
rts-count can only be set to 1 in the current version.
data-server-cluster configuration is not specified in
appian-topology.xml, it will run on the first node of Kafka listed in the
kafkaCluster configuration. The
appian-topology.xml file must be copied from
<APPIAN_HOME>/data-server/conf/ before starting the data server. The two copies of the topology file in both the locations must always be in sync, irrespective of the configurations specified.
The data files are located in
hs folder contains the historical store data and the
ss folder contains the real-time store (snapshot) data written on the disk. The
<APPIAN_HOME>/data-server/data/ directory should be backed up as a part of your regular backup procedures.
The Java Virtual Machine (JVM) and Go processes executed by the data server utilize approximately 200MB of RAM without any application data. On top of that, there is a negligible usage of RAM for application data by the data server. The background engine runs every hour and writes application data from memory to disk. Application patches and patch contents are the only functional objects stored in the data server in the current version.
The data server may take up to 40MB of disk space by default after the application is started. In addition to that, disk space will be occupied based on the number of application patches created on the site. Following table gives an estimation of the amount of disk space utilized by patches on the data server:
O = number of objects per patch; P = number of patches created in the environment; S = size on disk per object
|O x P x S||Disk space used by patches (approximate)|
|25 x 1,000 x ~1KB||25MB|
|25 x 100,000 x ~1KB||2.5GB|
Patches continue to reside on disk for audit purposes even after they are cleared.
To start or stop the data server, please refer Starting and Stopping Appian page.
Note that when logging out of Windows, the data server process started by the user using the script will stop. Instead, the data server can be installed as a Windows service and started and stopped using the Windows service management console. For instructions on controlling the data server as a Windows service see Installing data server as a Windows Service.
Logs generated by the data server reside in
<APPIAN_HOME>/logs/data-server/ folder. Following logs can be found in this directory:
data-server.log- The log containing the standard output from all components running on the server
hs-gw-<port>.log- The historical store log
rts-gw-<port>.log- The real-time store log
wd-<port>.log- The watchdog log
data-server-client.log- The log containing error messages at the data client level
To check the status or health of the data server, Execute
status.bat on Windows). The meaning of various machine status levels are as follows:
hs-gw) and the reader(
rts-gw-0) are both up and running.
hs-gw) is down or the reader(
rts-gw-0) is down.
Watchdog continuously monitors each component and ensures that system is up again in case of an isolated failure.
Other Error Scenarios
If the data server is unreachable when the application starts up or if the application is started before the data server is running, the application server will come up successfully and the following error message will be logged in the application-server log:
Failed to connect to data server. The ability to store and retrieve application patches will not be available until the connection is reestablished. Check the status of the data server.
Creating Application patches is the only functionality that will be affected if the data server is not available. The user will not be able to add objects to a patch if the data server is down. Following error message is displayed on the UI when a user attempts to do so:
Your patch cannot be reached at this time. Check that the data server is running. As a workaround, add your objects to an application and export the full application package.