Configuring Relational Databases

Overview

This article provides guidance on how to configure relational databases in Appian. Configuring relational databases brings your information together in a meaningful way and allows you to manage your data more efficiently.

Configuring Data Sources

There are three steps unique to creating a data source in Appian:

  1. Install the driver
  2. Configure a security domain
  3. Enter the data source property values

Make the changes to the corresponding files in the configuration repository (<REPO_HOME>) instead of in <APPIAN_HOME> as documented here. Then deploy the files using the configure script.

Install RDBMS Driver

Install one or more of the supported RDBMS drivers. Other databases and drivers can be configured, but are not supported.

To install the driver, place the driver .jar file in the <REPO_HOME>/tomcat/apache-tomcat/lib/

Note on the DB2 Driver

The db2 driver includes a reference to an external jar that is usually not available. This can result in an error when starting Appian. To resolve this issue, follow the steps in this db2 help article.

Provide Data Source Connection Information

In <REPO_HOME>/tomcat/tomcatResources.xml.<ENVIRONMENT>, configure the connection information for your data sources.

We recommend preserving the transaction isolation setting of READ_COMMITTED, (as shown in the examples) to avoid deadlocks and performance issues that may occur with multiple concurrent reads and writes to the same table if you set the transaction isolation to TRANSACTION_SERIALIZABLE.

Note: If you are upgrading from a previous release of Appian be sure that datasource declarations are ported over correctly. If they are not it could disrupt running processes that make use of query rules or the Query Database smart service. In particular be aware of the following case:

Encoding Data Source Passwords

To avoid storing data source passwords in plain text, you must run the Tools -> Encode passwords for use in data source configuration utility in the configure script to encode your passwords. Although an encryption library is used to transform the password, because there is no secret private key, this encryption should be considered merely an encoding.

MySQL

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Resource name="jdbc/MySqlDataSource"
	factory="com.appiancorp.tomcat.datasource.EncodedPasswordDataSourceFactory"
	type="javax.sql.DataSource"
	driverClassName="com.mysql.jdbc.Driver"
	url="jdbc:mysql://such_as_server.example.com:3306/such_as_mySQL_for_appian?useOldAliasMetadataBehavior=true&amp;useServerPrepStmts=true"
	username="your_plaintext_username"
	password="your_encoded_password"
	initialSize="5"
	maxTotal="200"
	defaultTransactionIsolation="READ_COMMITTED"
	maxWaitMillis="30000"
	minIdle="5"
	minEvictableIdleTimeMillis="90000"
	timeBetweenEvictionRunsMillis="450000"
	numTestsPerEvictionRun="200"
	accessToUnderlyingConnectionAllowed="true"
	validationQuery="SELECT 1"
	testOnBorrow="true"
/>

Oracle

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Resource name="jdbc/OracleDataSource"
	factory="com.appiancorp.tomcat.datasource.EncodedPasswordDataSourceFactory"
	type="javax.sql.DataSource"
	driverClassName="oracle.jdbc.driver.OracleDriver"
	url="jdbc:oracle:thin:@oracle_server.example.com:1521:ORACLE_FOR_APPIAN"
	username="your_plaintext_username"
	password="your_encoded_password"
	initialSize="5"
	maxTotal="200"
	defaultTransactionIsolation="READ_COMMITTED"
	maxWaitMillis="30000"
	minIdle="5"
	minEvictableIdleTimeMillis="90000"
	timeBetweenEvictionRunsMillis="450000"
	numTestsPerEvictionRun="200"
	accessToUnderlyingConnectionAllowed="true"
	validationQuery="select 1 from dual"
	testOnBorrow="true"
/>

The Oracle hostname, port, and service name are listed in <ORACLE_HOME>/.../server/NETWORK/ADMIN/tnsnames.ora.

SQL Server

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Resource name="jdbc/SqlServerDataSource"
	factory="com.appiancorp.tomcat.datasource.EncodedPasswordDataSourceFactory"
	type="javax.sql.DataSource"
	driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
	url="jdbc:sqlserver://such_as_sqlServer.example.com:1433;databaseName=such_as_MS-SQL_for_appian"
	username="your_plaintext_username"
	password="your_encoded_password"
	initialSize="5"
	maxTotal="200"
	defaultTransactionIsolation="READ_COMMITTED"
	maxWaitMillis="30000"
	minIdle="5"
	minEvictableIdleTimeMillis="90000"
	timeBetweenEvictionRunsMillis="450000"
	numTestsPerEvictionRun="200"
	accessToUnderlyingConnectionAllowed="true"
	validationQuery="SELECT 1"
	testOnBorrow="true"
/>

DB2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Resource name="jdbc/Db2DataSource"
	factory="com.appiancorp.tomcat.datasource.EncodedPasswordDataSourceFactory"
	type="javax.sql.DataSource"
	driverClassName="com.ibm.db2.jcc.DB2Driver"
	url="jdbc:db2://your_db2_domain.example.com:50000/such_as_db2_for_appian"
	username="your_plaintext_username"
	password="your_encoded_password"
	initialSize="5"
	maxTotal="200"
	defaultTransactionIsolation="READ_COMMITTED"
	maxWaitMillis="30000"
	minIdle="5"
	minEvictableIdleTimeMillis="90000"
	timeBetweenEvictionRunsMillis="450000"
	numTestsPerEvictionRun="200"
	accessToUnderlyingConnectionAllowed="true"
	validationQuery="select 1 from sysibm.sysdummy1"
	testOnBorrow="true"
/>

List Data Sources in custom.properties

After creating your data sources, you need to register the Appian and business data sources with Appian.

Appian Data Source

This required data source will be used exclusively for storing data relating to the Appian installation as a whole. The schema is controlled by Appian and does not include any tables designers create when building or importing Appian applications. There is only one Appian data source for each installation of Appian.

To register the Appian data source, list the JNDI name of the Appian data source in custom.properties under the conf.data.APPIAN_DATA_SOURCE property. Any supported RDBMS can be used.

  • For example, conf.data.APPIAN_DATA_SOURCE=jdbc/AppianPrimaryDS

The Appian data source must have a different JNDI name and use a different tablespace or database than the data source(s) used to store business data.

NOTE: Customized interactions with the Appian data source are not supported.

Business Data Sources

Business data source are used to store data relating to applications designers build within Appian. The schema is controlled by designers (or the administrators of pre-existing databases with which Appian integrates), not by Appian itself. At least one business data source is required to use data stores.

There are two ways to register a business data source with Appian: using the Appian Administration Console and listing it in custom.properties.

The Appian Administration Console approach is the recommended way to create or update data source configurations and is the only method supported for Appian Cloud, but data sources that were previously created and registered in custom.properties continue to work. To move a data source configuration from custom.properties to the Administration Console, create a data source in the Administration Console and give it the same name as a data source in custom.properties. If thename field in the Administration console matches a value in conf.data.datasources, the data source configured in the Administration Console will override the one registered in custom.properties.

To register business data sources with Appian using custom.properties, list the JNDI names of business data sources in custom.properties under the conf.data.datasources property. Multiple business data sources can be configured in the same Appian installation by listing multiple JNDI names using a comma separated list. For example: conf.data.datasources=<JNDI-NAME-1>,<JNDI-NAME-2>,<JNDI-NAME-3>

NOTE: DO NOT include the Appian data source in the list of business data sources. This configuration is not supported and will be explicitly prevented in a future release.

Configuring Restrictions on Query Results

Case-Sensitivity in Query Rule Text Comparisons

When creating a query rule that filters text results based on a text comparison (for example, a query rule that only returns results starting with the prefix USA), the case-sensitivity or insensitivity for text comparisons is determined by your RDBMS settings, not a setting configured in Appian.

To ensure your text comparison is case-insensitive, you need to change the default settings specific to your RDBMS.

  • MySQL: The collation determines text comparison and sort order.
    • As mentioned in the Internationalization section below, setting the default collate to utf8mb4_unicode_ci will make it case-insensitive.
  • Oracle: The system variable NLS_COMP determines case-sensitivity.
    • You can set the RDBMS to be case-insensitive system-wide by modifying the system variable to the following: alter system set NLS_COMP = linguistic;
  • SQL Server: The collation determines case-sensitivity and is case-insensitive by default. No changes are needed.
  • DB2: Does not support case-insensitive comparisons.
    • Consider generating an extra column with text converted to uppercase and converting your query argument to uppercase as well.

NOTE: External websites with information specific to your RDBMS can be found here.

Row Return Limits for the Query Database Smart Service

The data that is returned by the Query Database Smart Service only includes a maximum of ten rows of data (1,000 rows for Appian Cloud users).

To configure Appian to read additional rows of data from a database query, complete the following:

  1. Edit (or create) <APPIAN_HOME>/conf/custom.properties.
  2. Adjust the number of rows of data returned by changing the resources.appian.process.application.arv.limit=10 property to arv.limit=<desired-number>.

Query Rule Return Size Limits

The amount of memory it takes to return query results may affect performance, so the following recommended limits have been set:

  • Queries that result in more than 1MB of application server memory usage, by default, are canceled.
  • Queries that take longer than 10 seconds are also, by default, canceled.

These settings can be configured in the custom.properties file.

See also: Query Rule Limits

Node Input and Output Limits

Appian node inputs and node outputs that handle multiple values from a form are restricted to a limit of up to 1250 indices.

Note to Appian Cloud Users:

  • The default query result limit for Appian Cloud users is 1,000 rows.

Internationalization

In order to store an extended range of characters (including languages that use multiple-byte character sets and emojis), follow the instructions below for your database. These configurations must be used whenever multi-byte characters appear in a custom data type definition.

  • MySQL: When creating the databases, set the server's default character set to utf8mb4 and the collation to utf8mb4_unicode_ci, using the following command:
    • CREATE DATABASE example_db_name default character set = "utf8mb4" default collate = "utf8mb4_unicode_ci";
  • Oracle: When creating the database, select Use Unicode AL32UTF8.
  • SQL Server: No additional configuration required.
  • DB2: When creating the databases, set the codeset and territory to UTF-8 and us. Note that the pagesize is also defined during database creation.
    • create database example_db_name using codeset UTF-8 territory us pagesize 32768

The database collation settings determine the sort order of characters, which may be different for different languages. Choose the setting that is most appropriate for the primary language of the site. Although the site can be configured to support users of multiple languages, a single collation must be selected, which affects sorting for all languages. For most languages, the above settings are sufficient.

A site with Swedish as the primary language, however, may choose to set the collation settings in order to have characters that are unique to Swedish sort in the correct order. To do so, complete the following:

  • MySQL: set collate to utf8mb4_swedish_ci instead of utf8mb4_unicode_ci when running the command mentioned above.
  • Oracle: run the following command:
    • create or replace trigger after_logon after logon on schema begin execute immediate 'alter session set nls_language = ''SWEDISH'''; end;
  • SQL Server: run the following command with <DATABASE_NAME> replaced with your database name:
    • create database <DATABASE_NAME> collate Finnish_Swedish_100_CS_AS
  • DB2: run the following command with <DATABASE_NAME> replaced with your database name:
    • CREATE DATABASE <DATABASE_NAME> COLLATE USING UCA500R1_LSV PAGESIZE 32768

Time Zones

The database can be set to any time zone as long as the datetime values persisted from Appian are always returned as the same value when retrieved by Appian. Appian stores and retrieves datetime values as if they are set in the GMT time zone.

If you rely on database functions that are specific to the time zone, such as the NOW or CURTIME functions, it is recommended that you set your database to the GMT time zone to be consistent with Appian's handling of datetime time zones. Alternatively, you must be aware of your databases configured local time zone and shift the value returned by these functions to the equivalent value in GMT.

Note to Appian Cloud Users:

  • The MySQL databases provided with Appian Cloud are set to the GMT time zone.

Optimizing IBM DB2

In order to achieve maximum performance when using DB2 with Appian, the REOPT bind option must be configured to REOPT(ONCE).

See also: IBM DB2 bind option documentation.

Also note that a pagesize of 32768 or greater is required and must be set during the creation of the database.

To apply the REOPT(ONCE) configuration, complete the following for each database used for Appian data source and business data sources:

  1. On the DB2 server, navigate to <DB2_HOME>/bin.
  2. Execute the DB2 command line processor: ./db2
  3. Connect to the database used to store Appian data: connect to <database-name>
    • If multiple databases are used for Appian data source and business data sources, these steps must be followed for each.
  4. Run the bind command: bind ../bnd/db2clipk.bnd collection NULLIDR1
  5. Grant the user execute privileges: GRANT EXECUTE ON PACKAGE NULLIDR1.SYSSH200 TO <database-user>
    • Note: The grant may fail because the user does not have the authority to grant privileges to itself. Should this occur, you can run the above grant with another user that has admin privileges to the database.
  6. To grant another user admin privileges: GRANT DBADM ON DATABASE TO USER <some-other-database-user>

When using a database named appiandata and a user named appianuser, the output looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
(c) Copyright IBM Corporation 1993,2007
Command Line Processor for DB2 Client 9.7.4

db2 => connect to appiandata

   Database Connection Information

 Database server        = DB2/NT64 9.7.4
 SQL authorization ID   = APPIANUSER
 Local database alias   = APPIANDATA

db2 => bind ../bnd/db2clipk.bnd collection NULLIDR1

LINE    MESSAGES FOR db2clipk.bnd
------  --------------------------------------------------------------------
      SQL0061W  The binder is in progress.
      SQL0091N  Binding was ended with "0" errors and "0" warnings.

db2 => GRANT EXECUTE ON PACKAGE NULLIDR1.SYSSH200 TO appianuser
DB20000I  The SQL command completed successfully.

db2 => quit

Once configured, the individual data sources are able to specify a the REOPT(ONCE) option by using the connection parameter jdbcCollection&#61;NULLIDR1. The example data source configuration for DB2 above includes this option.

Additionally, it is recommended that the RUNSTATS command be used to periodically update the optimizer information about the database. Consult the IBM DB2 RUNSTATS documentation for details on its usage.

The following is an example of using the command with the built-in TP_FEED_ENTRY table in a schema called APPIANDATA:

1
2
3
4
5
    RUNSTATS ON TABLE "APPIANDATA"."TP_FEED_ENTRY"
    ON COLUMNS ( "CREATED_TS" , "AUTHOR"  )
    WITH DISTRIBUTION ON COLUMNS ( "GROUP_UUID" NUM_FREQVALUES 15 NUM_QUANTILES 25  )
    AND SAMPLED DETAILED INDEXES ALL
      ALLOW WRITE ACCESS;

Troubleshooting

The following basic steps can be performed to help troubleshoot connectivity issues.

  • Verify that the database is up and running properly using a database utility or monitoring tool.

  • Check that your database driver is stored in the proper location on your application server.

  • Check your custom.properties configuration settings.
    • Verify that you have the proper Appian data source configuration for your metadata repository.
    • Verify that you have the proper listing for your business data repository.
  • From the application view, attempt to create a new CDT from a database table. Verify that the data source shows up as an option and that it shows the appropriate schemas and tables.

Database Configuration Error Messages

The Appian web application will fail to deploy during application server start-up if the Appian data source is not configured in custom.properties or is unreachable due to an error. Possible error messages related to this include the following:

Error Code Error Message
APNX-1-4179-002 FATAL: The Appian data source cannot be reached. Access to the Appian data source is required. The Appian application will fail to deploy.
APNX-1-4179-003 FATAL: No value configured for conf.data.APPIAN_DATA_SOURCE in custom.properties. The Appian data source is required. The Appian application will fail to deploy.

The following runtime Appian error messages might also be attributable to an issue with your database configuration.

Error Code Error Message
APNX-1-4164-022 Failed to create data store (rule: [{0}], store: [{1}])
APNX-1-4164-026 JDBC connection error trying to execute query (type: [{0}], query: [{1}])

Oracle Errors

The following error may indicate a problem with your application server's data source configuration.

1
TNS:listener does not currently know of SID given in connect descriptor

MySQL Errors

The following error may indicate that the maximum number of simultaneous client connections to the MySQL database should be increased. Encountering this error is more likely in environments with multiple application servers and heavy load.

1
Too many connections

Based on internal performance testing, Appian recommends setting the following MySQL system variables to at least the values indicated here:

See Also

FEEDBACK