Log Streaming to Amazon S3

This page describes how to stream supported Appian Cloud logs, in real time, to an Amazon S3 bucket that you manage. The content applies to Appian Cloud administrators who want to archive and analyze Appian logs using AWS-native services such as Amazon Athena and Amazon OpenSearch, or third-party Security Information and Event Management (SIEM) tools.

How log streaming to Amazon S3 works

Appian Cloud environments can be configured to stream supported logs, in real time, to an Amazon S3 bucket that you manage. Once logs are stored in S3, you can archive, query, and analyze them using AWS-native services such as Amazon Athena, Amazon OpenSearch, or third-party Security Information and Event Management (SIEM) tools of your choice.

Log streaming to Amazon S3 operates on a push-based model, in which Appian Cloud environments are configured to deliver a stream of log objects directly to your designated S3 bucket. Logs are forwarded in near-real time as messages are written in the Appian Cloud environment. You can configure the connection through the Admin Console using either an Identity and Access Management (IAM) role or IAM user's access key ID and secret access key in your AWS account.

Benefits of log streaming to Amazon S3

With this service enabled on your Appian Cloud environment, you can use your existing AWS infrastructure to centralize, store, and analyze Appian log data. Some benefits of this service include:

• Native AWS integration: Store Appian logs directly in S3 without additional middleware, making them immediately accessible to any AWS-native service in your environment.
• Cost-effective long-term archival: Take advantage of S3 storage tiers and lifecycle policies to retain logs at the duration and cost profile that matches your compliance and operational requirements.
• Flexible analytics: Query logs in place using Amazon Athena, ingest them into Amazon OpenSearch, or forward them to any SIEM or analytics platform that supports S3 as a source.
• Improved security visibility: Continuously monitor sign-in activity, authorization events, and system changes in your Appian Cloud environment. Correlate Appian log data with events from other AWS workloads in a unified platform.
• Self-service configuration: Configure and update your S3 destination directly from the Admin Console. You retain control of the IAM role, IAM user, and policies in your AWS account.
• Compatible with BYOK: When Bring Your Own Key (BYOK) is configured for your site, you can reuse the site's BYOK key to encrypt logs streamed to Amazon S3.

Set up your AWS account and S3 bucket

Log data is transmitted to S3 over HTTPS (TLS 1.2 or higher). This integration does not use AWS PrivateLink or a VPN connection. The destination bucket must be reachable from Appian Cloud. Configure either an Identity and Access Management (IAM) role or IAM user in your AWS account and grant only the permissions required to list the destination bucket and write log objects. If you create an IAM user, you are responsible for credential rotation. Appian recommends following AWS IAM security best practices.

Authentication

You can configure authentication for log streaming using either of the following approaches:

• IAM role authentication
• IAM user authentication

IAM role authentication

To authenticate log streaming by role, create a dedicated IAM role in the same AWS account as the destination bucket. The role must meet these requirements:

• Create the role without a custom IAM path.
• Name the role with the case-sensitive prefix AppianLogStreaming-. For example, AppianLogStreaming-ChocolateFactoryLogs.
• Configure the role's trust policy to allow the Appian log streaming collector role for your environment to assume it.
• Attach a permissions policy which grants s3:ListBucket on the destination bucket and s3:PutObject on all objects in the bucket.

Your IAM role or user, destination S3 bucket, and AWS Key Management Service (KMS) key, if applicable, must be located in the same AWS partition as your Appian Cloud environment. For example, Appian Cloud environments are located in the aws partition, while Appian Government Cloud environments are located in the aws-us-gov partition. Use the corresponding partition identifier in every Amazon Resource Name (ARN) in the trust policy and role permissions policy that follow.

Trust policy

Apply the following trust policy to your role. Replace <partition> and <appian-organization-path> with the values that correspond to your Appian Cloud environment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowAppianLogStreamingCollector",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "ArnLike": {
          "aws:PrincipalArn": "arn:<partition>:iam::*:role/AppianLogStreamingCollector*"
        },
        "ForAnyValue:StringLike": {
          "aws:PrincipalOrgPaths": [
            "<appian-organization-path>"
          ]
        }
      }
    }
  ]
}

The aws:PrincipalArn condition restricts access to roles named AppianLogStreamingCollector*. The aws:PrincipalOrgPaths condition further restricts access to roles in accounts attached to Appian's designated organizational unit or one of its child organizational units. Both conditions must match for the Appian AppianLogStreamingCollector* role to assume your role AppianLogStreaming-ChocolateFactoryLogs.

Role permissions policy

Attach the following permissions policy to your role. Replace <partition> and <bucket-name> with your values.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListDestinationBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:<partition>:s3:::<bucket-name>"
    },
    {
      "Sid": "WriteDestinationBucketObjects",
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:<partition>:s3:::<bucket-name>/*"
    }
  ]
}

This policy grants Appian permission to write any object in the dedicated destination bucket, including objects written under an optional key prefix configured in the Admin Console.

IAM user authentication

To authenticate log streaming by user, create a dedicated IAM user and attach the same S3 permissions shown in the role permissions policy. Generate an access key for the user and enter it in the Admin Console when you enable log streaming. You are responsible for securely storing and managing these long-term credentials.

Bucket policy

If the IAM identity and S3 bucket are in the same AWS account, an identity policy is typically sufficient. A bucket policy used only to enforce controls (such as requiring TLS) does not also need to grant access to the IAM role or user. Note that an explicit Deny always overrides an Allow.

If the IAM identity and S3 bucket are in different AWS accounts, a bucket policy is required. The following example is a minimal bucket policy that grants access to your role. Replace all placeholders with your values, set <arn-of-your-iam-identity> to the full ARN of your IAM role or user, and add any other statements required by your organization.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowAppianRoleToListBucket",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<arn-of-your-iam-identity>"
      },
      "Action": "s3:ListBucket",
      "Resource": "arn:<partition>:s3:::<bucket-name>"
    },
    {
      "Sid": "AllowAppianRoleToWriteObjects",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<arn-of-your-iam-identity>"
      },
      "Action": "s3:PutObject",
      "Resource": "arn:<partition>:s3:::<bucket-name>/*"
    }
  ]
}

Do not grant s3:GetObject, s3:DeleteObject, S3 access control list (ACL), or bucket-management permissions to the IAM role or user.

Server-side encryption

You must configure encryption for the destination S3 bucket, then select that encryption in the Admin Console when you enable log streaming.

IAM permissions for the KMS key

Add the following statement to the permissions policy for the IAM role or user. Use the ARN of the specific customer-managed KMS key or site's BYOK key, not a key alias.

1
2
3
4
5
6
7
8
9
10
{
  "Sid": "AllowUseOfLogStreamingKmsKey",
  "Effect": "Allow",
  "Action": [
    "kms:GenerateDataKey"
  ],
  "Resource": [
    "arn:<partition>:kms:<aws-region>:<aws-account-id-where-your-kms-key-lives>:key/<kms-key-id>"
  ]
}

KMS key policy

The KMS key policy must also authorize the IAM role or user. Add the following statement to the existing key policy and set <arn-of-your-iam-identity> to the full ARN of your IAM role or user. Do not replace the existing key policy.

1
2
3
4
5
6
7
8
9
10
11
12
13
{
  "Sid": "AllowAppianLogStreamingRoleToUseKey",
  "Effect": "Allow",
  "Principal": {
    "AWS": "<arn-of-your-iam-identity>"
  },
  "Action": [
    "kms:GenerateDataKey"
  ],
  "Resource": [
    "*"
  ]
}

In a KMS key policy "Resource": "*" refers to the KMS key to which the policy is attached. It does not grant access to other KMS keys.

S3 object structure

Logs are delivered to the S3 bucket as compressed files using GZIP compression. Each compressed file contains JSON objects, with each object separated by a newline character. The JSON object includes the following fields:

For Tomcat logs, a message might look like:

1
2026-03-17 18:36:58,631 [Appian Timer - 1] WARN  com.appiancorp.common.sample.class - 1 services were missing users.

For CSV logs, a message might look like (note: a header line may not be present if the file was rotated):

1
2026-03-04 17:40:02,selftest-mRSIP2,Succeeded,127.0.0.1,Portal,Apache-HttpClient/4.5.13(Java/17.0.17),a4b166f0-219d-437c-b5f7-b7ae442ffc0c,d9ba5b6e-c1d9-4661-8d85-47e2b4c45690

Example JSON structure

1
2
{"file": "/usr/local/appian/ae/logs/tomcat-stdOut.log", "severity": "warning", "timestamp": "2026-03-17 18:36:58,631+00:00", "hostname": "mysite-0", "tag": "tomcat-logs", "message": "2026-03-17 18:36:58,631 [Appian Timer - 1] WARN  com.appiancorp.common.sample.class - 1 services were missing users."}
{"file": "/usr/local/appian/ae/logs/authz-audit.csv", "severity": "unknown", "timestamp": "2026-03-04T17:40:02+00:00", "hostname": "mysite-1", "tag": "login-audit", "message": "2026-03-04 17:40:02,selftest-mRSIP2,Succeeded,127.0.0.1,Portal,Apache-HttpClient/4.5.13(Java/17.0.17),a4b166f0-b5f7-b7ae,d9ba5b6e-8d85-47e2"}

Supported logs

The following logs can be forwarded to your S3 bucket. Select a tab to view the logs for that category.

Each row describes a single log file:

• Log: The name of the log file (without extension), linked to its detailed description.
• Format: The file extension (.csv or .log).
• Tag: The unique identifier used to label log objects in S3.

To add or remove logs from streaming, see Enable log streaming to Amazon S3.

Prerequisite checklist

Enable log streaming to Amazon S3

Before you begin, you must complete every item in the Prerequisite checklist section. Once all prerequisites are complete, follow these steps to enable log streaming to Amazon S3 in your Appian Cloud environment.

Step 1: Create a new connection in the Admin Console

1. In your Appian Cloud environment, access the Admin Console.

2. In the left navigation bar, click Log Streaming. The Log Streaming page displays.

   

   Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

3. Click New Connection. The Create Connection dialog displays.

4. In the Connection Type field, select AWS S3.

   Note:  You cannot change the connection type after you create the connection.

   

   Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Step 2: Enter connection details on the dialog

1. In the S3 Connection Details section of the Create Connection dialog, complete the following fields:

   Field	Description
   Name	The name to identify this connection.
   S3 Bucket Name	The name of the destination S3 bucket in your AWS account.
   AWS Region	The AWS region where the destination bucket resides. For example, us-east-1.
   Key Prefix	An optional prefix to prepend to all log object keys written by Appian. Use a trailing slash (appian/<folder>/) to place the logs inside a folder, or omit the trailing slash (appian/<file-name-prefix>) to store the log files prefixed with the given text.
   S3 Authentication	Select IAM Role (uses your AWS role, credentials auto-managed) to use temporary credentials or IAM User (uses static long-term credentials) to use an access key ID and secret access key.
   IAM Role ARN	For IAM Role, enter the ARN of your role whose name matches the AppianLogStreaming-* pattern.
   Access Key ID	For IAM User, enter the AWS access key ID for the IAM user you created. This value must be 20 characters, alphanumeric, and generally starts with the AKIA prefix. For example, AKIAIOSFODNN7EXAMPLE.
   Secret Access Key	For IAM User, enter the AWS secret access key for the IAM user. This value is stored securely and is not displayed again after you save.
   S3 Encryption Method	Select Default encryption (SSE-S3), Self Managed Key (SSE-KMS), or Reuse the Site's BYOK (SSE-KMS) based on the configuration of server-side encryption in your environment. The BYOK option is available only when BYOK is configured for your Appian Cloud environment.
   KMS Key ARN	For Self Managed Key (SSE-KMS), enter the full ARN of your KMS key. Use the key ARN, not a key alias. For example, arn:aws:kms:us-east-1:123456789012:key/abc-123.

Step 3: Test the connection in the dialog

1. In the Create Connection dialog, click Test Connection and confirm it is successful. The configuration cannot be saved unless the connection is valid. If validation fails, verify the following:
   • The bucket name and region are correct.
   • The IAM role name and ARN or IAM user credentials are correct.
   • For IAM role authentication, the trust policy uses the correct partition and Appian organization path and allows AppianLogStreamingCollector* roles.
   • The identity and bucket policies allow s3:ListBucket on the bucket and s3:PutObject on all objects in the bucket.
   • No identity policy, bucket policy, AWS Organizations service control policy, VPC endpoint policy, or permissions boundary explicitly denies access.
   • If using SSE-KMS, the identity and KMS key policies allow use of the selected key.

Step 4: Select logs and save the connection

1. In the Log Selection section, select the minimum log severity level for filtering log output. This setting applies only to log files that contain severity level metadata. Log entries that do not include a severity level, as well as all CSV log files, are streamed in their entirety regardless of the selected severity level.
2. Select the logs you want to stream to the S3 destination. See Supported logs for a list of available logs.
3. Click CREATE to save your configuration.
4. Confirm log delivery by inspecting your S3 bucket for new objects under the designated key prefix.

Regional failover considerations

Because self-service log streaming allows you to independently manage your logging destinations directly through the Admin Console, customer-managed log receivers (such as Amazon S3 buckets) are frequently deployed in the same primary AWS region as the Appian environment. For instance, if an environment hosted in us-east-1 streams logs to an S3 bucket in us-east-1, a regional outage impacting the primary region will also render the destination bucket inaccessible.

Even if you have Enhanced Business Continuity and your site is re-created in your backup region, log streaming will not resume automatically if the destination bucket resides in the failed primary region. The bucket is a region-specific resource that you manage, so it remains inaccessible until you provide a destination in an available region.

To ensure logs continue to stream during a regional failover, execute one of the following remediation strategies:

• Proactive multi-region setup (recommended): Provision a cross-region replicated or secondary S3 bucket in the secondary backup region in advance. Ensure that the appropriate IAM role or user policies and KMS key permissions (if using SSE-KMS encryption) are pre-configured to grant write access. We recommend adding a new streaming connection to this bucket and leaving it disabled after ensuring that Test Connection passes.
• Reactive reconfiguration: In the event of an outage, create a new S3 bucket in an unaffected AWS region, then update the target bucket, AWS region, and authentication/encryption settings in the Admin Console under the self-service log streaming configurations.

Feedback