Monitoring and Troubleshooting Kafka Integrations

Overview

This page provides guidance on monitoring and troubleshooting Kafka connected systems and event consumers. It covers the Event Consumer Monitoring tab, the log files that Appian produces, and the email alerts Appian sends when an event consumer encounters an error.

The Event Consumer Monitoring tab is the primary destination for observing and troubleshooting event consumers. Use the tools on this tab to check each event consumer's runtime status, activate or deactivate a consumer, and inspect the individual events a consumer processed. Log files and email alerts complement the monitor by helping detect and identify problems using external incident response systems.

Monitor event consumer activity

The Event Consumer Monitoring tab gives you a runtime view of native Kafka event consumers without opening each design object or scanning CSV log files. From the Appian Monitor page, you can see each event consumer's status, activate or deactivate a consumer, and drill into the individual events a consumer processed to trace what happened to each one.

To open event consumer monitoring, go to the Monitor view and click EVENT CONSUMER MONITORING.

View event consumer status

The event consumer list on the Appian Monitor page shows every event consumer you can access, along with its runtime status and recent activity. The following table describes each column:

Column	Description
Consumer	The name of the event consumer.
Connected System	The Apache Kafka connected system the event consumer uses.
Topic	The Kafka topic the event consumer is listening to.
Failed Events	The number of original events that the event consumer consumed from the Kafka topic but failed to process in the monitoring window (last 7 days maximum).
Reprocessed Events	The number of unique failed original events that were reprocessed at least once. When an event is reprocessed, the count increases by 1, regardless of whether the reprocessing succeeded or failed. However, reprocessing the same event multiple times does not change the count.
Status	The event consumer's current status: Active, Inactive, or Consumer Error.

Filter the event consumer list

You can filter the event consumer list to find a specific consumer or group of consumers. Filter the list by the following:

• Connected System: Show only event consumers that use a specific Apache Kafka connected system.
• Status: Show only event consumers with a specific status, such as Active with Failed Events or Consumer Error.

Activate or deactivate an event consumer

You can start or stop an event consumer directly from the Event Consumer Monitoring tab without opening the design object in Appian Designer. Activating and deactivating from the Appian Monitor page is useful when you need to restart a consumer after external broker maintenance or stop a misbehaving consumer quickly.

To activate or deactivate an event consumer:

1. In the grid, click the event consumer you want to start or stop.
2. On the event consumer page, do one of the following:
   • If the event consumer is inactive, click START CONSUMER to start it.
   • If the event consumer is active, click STOP CONSUMER to stop it.

View failed events for an event consumer

You can drill into an event consumer's details to see the individual events that failed to process.

To view failed events for an event consumer:

1. In the Monitor view, go to the Event Consumer Monitoring tab.
2. Click the name of the event consumer whose events you want to view.

The event grid describes each event with the following columns:

Column	Description
Partition - Offset	The Kafka partition and offset the event was consumed from. Reprocessed events are represented an icon.
Value	The content of the event payload.
Filtered In	Whether the event met the filter condition
Timestamp	When event processing finished. For a failed event, the timestamp is when the last error occurred.
Status	Whether Appian failed to process the event.

View event details

You can open a single event to inspect its payload and any error details. The event detail view includes the following:

• Event Details
  • Status: Whether Appian processed the event successfully or the event failed.
  • Topic: The Kafka topic the event was consumed from.
  • Partition: The Kafka partition the event was consumed from.
  • Offset: The event's position within the Kafka partition.
  • Filtered in: Whether the event met the filter condition.
  • Completed at: When event processing completed. For a failed event, the timestamp is when the last error occurred.
  • Total time: How long the event took to process, from when it was consumed to when processing completed.
  • Reprocessed events contain these additional fields:
    • Reprocessed by: The user who reprocessed the event.
    • Reprocessed at: When event reprocessing completed.
    • Comment: Comment included by the user who initiated reprocessing.
    • Original event: The original event consumed from the Kafka topic.
• Event Payload: The event value as it was consumed from the broker.
  • Key: The Kafka message key used to route the event to a partition.
  • Value: The content of the event payload.
  • Header: The Kafka message headers included with the event.

When viewing event details for a reprocessed event, you'll also see a link to the original event consumed from Kafka. Click BACK TO EVENT to view the original event.

To reprocess an event from the event details, click REPROCESS.

Search and filter events

You can search and filter the event listing to locate a specific event, such as when you reconcile Kafka publish counts against Appian processing outcomes. Search operates only on the events shown on the current page of the listing, not across the entire dataset for the selected time range. Search the event listing in the following ways:

• Keyword search: Search across the columns in the event listing.
• Partition-offset search: Search for an event by its Kafka partition and offset.

You can also filter the event listing by the following:

• Event Type: Filter by the type of event, such as Reprocessed.
• Filtered In: Filter by whether the event passed the event consumer's filter (Yes or No).
• Time Range: Filter by when the event was processed (All, Last 30 Minutes (Default), Last 1 Hour, Last 12 Hours, Last 24 Hours, Last 3 Days, or Custom).

Event monitoring data retention

Event consumer monitoring data on the Appian Monitor page shows a recent window of activity. Only failed event data is retained. Original failed event data is available for the last 7 days.

If you choose to reprocess an event, the retention period for the original failed event does not change. For example, if you reprocess an event four days after it originally failed, the original event's data would expire three days later, but the reprocessed event's data would be available for 7 days after reprocessing.

Who can view event consumer monitoring data

Access to event consumer monitoring data uses the role-based access control (RBAC) defined on each event consumer. Appian uses the event consumer's existing roles rather than introducing separate monitoring-specific roles, and enforces access per event consumer. The following table shows which actions each role can perform on the Appian Monitor page:

Action	Administrator	Editor	Viewer
View monitoring data and reprocessed data	Yes	Yes	Yes
View aggregated metrics	Yes	Yes	Yes
Manually reprocess an event	Yes	No	No

Appian enforces this access control for every request. If a user's access to an event consumer is removed, they can no longer view that event consumer's event data on their next refresh.

How event consumer logs and email alerts relate to monitoring

Event consumer logs and email alerts work alongside the Appian Monitor page. The event_consumer_errors.csv and event_consumer_events_errors.csv log files and the error emails help you detect and identify that a failure occurred, while the Appian Monitor page is where you trace and investigate the individual events involved. For the log file details, see Troubleshoot performance and usage. For the email alert details, see Email error alerts.

Reprocess a failed event

When an individual event fails to process, you can reprocess that single event from the Event Consumer Monitoring tab after you resolve the underlying issue. Reprocess a failed event to recover an isolated failure—such as a one-off data problem, a temporary downstream outage, or a transient record-sync failure—without replaying the entire Kafka topic. You can reprocess the event unchanged, or edit its key, value, or headers first.

Reprocessing one event runs that event through the event consumer's pipeline again: deserializing the event, applying the event filter, and invoking the event handler action. Each reprocessing attempt creates a separate, read-only event entry that is linked to the original failed event and has either a Success or Failed outcome.

Prerequisites for reprocessing a failed event

Before you reprocess a failed event, make sure the following are true:

• Permission: You have Administrator permission on the event consumer. Both editing and reprocessing an event require Administrator permission.
• Eligible event: The failed event is retained and within its retention window. See Reprocessing eligibility and limitations.
• Handler safety: You have assessed whether running the event consumer's handler action again is safe and idempotent.
• Resolved cause: For a failure caused by an external or transient issue, you have resolved the underlying cause before you reprocess the event unchanged.

Reprocess a failed event without changes

Reprocess a failed event without changes when the underlying issue was external or transient and the original event payload is correct For example, if a downstream service was experiencing an outage but has recovered, you can reprocess the event to ensure the data is received by the other system.

To reprocess a failed event without changes:

1. Go to the Event Consumer Monitoring tab on the Appian Monitor page.
2. Open the event consumer and locate the failed event.
3. Open the event to view its details.
4. Reprocess the event without editing the payload.

Appian creates a reprocessed event entry linked to the original event and records whether the attempt succeeded or failed.

Edit and reprocess a failed event

Edit and reprocess a failed event when an isolated problem in the event data caused the failure (such as a malformed timestamp or a field with the wrong data type). Editing creates a corrected copy of the event for the reprocessing attempt. The original event payload remains preserved for auditing purposes.

To edit and reprocess a failed event:

1. In the Monitor view, go to the Event Consumer Monitoring tab.
2. Open the affected event consumer and locate the failed event.
3. Click the event to view its details.
4. Click Edit to change the key, value, or headers.
5. (Optional) If you want to record why you changed the event, enter a comment.
6. Click REPROCESS to submit the edited event.

Appian creates a reprocessed event entry that stores the edited payload used for the attempt, links it to the original event, and records whether the attempt succeeded or failed.

Verify a reprocessing attempt

Each reprocessing attempt has a final outcome of Success or Failed. There is no in-progress status. If a transient error occurs when you reprocess an event, the reprocessing action returns an error and you can try again.

Each reprocessing attempt appears as a separate entry in the original event's chronological reprocessing history. Every attempt links directly to the original event rather than to a previous attempt, so the reprocessing history is a flat list rather than a chain. You can select any retained entry—the original event or an earlier reprocessing attempt—as the source for a new reprocessing attempt.

When you view reprocessed event details, you can view the original event payload and the original event's processed timestamp, as long as the original event is within its retention window.

Reprocessing eligibility and limitations

You can reprocess the following retained event entries:

• Failed original event: An original event that failed to process and is within its retention window.
• Reprocessing attempt: A retained reprocessing attempt, whether it succeeded or failed, that is within its retention window.

Reprocessing a failed event has the following limitations:

• Retention window: You can reprocess an event entry only until its own retention window expires. See Event monitoring data retention. Failed original events, failed reprocessing attempts, and successful reprocessing attempts are retained for 7 days.
• No attempt limit: Appian does not limit the number of manual reprocessing attempts for an event.
• Single event only: You can reprocess one event at a time.

Monitoring performance and usage

Performance logs are written to the APPIAN_HOME/logs/perflogs directory. These logs are written as CSV files for observation and analysis using common spreadsheet tools. The information in them can be used to learn the broad spectrum of the actions performed by consumers and how long those actions take to complete.

Logs are accessible only by system administrators.

Apache Kafka connected system log

Changes to connected system objects are recorded in a comma-separated value (CSV) file, design_objects.csv, in the <APPIAN_HOME>/logs/audit directory. You can refer to this log file to learn when a Kafka connected system object was created, updated, or deleted, and which user took the action.

Column	Description
Timestamp	Date and time when the object was changed
User UUID	The UUID of the user who took the action
Username	The username of the user who took the action
Object UUID	The UUID of the affected object
Object Type	The type of the affected object
Action	The action taken on the object

Troubleshoot performance and usage

Error logs specific to event consumer objects are written to the APPIAN_HOME/logs/event_consumer directory. These logs are written as CSV files for observation and analysis using common spreadsheet tools.

Logs are accessible only by system administrators.

Event consumer system log

This log provides details when an error impacts an event consumer and causes Appian to stop the consumer from processing any more events.

The log data is written to a comma separated value (CSV) file (event_consumer_errors.csv) in the <APPIAN_HOME>/logs/event_consumer directory.

Column Name	Column Description
Timestamp	Date and time when an error occurred
Name	Name of the event consumer design object where the error occurred
MCDO UUID	UUID of the event consumer design object where the error occurred
Error Code	Appian code denoting the error
Error Message	Error message

Event consumer events log

This log captures the details of errors that impact individual events within an event consumer design object.

The log data is written to a comma separated value (CSV) file (event_consumer_events_errors.csv) in the <APPIAN_HOME>/logs/event_consumer directory.

Column Name	Column Description
Timestamp	Date and time the error occurred
Name	Name of the event consumer design object where the error occurred
UUID	UUID of the event consumer design object where the error occurred
Topic Name	Name of the topic being consumed
Offset	Offset within the partition from which the event was consumed
Partition	Partition number from which the event was consumed
Message Size	The size of the event in bytes
Error Code	Appian code denoting the error
Error Message	Error message

Email error alerts

When an event consumer encounters an error, Appian takes a few actions:

• The event consumer is stopped.
• Two email alerts are sent to the event consumer's administrators. One email tells you that the consumer has been stopped and the other provides information about the error and links to investigate further.

How email alerts work

Email alert behavior is configurable for each event consumer. In the Error Handling & Alerts Configurations section of an event consumer, you choose how Appian sends alerts to administrators. Event consumers support two alert modes:

• Digest alerting: Appian aggregates event errors into a periodic summary email. A digest email is sent for an interval only when at least one error occurred during that interval.
• Alerts disabled: Appian sends no native email alerts for individual event errors.

Event consumers without an explicit alerts configuration use digest alerting, because Appian applies safe default values.

Each email alert includes:

• Event Consumer: Name of the event consumer where the error occurred.
• Error details: Appian error code and error message.
• Event Metadata (for individual event errors only): Topic name, partition number, and offset.
• Error Timestamp: Date and time when the error occurred, displayed in your timezone.
• A button to view the event consumer object in Appian Designer.
• A link to view system logs.

Alerts are sent immediately when deactivation errors occur. For individual event errors, a digest alert is sent at the configured digest interval only when at least one error occurred during that interval. If your team uses an incident management tool like PagerDuty or OpsGenie, you can use these emails to start an incident response.

Email alert permissions

To receive email error alerts for an event consumer, make sure:

• Outbound emails are turned on for the environment.
• You have Administrator permissions to the event consumer object in that environment.

To access the event consumer from the link in the email, you must have the Designer user role assigned, which gives access to Appian Designer.

Stream event consumer logs to Amazon S3

Event consumer logs can be streamed to an Amazon S3 bucket once log streaming is enabled in the Admin Console. Streaming event consumer logs lets you forward log data to third-party monitoring tools (such as Splunk, Dynatrace, or Datadog) where you can build custom dashboards and configure proactive alerting for your Kafka integrations.

The following event consumer logs are available for streaming:

• Event Consumer Performance (Details): Per-consumer metrics including latency percentiles, throughput, and message sizes.
• Event Consumer Performance (Summary)): Aggregate totals across all consumers on the node.
• Event Consumer Error Log: Consumer-level errors that caused deactivation, such as circuit breaker trips or authentication failures.
• Event Consumer Events Error Log: Individual event processing errors, including topic, partition, offset, and error details.
• Event Consumer Audit: Design-time operations on Event Consumer objects, including create, update, delete, activate, and deactivate actions.

Log streaming requires a Professional or Signature Success Plan. For setup instructions, see Log Streaming to Amazon S3.

Feedback