This page describes how to use record event data to mine Appian applications.

Overview

You can mine data directly from your Appian applications when you use Records as a data source. This allows you to seamlessly connect to your application data and leverage record events to more easily mine your business processes.

Record events allow you to track how different parts of your business process operate. As a best practice, each record type in your application should have record events configured. This means that each of your major business concepts should have a corresponding Event History record type and Event Type Lookup record type. When you generate these record types as part of your event configuration, they are specifically structured so you can easily prepare the data for mining—allowing you to gather insights faster.

The generated Event History and Event Type Lookup record types provide your core event data, like who performed an activity and when. If there are other record types that include attributes that you want to explore, like order status, shipping date, or total cost, you can add data from those record types as well.

If you are not using record events, you still have the flexibility to use record types in process mining. Before using those record types, review What makes a good data set? to ensure you can easily mine that data.

Requirements

• Appian version 23.2 or later to mine data using record events.

  Note:  If you are not using record events, you can import record data beginning in Appian version 22.3.

• A record type with data sync enabled.
• A service account with at least Viewer permission to the record types you want to mine.

Mine Appian applications

It's easy to mine data from an Appian application when you use the Records data source. This allows you to import record data into data sets and transform that data to meet your mining needs. When you import data from a generated Event History record type and Event Type Lookup record type, it's even easier to prepare your data since the data structure is specifically created for mining. Once the data is transformed, you'll execute the transformation project to transform your record data into an event log and load it into Process Mining.

The following instructions detail how to import and transform data from a generated Event History record type and Event Type Lookup record type.

Add a Records data source

Add a Records data source to import record data into Mining Prep.

Before you begin these steps, make sure you have a service account API key.

To add a Records data source:

1. Sign into Mining Prep.
2. Go to the Data Management tab.
3. In the Data Sources section, click ADD.
4. Select Records.

5. Configure the following properties:

   Property	Value
   Name	Enter the name you want to display in the list of data sources.
   URL	Enter the URL of your Appian Designer environment. For example, https://mysite.appiancloud.com.
   API Key	Enter the service account API key.

6. Click SAVE.

Import record data

Once you've added the Records data source, you can import the record data as data sets. You can only import data from record types that your Process Mining service account can access. You can import data from multiple record types, and each record type will import into a unique data set.

If you configured record events, you should import both the Event History record type and the Event Type Lookup record type for the processes you’re investigating. For example, if you want to investigate your order-to-cash process, you could import the Order Event History record type and the Order Event Type record type.

To import data from your event record types:

1. Click the Records data source.
2. Click ADD.
3. Click IMPORT by the Event History record type you want to import.
4. Click ADD.
5. Click IMPORT by the Event Type Lookup record type you want to import.

Consider importing any additional record types that can enrich your data set. For example, you may consider importing the record type that you’re tracking events for, like Orders, Customers, or Order Items.

Create transformation project

Once you've imported the record data into different data sets, you can create a transformation project to transform the data sets into an event log for mining.

To create a transformation project:

1. Go to the Transformation Project tab.
2. Click ADD.
3. Enter a Name for the project. This will be the name of your event log.
4. Click SAVE.

Transform record data

Now that you've created a transformation project, you'll select the data to transform and apply transformation actions.

If you imported an Event History record type and an Event Type Lookup record type, you’ll need to combine the data into a single data set to see all event information in one place. You’ll also need to identify the data properties, like the case ID, events, and any case attributes.

Select a data source

First you need to populate your project with a data set. You can only populate this building block with one data set to start, but you can add data from other data sets later.

As a best practice, start by using the Event History data set as your data source.

To select the Event History data set as your data source:

1. In the USE DATA SOURCE panel, select the Event History data set.
2. Click USE DATA SOURCE.

Combine data

Now that you've selected the Event History data set as the data source, you'll need to add the Event Type Lookup data set to the project so you can display the different activities that occur in the events. Adding this data will combine the two data sets into one.

To combine the Event History and Event Type Lookup data sets:

1. In the TRANSFORMATION ACTIONS panel, click ADD.
2. Select Add Data.
3. In the ADD DATA panel, select the Event Type Lookup data set.

4. Configure the following properties:

   Property	Value
   Select key column from upper table	Select the eventTypeId field.
   Select matching key column from lower table	Select the Id field.

5. Click PREVIEW to verify the columns are mapped correctly.
6. Click SAVE.

Add any additional data to the project that will enrich your analysis.

Identify the case ID

Next you'll need to identify the case ID, which is the unique identifier of the business case.

If you imported the generated Event History record type, this will be the recordId field. This is the field that links each event to a specific record in Appian.

To identify the the case ID in the Event History and Event Type Lookup data sets:

1. In the IDENTIFY section, click CASE ID.
2. In the IDENTIFY CASE ID panel, select the recordId field.
3. Click SAVE.

Identify the events

Identifying the events is one of the most important parts of mining prep. This specifies the fields that store the types of activities that can occur, the timestamp when the event occurred, and who started it.

To identify the events in the Event History and Event Type Lookup data sets:

1. In the IDENTIFY section, click EVENTS.
2. In the IDENTIFY EVENTS panel, click ADD EVENTS.
3. Select MULTIPLE EVENTS. This allows you to select different columns for the activity and for the timestamp.
4. For Name of Event Group, enter a name for the group of events. For example, Order activities. This is only used in Mining Prep and doesn't display in Process Mining.
5. For Event Name Column, select the name field from the Event Type Lookup data set.
6. Leave the default START selected. By default, the generated Event History record type only has one timestamp field. If you added additional date and time fields to the Event History record type, you can select START & END to identify the beginning and end of an event.

7. For Categorical Attribute Columns, select the user field from the Event History data set.

   Note:  All Event Attributes are optional, but recommended. Event attributes allow you to specify any additional characteristics related to an activity.

8. Click SAVE.

(Optional) Identify case attributes

If you imported other record types, like the record type for which you're tracking events, you can add case attributes to the project to further enrich your data. While event attributes describe characteristics of an activity, case attributes describe characteristics of a case.

Each case can have one or more attributes that describe it in more detail. For example, in an transformation project containing data about your order-to-cash process, you may want to include order name, customer name, or order cost as case attributes.

For more information, see Case Attributes.

Load and execute the transformation project

Once you've finished setting up your transformation project, you'll execute the project to apply the transformation steps and load the event log into Process Mining.

To load and execute the transformation project:

1. Return to the Transformation Projects tab.
2. Select the checkbox next to your transformation project.
3. Click TRANSFORM & LOAD.
4. Select the environment in which to load the event log. The API key is automatically populated for you.
5. Click OK.

That's it! Now you can navigate to your Process Mining environment to start investigating your process.

Update your data

To ensure you always have the latest data from your record types, we'll automatically update the data in your records data sets whenever you execute or schedule updates to a transformation project.

You can manually reload your data for in-progress transformation projects. Since you typically won't set up a transform and load schedule until your project is complete, manually reloading your data keeps it fresh during this stage.

To manually update your data set:

1. Go to the data set that corresponds with a record type.
2. Click RELOAD.