Process Mining is deprecated with Appian 24.2 and will no longer be available in an upcoming release. Instead, we encourage customers to use Process HQ to explore and analyze business processes and data. |
After you've created your data sets, the next step is to create a transformation project.
On the Transformation Projects tab, you can create transformation projects to transform data sets into event logs. Mining Prep must transform data sets into event logs for process mining analysis. An event log is a list of events that process mining uses to analyze processes. Events represent activities in process mining. Activities are tasks in a business process that may be automated or performed by a human. They are often associated with start and end time stamps, who performed the activity, or how much the activity cost.
To create a transformation project, click the Add button on the Transformation Projects tab. You can also create a transformation project directly from a data set by clicking the Create Transformation Project button.
Transformation projects consist of:
Once you've built the transformation project, you can click the Transform & Load button to execute the transformation actions and load the event log into Process Mining.
Transformation project statuses indicate the execution status of the project whenever you manually or automatically execute a transformation project.
Statuses display in the Status column alongside a time stamp that indicates when the status occurred:
Use building blocks to organize your transformation projects. A building block is automatically created when you add a transformation project.
Building blocks can contain one or more data sets.
In each building block, you identify the case ID, events, and attributes for the data set. You must define these properties before you can run the final transformation. You also add transformation actions to building blocks. The final transformation is comprised of a series of transformation actions. A transformation action defines a single data manipulation instance.
Tip: Transformation actions run inside the building blocks in which they are defined. For smaller projects, it may make sense to keep all data sets in a single building block, so you only have to add the transformation actions once. As projects grow, you may wish to add more building blocks to more clearly organize data sets. However, when you have multiple building blocks, you will have to add transformation actions to each block.
Building blocks can contain one or many data sets. If you add multiple data sets to a single building block, they must share a common column of values to serve as a key.
During the final transformation, Mining Prep automatically combines all valid building blocks into one event log for process mining analysis.
After you add a new building block, you can choose to add an existing or new data set.
To add an existing data set to a building block:
To add a new data set to a building block:
Transformation actions define changes to the data's format, structure, and name in the event log. You add transformation actions from the building-block level, and these actions only apply to data inside the building block.
To create a complete event log that process mining can analyze, it must contain the following items:
Tip: Although case attributes are optional, they can be helpful during the process mining phase. Case attributes that you define during Mining Prep are available as filters in process mining and can be used as dimensions when creating charts in a dashboard.
It is common to need to transform your data set so that you can define these properties. For example, events must contain columns in a datetime format to serve as time stamps. You can add a date conversion transformation action to a column so that it contains the correct data type. See Identify data properties for more information.
Transformation building blocks store transformation actions. You access these actions from a building block.
To add a transformation action:
Mining prep supports the following transformation actions:
The merge columns transformation action adds a new column that combines values from two or more columns, separated by characters you define.
For example, if you have an attribute that's associated with an ID, you can combine these into a single column with the merge transformation action.
To merge columns:
-
.The match columns transformation action adds new columns and populates them according to match conditions.
To create a new column, enter the new column name in the New Column Name field.
After creating the column, you need to specify its data type in the Column Type field. In the preceding example, the new column is a string
type. The following table describes the column types you can select:
Type | Description | Example |
---|---|---|
String | Text or combinations of alphabetical and numerical characters. | jdoe44 |
Integer | Whole, non-decimal numbers. | 8675 |
Double | Double-precision floating-point numbers. | 1403.22 |
After selecting the data type, you can define the conditions that determine how the match column populates. Each condition starts with a comparison between two columns. If the comparison is true
, the data from the Then field is returned; if the comparison is false
, the data from the IF NOT THEN field is returned.
The comparison operators you can choose from are:
Comparison Operators | Description | Example |
---|---|---|
= | is equal to | 1=2 returns false |
!= | is not equal to | 1!=2 returns true |
Contains | contains | "booker12" contains b returns true |
Is null | Empty cell | "Jane Doe" is null returns false |
Is not null | Not empty cell | "Jane Doe" is not null returns true |
If you have more than one condition you want to define, click Add Condition to add additional conditions.
The replace values transformation action replaces values in existing columns based on conditions you define. For example, you may need to replace a special character or abbreviation in an attribute column.
To start, select an existing column from the Select Column field.
After selecting the column, you can define the conditions that determine how the column values are replaced. Each condition starts with a comparison between two columns. If the comparison is true
, the data from the Then field is returned; if the comparison is false
, the data from the IF NOT THEN field is returned.
The comparison operators you can choose from are:
Comparison Operators | Description | Example |
---|---|---|
= | is equal to | 1=2 returns false |
!= | is not equal to | 1!=2 returns true |
Contains | contains | "booker12" contains b returns true |
Is null | Empty cell | "Jane Doe" is null returns false |
Is not null | Not empty cell | "Jane Doe" is not null returns true |
If you have more than one condition you want to define, click Add Condition to add additional conditions.
The filter transformation action reduces the number of rows in your data set to only those that match filter criteria.
Filters on a column of type string
are case sensitive and support the following comparison operators:
Comparison Operators | Description | Example |
---|---|---|
= | is equal to | 1=2 returns false |
!= | is not equal to | 1!=2 returns true |
Contains | contains | "booker12" contains b returns true |
Is null | Empty cell | "Jane Doe" is null returns false |
Is not null | Not empty cell | "Jane Doe" is not null returns true |
Filters on a column of type datetime, integer, or double additionally support the following comparison operators:
Comparison Operators | Description | Example |
---|---|---|
< | Less than | 1<2 returns true |
> | Greater than | 1>2 returns false |
<= | Less than or equal | 1<=2 returns true |
>= | Greater than or equal | 1>=2 returns false |
These operators allow you to filter ranges of values in your data set.
Additional information from other data sets can enrich your process mining analysis. This might be data from a different system like your CRM system or internal tables from your business unit. With the add data transformation action, you can add supplemental data sets to your transformation project.
To add the add data transformation action to a building block:
Note: The add data transformation action only adds rows where values match between the new and source data key columns.
After adding data, you can use any new columns as events or attributes, and you can continue to transform the new combined data set.
Use the rename column transformation action to change the name of an existing column. For example, you might want to change the name of a column to be more descriptive or contain more context about the data it contains.
Case and event attributes are often good columns to rename since they are available as filters in Process Mining after transforming and loading. Rename these columns to make attribute filters more understandable for other users in Process Mining.
To rename a column:
Use the date conversion transformation action to standardize dates within your transformations and convert strings into UTC time stamps. In addition to case IDs and events, time stamps are a fundamental component of the data used for process mining.
Note: The date conversion transformation action converts a string into the UTC time zone based on the original time zone of the data set. The result of this conversion is UTC, not your local time zone.
When you add a date conversion transformation action, you must specify the following:
MM/dd/yy hh:mm
string matches data like this: 11/23/1955 12:00
.datetime format strings use the following case-sensitive characters:
Character | Description | Example |
---|---|---|
y | year | 1996 |
M | month | Jul or 07 |
d | day | 10 |
a | half day | PM |
h | hour (0-23) | 0 |
m | min | 30 |
Click Preview to verify the transformation result, before clicking Save.
Mining Prep saves, operates, and previews all date and time information using the Coordinated Universal Time (UTC) time zone. When you perform date conversions, you must specify the format of the time stamp and the local time zone of the data set.
When you preview and save the date conversion, Mining Prep converts the value from your selected time zone to UTC.
For example, let's say your data set contains a column with time stamps that look like this: 2016/01/04 12:09:44
.
You would specify the format string as yyyy/MM/dd hh:mm:ss
for this data. Assuming the event occurred in McLean, VA, and your IT system uses this local time zone when creating the log, you would select US/Eastern
as the respective time zone.
After you save the date conversion, time stamp will now display as 04.01.2016 05:09:44
UTC. This represents 2016/01/04 12:09:44
EST.
The match pattern transformation action replaces or extracts data from a column based on regular expression (regex) patterns. Matched data is extracted or replaced into the Output column. The Regular Expression field supports PCRE2 syntax.
To match a pattern:
Note: The input can be any data type, but the output will always be a string.
Let's look at a couple use cases that involve a column containing email addresses.
First, imagine you'd like to extract all of the email domains from the email addresses column into a new column. This would be useful to use as a company name case attribute.
(?<=@)[^.]+(?=\.)
If the email address column contained karen.anderson@appian.com
, the output column would display appian
. Here's how the preview looks for this pattern match:
Next, imagine you'd instead like to replace the email domains but leave the rest of the email address.
(?<=@)[^.]+(?=\.)
appiancorp
If the email address column contained amy.lee@appian.com
, the output column would display amy.lee@appiancorp.com
.
To create a complete event log that process mining can analyze, it must contain the following items:
Tip: Although case attributes are optional, they can be helpful during the process mining phase. Case attributes that you define during Mining Prep are available as filters in process mining and can be used as dimensions when creating charts in a dashboard.
It is common to need to transform your data set before you can define these properties.
The case ID is the unique identifier of a business case, which usually consists of a combination of letters and numbers. A business case represents a real business transaction in an IT system. A case often consists of multiple events. Each event can have unique time stamps and case attributes.
In process mining, a business case runs through a particular process variant; therefore, it is the image of exactly one process cycle. The case ID identifies each case individually.
To identify the case ID property from a building block:
The digital representation of when an activity occurred. An instance of an activity. All events that are assigned to a business transaction form a unique case. Similarly, a business process consists of several activities. An event has an associated case ID, time stamp, and other contextual data. The time stamps correspond to the start or end time of the event.
Not all events have start and end time stamps, but you need at least a start time stamp to successfully define an event. Additionally, process mining can only calculate the duration of events if a start and end time stamp exist.
Note: You can only identify events if Mining Prep recognizes a column of type datetime. If you don't have datetime columns in your data set, Mining Prep will automatically prompt you to select a column from your data set to perform a date conversion.
Mining Prep supports two event identification types:
To identify single events:
To identify multiple events:
Case attributes are an optional property of an event log and describe characteristics of a case. Case attributes help you make observations such as how costly a deviation or variant is in your process. Each case can have one or more attributes that describe it in more detail. For example, customer or vehicle types can be case attributes. Attribute characteristics can vary from case to case.
You can select the following attribute types:
Attribute | Description | Valid Types |
---|---|---|
Numerical | Numeric characteristics like “total payment amount.” | integer , double |
Impact | Numeric characteristics that are useful when summed together. In the Process Mining module, the Variants and Deviations pages display the summed values of impact attributes beneath the Impact column. | integer , double |
Categorical | Descriptive characteristics like “responsible department.” | integer , double , string |
Numerical and categorical attributes that you define in Mining Prep are available as filters in the Process Mining module.
Note: Make sure you use valid data types with your attributes to prevent transformation errors later on. Appian displays a warning if you select a column with an invalid data type. Note that if your data uses a comma as a decimal separator, it is assumed to be a string
. This may not be accurate depending on your specific data set.
Each column in your data set is associated with a data type. The data type indicates what kind of data the column contains. This is important to keep in mind, because you can only merge columns of the same data type. To help you keep track each column's data type, Mining Prep displays a data type icon in the column headers of your data previews.
The following table describes the possible data types and associated data type icon:
Data Type | Icon | Example |
---|---|---|
string | angela.lewis |
|
integer | 20 |
|
decimal | 20.50 |
|
datetime | 20.02.2022 12:10:13 UTC |
When your transformation project is complete, you are ready to run the transformation and load the event log into Process Mining.
To transform and load your project into Process Mining:
Click Transform & Load.
Once you've manually transformed and loaded a transformation project, you can schedule these actions to happen automatically.
After a successful transform and load in Mining Prep, a Success message displays with a link to your Process Mining environment. This link will navigate you to the discovered model that's based on the newly created event log.
To access your Process Mining environment:
Alternatively, you can access Process Mining at any time from the navigation menu.
Note: To access any links between Mining Prep and Process Mining, your admin must configure the miningBaseUrl option during installation.
You can define a schedule for your transformation project to load updated data to Process Mining for continuous analytics, monitoring, and notifications.
Mining Prep automatically extracts, transforms, and loads the updated data from your source system to Process Mining on a reoccurring basis.
See Schedule for more information.
Transform and Load Data