Build Your Best Data Fabric

This page outlines best practices for building a scalable, performant, and AI-ready data fabric. Review and implement these suggestions as you create record types to ensure a strong foundation for your applications.

Use synced record types

Record types are the foundation of Appian's data fabric. As a best practice, you should create synced record types whenever possible so you can take advantage of automatic performance optimization and all of our data fabric capabilities.

If syncing your data isn't an option, you can create unsynced record types to directly access data, while maintaining access to powerful features like relationships, custom record fields, and record-level security.

If your existing applications were built using custom data types (CDTs) and data store entities (DSEs), you can create record types specifically for Process HQ, AI Copilot, and new additions to your app. You can even generate multiple record types to quickly connect to your existing database tables.

Scale to large data volumes

Appian provides the flexibility to work with any volume of data. You can access as many rows as your business requires across both synced and unsynced record types, ensuring your data fabric scales alongside your enterprise needs.

For most use cases, no special configuration is required to support large datasets. However, as your data volume grows, consider the following practices to maintain optimal performance:

• Sync only the data you need. Syncing unnecessary data increases resource consumption and can slow down queries. Use sync filters to limit synced data to what your application actually uses.
• Use incremental syncs. Incremental syncs capture only new or changed data, making them faster and less resource-intensive than full syncs. Use incremental syncs for data that changes frequently by external systems.
• Monitor query performance. Use the Query Performance Monitor to identify slow queries and optimize them, regardless of whether your record type is synced or unsynced. If you plan to connect to a large-volume data source, conduct performance testing to verify that your queries perform well at the load and volume you expect.

If your capability tier includes a synced row limit, you can use different sync options and data modeling strategies to ensure your record types include the most relevant data.

Data modeling strategies for large datasets

Use these sync options and data modeling strategies to access the data you need:

Example: Use only the most recent data

An Event History record type often captures millions of rows of data. Because an application may only require a timeline of the most recent activity, you can enable Keep data available at high volumes. Appian then dynamically syncs the latest events and automatically removes older rows to keep the record type active and performing optimally within your tier's row limit.

Example: Use synced and unsynced record types

If you have a database table that you've already tuned to performantly handle millions of orders, you might use a combination of synced and unsynced record types.

In this example, you could:

• Create an unsynced record type for All Orders: Use this record type to build reports and processes in your application. Since it's unsynced, the record type leverages the performance configurations on the source.
• Create synced record types for Active Orders and Closed Orders:
  • Apply sync filters to each record type based on the order status.
  • Use the synced record types in Process HQ to analyze and optimize your order business process since unsynced record types aren't supported in process insights.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Sync changes efficiently

To keep your synced data current with minimal resource usage, choose the sync option that matches how your data changes:

When multiple record types point to the same database table, Appian will start a sync on each record type whenever a smart service sync occurs on that table. When this occurs, you can expect a proportional increase in sync requests.

For applications with frequent writes, use this pattern to sync data independently from the writes to the source.

Build a unified data model

You should follow common data modeling practices and patterns, such as:

• Creating a record type for each of your business concepts or processes.
• Storing lookup data in a separate record type.
• Managing lists with one-to-many relationships.
• Effectively modeling many-to-many relationships.
• Managing documents in a separate record type.

Create bi-directional record type relationships

By default, a relationship only exists on the record type where it was created. To help both people and AI models easily find related information, you should create bi-directional relationships so the relationship exists on both record types.

For example, if you add a many-to-one relationship from the Case record type to the Case Type record type, you should also add a one-to-many relationship from the Case Type record type to the Case record type.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

You can quickly add bi-directional relationships using suggested relationships.

Provide meaningful metadata for users and AI

Data is more valuable to users and AI when it includes clear context about how it was created, what it's used for, and what it means.

The following recommendations outline specific metadata to include in your data fabric.

Record type display names

In Process HQ, users see the Display Name configured for each record type.

When configuring the record type's display name:

• Use distinct business terms like Case Managers and Human Resource Employees.
• Avoid defaults like CM Employee and HR Employees.

Record type descriptions

You can generate a record type description with AI Copilot to quickly add a clear and comprehensive description.

When writing your own descriptions:

• Explain the record type's purpose in 1-2 sentences.
• Include key relationships or constraints when relevant.
• Use plain language that business users can understand.
• Ensure someone unfamiliar with your app can understand the purpose.

Example #1: A record type called Case Type has a complex structure that isn't obvious from the name alone. An effective description that explains this structure could be:

Contains the different types of cases. There are categories of cases and there are types within those categories. If the isCaseCategory field is set to true, the case type is a category, otherwise it is a type within a category. For example, Commercial is a category and Demolition a type within the Commercial category.

Example #2: To specify a many-to-many relationship between Employee and Project, you need to use a third record type, called Assignments. The third record type could have a description like:

Maps employees to projects. For example: Employee A and Employee B work on Project 1, Employee B and Employee C work on Project 2. Project 1 and 2 each have two employees assigned to work on them. Employee A and Employee C each work on one project, Employee B works on two projects.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Relationship names

Relationship names appear in records-powered components and in Process HQ when users build their own reports. AI models also use this information to better understand your data.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Keep the following in mind when naming relationships:

• Ensure the relationship name matches the name of the related record type.
• Use camel casing to format your relationship names.
• Remove application prefixes from relationship names. For example, use supportCases instead of RE_supportCases.

Field display names

You can configure a Display Name for each record field.

When configuring a display name for a field, consider the following:

• If you use a field to handle soft deletions, instruct AI on how to handle them.

  For example: In an isActive field, the description could be, Indicates if the group still exists or not. When asking AI Copilot to return a list of departments, this field should be used to filter for only active departments, unless otherwise stated.

• Indicate the business reasons for why a field may be empty.

  For example: In an endDate field, the description could be, The date an employee leaves the organization. If this field is empty, it means the employee is active.

Field descriptions

While record type descriptions provide overall context, field names and descriptions add important details about individual data points. Good field names, display names, and descriptions provide users and AI with additional context.

Keep the following in mind when providing field descriptions:

• If the field's display name is self-explanatory, you do not need to add additional information in the field's description.
• Avoid using developer terminology (like "record type" or "synced") if the record type appears in Process HQ since this information is available to business users.
• List synonyms or acronyms specific to the field in its description to provide additional context. This keeps field descriptions focused and prevents unnecessary repetition.

Example: In the Case record type, each record has a dueDate field which contains Date and Time data. You also have a custom record field (daysTilDue)that calculates the difference between today's date and the due date, so you can quickly see how many days are left until the deadline.

To help a person or AI model understand the information that this custom record field contains, you could add the following field description: Number of days left until a deadline; or number of days until service-level-agreement (or SLA).

Optimize field usage

In addition to adding field metadata, like field display names and descriptions, you should consider the following when creating and modifying fields.

Create custom record fields

Sometimes, a record type won't explicitly contain the necessary data to answer a question. For example, you may need to aggregate, extract, or change your existing data to show specific insights. This is a good opportunity to use custom record fields.

Example: You have an Order Item record type which contains fields like cost and salesPrice. You can create a profit field to the record type, which subtracts cost from salesPrice.

See additional examples of when to use custom record fields.

Create a full name field

If a record type contains information about people, include a field with the full name of the user. This helps users and AI models identify individuals accurately.

To provide a full name, use one of the following methods:

• Use a standardized email format, such as firstName.lastName.
• Create a custom record field to extract the name from an email or username field and save it as a new field called fullName.

Example: The Case record type stores assignee usernames as j.smith. A user may ask AI Copilot a question such as "How many cases are assigned to John Smith?". However, the username doesn't include the first name. To help the AI model successfully reference the correct user when answering this question, you could create a custom record field that includes the first and last name from the username.

If you're having trouble getting accurate responses for questions including usernames, contact Appian Support to update your site's properties to use the appropriate format.

Use Extra Long Text fields to store large volumes of data

Extra Long Text fields allow you to store and display large amounts of unstructured data, such as performance reviews, in an interface.

The behavior and limits of this field type depend on whether you use a synced or unsynced record type:

• Synced record types: Extra Long Text fields support up to 64,000 characters. If a field exceeds this character limit, the sync will fail.
• Unsynced record types: Extra Long Text fields have no character limit. You should use an unsynced record type to manage large, unstructured data sources that exceed the 64,000 character limit.

Extra Long Text fields can be queried using a!queryRecordType() and a!queryRecordByIdentifier(). To ensure optimal performance, these fields cannot be used in filters, sorts, aggregations, or records-powered components.

You can configure an Extra Long Text field when you create a record type or generate a database table. You can also change an existing Text field to an Extra Long Text field at any time.

Change a Text field to Extra Long Text

To change a Text field to type Extra Long Text:

1. In the record type, go to Data Model.
2. Click MAP RECORD FIELDS.

3. For Record Field Type, use the dropdown to select Extra Long Text.

   

   Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

4. Click FINISH.
5. Click SAVE.

Reusable validations and interfaces for fields

You can create and add common validations and reusable user interfaces to your existing record fields. Configuring validations and interfaces for record fields that can be reused throughout your application allows for convenient, reliable, and uniform reuse without the need to continually configure the way a field displays or validates.

The following sections provide general recommendations on when to configure these reusable elements.

Reusable record field validations

Using validations in interfaces ensures users submit correct information in the right format. Record field validations let you define commonly used validations once and apply them throughout your apps.

For example, if you have an email field used in multiple interfaces, create validations to ensure inputs include an @ symbol or use a specific domain.

The expression in the Validations section in the record field might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{
  if(
    a!isInText(
      text: recordType!Employee.fields.email,
      subtext: "@"
    ),
    "",
    "Email address must contain an @ symbol."
  ),
  if(
    a!endsWith(
      text: recordType!Employee.fields.email,
      endsWithText: "appian.com"
      ),
    "",
    "Email address must be from the appian.com dommain."
  )
}  

Instead of writing that expression every time you use the record field in an interface, you can define the expression one time in the record field and call it wherever you need it in your app.

The validations configured on a record field are not automatically evaluated and you must use the a!applyValidations function function to reference them. This ensures that the reusable validations are being run intentionally.

For example, here's what using the a!applyValidations() function to reference your record field validations might look like when used in a reusable input interface for a record field:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
a!textField(
  label: recordType!Employee.fields.email.properties.displayName,
  labelPosition: "ABOVE",
  showWhen: fn!true(),
  value: ri!employee[recordType!Employee.fields.email],
  saveInto: ri!employee[recordType!Employee.fields.email],
  align: "LEFT",
  validations: {
    a!applyValidations(
      recordField: recordType!Employee.fields.email,
      context: ri!employee
    )
  }
)

Only add validations to the record field that you want to apply consistently throughout your app. If you need to run one-time validations on a record field to meet the needs of a specific interface or expression, you can define additional validations using the a!applyValidations function. These one time validations are not saved as part of the record field validations and are not reusable.

Reusable record field interfaces

Configuring reusable user interfaces for record fields speeds up development by letting you reference the field and interface directly, rather than reconfiguring each time.

You can add two reusable user interfaces to a record field:

• A read-only interface: Defines what the record field should look like when used in read-only interfaces, like summary views.
• An input interface: Defines what the record field should look like when used in input interfaces, like forms.

You can either generate interfaces for the record field directly from the record field or add your own custom ones.

The generated interfaces include the component associated with the record field, the basic component configurations, and connections to the record field. And, if a record field has validations configured, the generated input interface for the record field will automatically use the a!applyValidations() function.

For example, if you have a record field for email addresses, the expression for the generated input interface might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
a!textField(
  label: recordType!Employee.fields.email.properties.displayName,
  labelPosition: "ABOVE",
  showWhen: fn!true(),
  value: ri!employee[recordType!Employee.fields.email],
  saveInto: ri!employee[recordType!Employee.fields.email],
  align: "LEFT",
  validations: {
    a!applyValidations(
      recordField: recordType!Employee.fields.email,
      context: ri!employee
    )
  }
)

To keep your applications clutter-free, only create reusable user interfaces for record fields that will be used in interfaces multiple times throughout your application.

Learn more about adding reusable interfaces and validations to record fields.

Protect sensitive data

Enterprise data often includes information that must be protected for business, privacy, or regulatory reasons. For example, you may want to restrict customer information to the salesperson working with a client, or you may need to allow only managers and executives to view company financial data.

As you construct your data model, identify the information that requires protection and the users or groups that are allowed to access it. When you build the record type that includes sensitive data, you can then configure security controls to ensure protected record data can only be viewed by certain users or groups.

Appian's AI features also respect all object-level, record-level, and field-level security configured on the record type. If AI Copilot provides answers that don't seem correct, it may be because of how security is set up for a given user.

In Process HQ, users can only access and ask questions about the record types that are available as datasets in the Data Catalog. By default, record types are not available as datasets, so you'll need to configure record types to show as datasets in Process HQ.

When you show a record type as a dataset, you can also use that record type in the data fabric chatbot component.

Feedback