Configure Smart Search

This page explains where and when to use smart search, how smart search works, and how to implement and use it in your applications.

What is smart search?

By default, any filters, search boxes, and chatbots in your applications use traditional keyword search to find exact or partial matches in your data. For example, if you enter "electricity outage" in the search box of a read-only grid, it will only return rows that include any of these words.

With smart search, you don’t have to know the exact value you’re looking for to get relevant results. Instead, if you use smart search in a read-only grid and search for "electricity outage," smart search could return results that mention "power failure," "power outage," or "electricity is down."

Smart search is an AI-powered semantic search that allows you and your application users to intuitively search through the data in your synced record types.

Why use smart search?

When you use smart search in your applications, you can easily:

• Uncover hidden connections: Smart search identifies the right data even when different terminology is used. This makes it easier to find similar or related data.
• Investigate patterns: Smart search can help you recognize trends across your data, such as similar or duplicate cases logged at the same time by different users.
• Find what you need faster: Smart search uses similarity scores to automatically rank search results. Higher scores show closer semantic matches.
• Integrate with other AI features to accelerate workflows: Combine smart search with AI skills to use the search results to automate decision making.

Where to use smart search

You can use smart search in the following locations to allow you and your users find the information you need:

• The search box in read-only grids. You can use smart search in the search box of a read-only grid so users can leverage both keyword search and smart search to quickly find the information they need. How you configure smart search on the grid depends on whether you're using the default search available in the grid, or a custom search box configured with other components.

  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

  Note:  Smart search is not available in the search box for record lists.

• a!queryRecordType() expressions. Within an a!queryRecordType() function, you can configure an a!queryFilter() expression using the "search" operator. This will automatically apply smart search on the text used in the value parameter of the filter. Then, you can display the results of your query in interfaces or expression rules, or use the results to determine the flow of your process models.

  For example, you could query all cases and use smart search to identify similar cases based on case names and descriptions. Then, you can display the similar cases at the top in a card layout so users can explore similar cases and their resolution, as shown below.

  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

• Process models. Smart search can work behind the scenes in a process model to support automated decision-making. By combining smart search with AI skills, you can deliver next-step suggestions without user input. This approach helps you automate common decisions and deliver consistent, informed actions across similar cases.
  
  

• Data fabric chatbots. Smart search is automatically applied to a data fabric chatbot when you use a record type with smart search enabled in the recordTypes parameter. This allows the chatbot to understand the intent behind a user's question, not just the specific keywords they use.

When to use smart search versus keyword search

Appian provides several ways for users to search and filter data, including:

• Smart search. A search mechanism that finds a broader range of results by analyzing the meaning behind your search query, not just the specific words.
• Keyword search. A search mechanism that returns a focused set of results by looking for exact or partial matches to your search term.
• Filters. You can configure filters with a!queryFilter() or as user filters. You should not enable smart search on lookup data; instead, you should you should use filters.

You can use these search or filter methods by themselves, or combine them to create a powerful and intuitive search experience. Understanding when to use each option will help you build the best experience for your users.

Use this table to decide which combination of search and filters best fits your users' needs and the data in your record type:

How does smart search work

When you enable smart search in your record type, you’re integrating a powerful semantic search capability across your data fabric. This allows your users to find relevant information even if they don't use the exact keywords in their search.

When you implement smart search in your applications, users can find information by describing what they are looking for. Smart search compares the conceptual meaning of their search to the text in your record fields—uncovering connections, detecting patterns, and identifying related data across rows of data.

Each search result provides specific information about why it was returned, including a similarity score that ranks how relevant it is, and the source content that shows where the match was found. You can use these details in your application by referencing the searchResults property in the record type to filter, sort, or display the results.

To perform a semantic search, Appian creates a dedicated search index of the data in your synced record type. This index is updated whenever a sync completes on the record type, so smart search is always searching across the latest data in your record type.

The rest of this section goes into more detail about these aspects of how smart search works.

Similarity score

A similarity score is a number between 0 and 1 that indicates how closely a field matches your search. A similarity score is not a percentage match, instead, it is a mathematical value based on the semantic meaning of the content, not a keyword-by-keyword comparison.

In smart search, every search result gets a similarity score, with higher scores meaning the result is more relevant to what you're looking for. While a similarity score of 1 always represents an exact match, the significance of any other score is relative. What qualifies as a strong or weak match depends entirely on your data and how you're using smart search.

The list below provides a general starting point for interpreting similarity scores.

• Exact Match: 1
• Strong Match: 0.8 or greater
• Moderate Match: Between 0.6 and 0.79
• Fair Match: Between 0.4 and 0.59
• Weak Match: 0.39 or below
• No Match: null

Example 1: Case record type

For example, let's say you have smart search enabled on the description field in the Case record type. This field contains narrative, sentence-based text about a support case.

When you search for customer reporting an incorrect invoice charge in a case grid, you may see the following results. Based on the returned results, you determine that a score starting at 0.8 is a strong match for this field since it identifies direct complaints. A score of 0.2 or less is considered a weak match since it found related but incorrect topics (like requesting a copy instead of reporting an error).

Example 2: Order record type

Now let's say you have smart search enabled on the description field in the Order record type. This field contains less structured text about customer orders, often mixed with product features and notes.

When you search for raincoats in black in an order grid, you may see the following results. In the returned results, you notice that more relevant results are returned at a lower similarity score. As such, for the description field in the Order record type, you decide that a strong match actually starts at 0.3.

Primary similarity score

When you search across multiple fields, a row may have several search matches. The highest similarity score found in the row is called the primary similarity score. You can use the primary similarity score to filter and sort data by the strongest search matches, regardless of which field they're found in.

For example, let's say you enabled smart search on the fields title, description, and name. When you search for the term "electrical outage" in a read-only grid, the grid will look for matches in each of these fields, as seen in the table below.

To view the strongest matches first, you could sort the grid by the primary similarity score so the row with "Electrical is down" appears first.

Using similarity scores

You can use the similarity score to filter or sort results, making it easier to focus on the information that matters most. To reference the similarity score, as well as other search result data, you can use the searchResults property of the record type.

As a best practice, you should sort and filter data by the primary similarity score since you're likely searching across multiple fields. To reference the primary similarity score, use the following reference:

recordType!<Record Type Name>.searchResults.allSearchFields.similarityScore

You can use similarity scores in your application in a couple of ways:

• In a read-only grid:

  • You can sort and filter a grid by similarity score.

  • Add a column in the read-only grid to display the quality of the similarity score using icons or tags. For example, if you determine that a strong score is 0.8, you could display "Strong" as a tag in that column. You should not display the numerical similarity score to application users, since understanding their significance is data and case specific. See an example displaying this column in a grid.

  • Set a similarity score threshold to filter out less relevant search results. By default, the threshold is 1 (only keyword matches), but lowering it will return both keyword and smart search matches with scores at or above the threshold.
    
    

• In an a!queryRecordType() expression:
  • You can sort and filter a query by similarity score. When you are searching across multiple fields, you must combine the search filters with an "OR" operator to return a primary similarity score.

  For example, the following expression searches the same value on the title and description fields. To search through those fields and return accurate primary similarity scores, the two search filters are combined with an "OR" operator.

  1
  2
  3
  4
  5
  6
  7
  8
  9
  10
  11
  12
  13
  14
  15
  16
  17
  18
  19
  20
  21
  22
  23
  24
  25
  26
  27
  28
  29
  30
  31
  32
  33
  34
  35
  36
  37
  38
  39
  40
  41
  42
  43
  44
  45
  46
  47
  48
  49
  50
  51
   a!localVariables(
    local!searchTerm: "software update",
    a!queryRecordType(
      recordType: recordType!SSTA Case,
      fields: {
        recordType!SSTA Case.fields.id,
        recordType!SSTA Case.fields.title,
        recordType!SSTA Case.fields.description,
        recordType!SSTA Case.searchResults.allSearchFields.similarityScore,
  
      },
      filters: a!queryLogicalExpression(
        operator: "AND",
        logicalExpressions: a!queryLogicalExpression(
          operator: "OR",
          /* Search for the same term across title and description */
          /* Return results if they appear in either field */
          filters: {
            /* Search on the title field */
            a!queryFilter(
              field: recordType!SSTA Case.fields.title,
              operator: "search",
              value: local!searchTerm,
  
            ),
            /* Search on the description field */
            a!queryFilter(
              field: recordType!SSTA Case.fields.description,
              operator: "search",
              value: local!searchTerm
            )
          }
        ),
        /* Only return results with a 0.5 similarity score or higher */
        filters: a!queryFilter(
          field: recordType!SSTA Case.searchResults.allSearchFields.similarityScore,
          operator: ">=",
          value: 0.5
        )
      ),
      pagingInfo: a!pagingInfo(
        startIndex: 1,
        batchSize: 100,
        /* Sort by primary similarity score so the best matches are at the top */
        sort: a!sortInfo(
          field: recordType!SSTA Case.searchResults.allSearchFields.similarityScore,
          ascending: false()
        )
      )
    )
   )
  

Source content

The source content is the piece of text where the search term was found. You can reference the source content from the searchResults property of the record type, and use it to:

• Test and verify the information associated with the similarity score. By reviewing the source content alongside the similarity score of each result, you can determine what score represents a strong match versus a weak match for your specific data.

  For example, you could create a query that searches the description field by the term server down. In your query, you can return both the similarity score and the source content to see what content caused the match. You may see that a row had a score of 0.8 because the source content had the phrase “server went down last night,” which indicates that it’s a strong match.
  
  

• Send the matched search term and its surrounding text to an AI skill, so you can take the search results a step further.

  For example, you could have a custom interface that uses smart search to return relevant order documents. After users search for a specific document, you can pass the search results to a process model with a Prompt Builder AI skill. This AI skill can parse the source content to format and display each paragraph where the search term was found, so users can quickly identify the relevant information in the document itself.

However, you cannot use the source content to filter or sort a query; instead, use the similarity score.

Search matches

Smart search can return multiple matches per field when searching extra long text and document fields. These matches appear in the results of a query in the searchMatches property, and they can help you understand why certain data was returned.

In the results, searchMatches is a list of maps that show where your search matched the field content. Each item in the list includes:

• similarityScore: A number between 0 and 1 that shows how closely the content matches the search query.
• sourceContent: The piece of text where the search term was found, helping you understand the context.
• page: The page number where the match was found. This is only available for Document fields that contain PDFs.

By default, smart search includes up to 10 matches per field and are sorted by the similarity score in descending order. Learn how to filter or sort by page or source content.

Indexing

Indexing is the process smart search uses to prepare your synced data for searching. Indexing automatically occurs on the record type when:

• You deploy the record type. Appian will index all rows.
• You add or update fields enabled for smart search. Appian will index all rows.
• A full sync, incremental sync, or smart service sync completes. Appian only indexes data that has changed since the last sync, which typically makes indexing faster.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

If a sync fails on the record type and the record data becomes unavailable, then smart search will also be unavailable, since there's no data to index. In other cases, the sync may succeed on the record type, but certain data may fail to index due to a document being too large or from exceeding the limit of data that can be indexed.

You can configure an indexing failure tolerance level when you enable smart search on your record type, so you can determine how many rows can fail to index before smart search is taken offline. This helps balance accuracy and availability based on your needs.

Usage considerations for smart search

Before enabling fields for smart search, consider the following.

Region availability

Smart search is available for environments in select regions. If your Appian environment isn't in a supported region, you can elect to send your data to a supported region. This doesn't change your environment's region. Contact Appian Support to learn more.

Supported record types

Smart search can only be enabled on synced record types.

Supported fields

Smart search can be enabled on the following field types:

• Text fields.
• Extra long text fields.
• Document fields.

You should not enable smart search on fields that store lookup data, like statusValue or priority. Instead, you should configure a user filter to filter by the list of static values.

Row considerations

For each record type, smart search can search through:

• 1.5 million rows for text fields.
• 500,000 rows for extra long text fields.
• 100,000 rows for document fields.

If a field exceeds this row limit or a document is over 5MB, indexing will fail, which will take smart search offline. To prevent indexing failures, consider configuring a Limited indexing failure tolerance level when enabling smart search.

Supported document formats

Smart search can only search through documents of type:

• DOCX
• HTML
• MD
• PDF (not image-based or scanned)
• TXT

Smart search indexes the text content within your documents. While the system can process images containing readable text (such as scanned-text documents), it cannot index purely graphic or illustrative images that lack extractable text.

Documents containing only non-textual images cause indexing to fail, which takes smart search offline. Ensure your documents contain searchable text to avoid this error.

Supported document sizes

Each document must meet these size requirements:

• Total file size: 10MB or less.
• Single page size: 5MB or less.

Files that exceed these size limits cause indexing to fail, which will take smart search offline. To prevent smart search from going offline, consider configuring a Limited indexing failure tolerance level when enabling smart search.

Search result considerations

Smart search can return up to 100 results per search. To refine the results, you can:

• Filter by a similarity score above a set threshold.
• Sort by similarity score in descending order.
• Use a batch size or filters to narrow the search results.

The appropriate similarity score threshold or filter value is dependent on your use case and data. While Appian provides a starting point on what similarity scores may return the most relevant results, you will need to test and review what scores return strong or weak matches in your own applications.

Indexing failure tolerance level

You can configure an indexing failure tolerance level when enabling smart search or anytime thereafter to ensure that smart search remains online, even when a small number of records fail to index. Allowing some indexing failures prevents minor data issues from disabling smart search functionality across the entire record type.

You can choose one of the following tolerance levels from the Smart Search page of the record type:

Performance considerations

Search time increases as the number of rows and searchable fields grows. Generally, Text fields index the fastest. Extra Long Text fields index more slowly than Text fields, and Document fields may take the longest.

To ensure optimal performance:

• Only enable smart search for fields that are strictly necessary for your users.
• If you do not need to search the full content of an Extra Long Text or Document field, consider extracting key metadata into a Text field and indexing that instead.

Smart search indexing will almost always take more time to complete than the sync itself, regardless of the field type. This is expected behavior because indexing involves additional processing beyond what a standard sync requires. The difference is most pronounced during first-time indexing. Subsequent indexing is typically much faster because only rows updated since the last index are processed.

For documents near the size limits, indexing time can increase further. We recommend validating indexing time in your environment if you plan to index a large number of large documents.

Enable smart search

To enable fields for smart search in a synced record type:

1. In the record type, go to Smart Search.
2. Choose one of the following:
   • Up to three text fields.
   • One extra long text field and one text field.
   • One document field and one text field.

   Tip:  To enable smart search for documents, the document ID field must be of type Document. If your record type uses a number (integer) field to store the document identifier instead, you'll need to move documents to a document management record type.

3. Select an Indexing Failure Tolerance Level for the record type:
   • Strict: (Default) No tolerance for failures.
   • Limited: Some tolerance for failures.
   • Relaxed: Full tolerance for failures.
4. Click SAVE.

Saving starts a full sync. When the sync completes, the indexing process for smart search will start. You can check the indexing status on the Smart Search tab on the record type's Sync History page or on the Record Sync Status tab in the Monitor view.

Implement smart search

After you enable fields for smart search, you can implement it throughout your applications.

You can implement smart search:

• In the default search box in a read-only grid.
• In a custom search box in a read-only grid.
• In filters in an a!queryRecordType() expression.
• In a data fabric chatbot.

This section explains how to configure these functions to use smart search.

In the default search box in a read-only grid

When a read-only grid uses a record type as a source, a search box will display by default at the top of the grid. This search box defaults to using only keyword search, but you can configure it to use both keyword search and smart search with minimal setup.

To use both keyword search and smart search in the default search available in a read-only grid:

1. In the read-only grid, ensure the showSearchBox parameter is set to true.

2. Set the similarityScoreThreshold parameter to a value less than 1. See general guidelines for using similarity scores to help pick your threshold.

   Tip:  Consider adding a column to illustrate the quality of the similarity score in this grid. This will give you quick insight into why certain results are higher in the grid than others and allow users to sort by match quality. See this recipe for an example.

3. Add columns to the grid to display fields with smart search enabled.

Then, whenever users enter search terms in the search box:

• The grid will use smart search for any text, extra long text, or document fields that have smart search enabled, and use keyword search for all other field types.
• Smart search and keyword search are always applied to these fields, even if the columns are hidden in the grid.
• Keyword search matches return a similarityScore of exactly 1.
• Smart search matches return a similarityScore less than 1.

In a custom search box in a read-only grid

If you want to customize the search box and its styling, you won't be able to use the default search box in a read-only grid. Instead, you can use a separate component, like a text field, to capture the user input and then use that input to filter the grid's data. In these cases, you can still implement smart search on your custom search box.

In a custom search box, you cannot implement smart search using the similarityScoreThreshold parameter that the default search box relies on. Instead, a custom search box requires you to configure an a!queryFilter() function with the "search" operator in a!recordData().

To implement smart search in a custom search box in a read-only grid:

1. Create a local variable to store your search terms. For example, local!searchTerm.
2. In the text field used for the search bar, set the value and saveInto parameters to local!searchTerm. This will store the entered terms in the local variable.
3. In the read-only grid, use a!recordData() to configure the source.
4. In the filters parameter of a!recordData(), configure an a!queryFilter() function as follows:
   • For field, enter one of the record fields you enabled smart search on.
   • For operator, enter "search".
   • For value, enter local!searchTerm.
   • For showWhen, use the a!isNotNullOrEmpty() function to only apply this filter when a search term is entered. For example, a!isNotNullOrEmpty(local!searchTerm).

5. (Optional) Add additional a!queryFilter() functions using the "search" operator for each record field you want to search on.

   Tip:  You can choose how to evaluate multiple filters using a!queryLogicalExpression(). This allows you to combine all a!queryFilter() functions using an "AND" or "OR" operator.

6. (Optional) Add another a!queryFilter function and filter by similarity score to limit results to stronger matches.

   For example:

   1
   2
   3
   4
   5
    a!queryFilter(
        field: recordType!Case.searchResults.allSearchFields.similarityScore,
        operator: ">=", 
        value: 0.5
    )
   

See the Configure a Custom Search Box in a Read-Only Grid: Smart Search recipe for a full expression implementing smart search in a custom search box.

In filters in an a!queryRecordType() expression

In addition to grids, you can also leverage smart search in the a!queryRecordType() function. This allows you to query the data you need, even if you're not sure of the exact terms, and display those items in interfaces or pass them to process models.

For example, you may want to query cases that are similar to the one you're looking at, and then display those related cases as additional information on your interface. You could create a query to return those similar cases and display them in a card component.

You can sort, filter, and display the results of your searches using the searchResults property of the record type. For example, you can filter a query using recordType!Case.searchResults.allSearchFields.similarityScore so it only returns results that have a certain similarity score or higher.

To use smart search in an a!queryRecordType() expression:

1. Use a!queryRecordType() to build your query.
2. In the filters parameter of a!queryRecordType(), configure an a!queryFilter() function as follows:
   • For field, enter one of the record fields you enabled smart search on.
   • For operator, enter "search".

   • For value, enter the value you want to search on.

     Tip:  If you want to dynamically filter the query based on a selection or text input, create a local variable to store the value and pass it to this filters parameter.

3. (Optional) Add additional a!queryFilter() functions using the "search" operator for each record field you want to search on.

   Tip:  You can choose how to evaluate multiple filters using a!queryLogicalExpression(). This allows you to combine all a!queryFilter() functions using an "AND" or "OR" operator.

4. (Optional) Add another a!queryFilter function and filter by similarity score to limit results to stronger matches.

   For example:

   1
   2
   3
   4
   5
    a!queryFilter(
        field: recordType!Case.searchResults.allSearchFields.similarityScore,
        operator: ">=", 
        value: 0.5
    )
   

In a data fabric chatbot

To use smart search in a data fabric chatbot, reference a record type with smart search enabled in the recordTypes parameter. This will automatically apply smart search to the data fabric chat queries.

Search results

When you run a query or search on a read-only grid that uses smart search, the returned DataSubset includes the following fields:

• similarityScore
• sourceContent
• searchMatches (available for extra long text and document fields only)

You can access similarity scores and related search information directly from the DataSubset results.

For example, suppose you run the following query:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
a!queryRecordType(
  recordType: recordType!Case,
  fields: {
    recordType!Case.fields.id,
    recordType!Case.fields.title,
    recordType!Case.fields.description,

    /* Returns similarityScore, sourceContent, and sourceField for the field with the strongest match */
    /* In this example, it could be title or description, because we are searching across those two fields */
    recordType!Case.searchResults.allSearchFields,
   
   /* Returns similarityScore and sourceContent for the title field */
    recordType!Case.searchResults.title,
   
   /* Returns similarityScore and sourceContent for the description field */
    recordType!Case.searchResults.description, 
  },
  /* Search both the title and description fields */
  filters: a!queryLogicalExpression(
    operator: "OR",
    filters: {
      a!queryFilter(
      field: recordType!Case.fields.title,
      operator: "search",
      value: "software update"
    ),
      a!queryFilter(
      field: recordType!Case.fields.description,
      operator: "search",
      value: "software update"
    ),
   }
  ),
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    batchSize: 2,
    sort: {
      /* Sort by primary similarity score so the best matches are returned at the top */
      a!sortInfo(
        field: recordType!Case.searchResults.allSearchFields.similarityScore,
        ascending: false
      )
    }
  )
).data

In this example, the query returns the following.

In each case, you'll see the following search results:

• searchResults.allSearchFields shows the primary similarity score (similarityScore), where the match occurred (sourceContent), and which field contained the primary similarity score (sourceField).
• searchResults.title and searchResults.description show the similarity score and source content returned for those specific fields. We included these two specific search results in our example to illustrate why the primary similarity score points to the description field.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
data (List of Case)
Case
  id: 2
  title: "Software update failure"
  description: "Attempted to update ERP system, update failed midway. Rolled back to previous version."
  searchResults.allSearchFields
    similarityScore: 0.5
    sourceContent: "Attempted to update ERP system, update failed midway. Rolled back to previous version."
    sourceField: description
  searchResults.title
    similarityScore: null
    sourceContent: null
  searchResults.description
    similarityScore: 0.5
    sourceContent: "Attempted to update ERP system, update failed midway. Rolled back to previous version."
Case
  id: 44
  title: "Application errors after update"
  description: "Shortly after applying a software update to one of our internal web apps, users started reporting errors when accessing certain features. We've had to roll back the update but some issues persist."
  searchResults.allSearchFields
    similarityScore: 0.4
    sourceContent: "Shortly after applying a software update to one of our internal web apps, users started reporting errors when accessing certain features. We've had to roll back the update but some issues persist."
  searchResults.title
    similarityScore: null
    sourceContent: null
  searchResults.description
    similarityScore: 0.4
    sourceContent: "Shortly after applying a software update to one of our internal web apps, users started reporting errors when accessing certain features. We've had to roll back the update but some issues persist."

Sort and filter by similarity score

In most cases, you'll want to sort search results in descending order by similarity score to see the most relevant results first.

As a best practice, Appian recommends sorting or filtering by the primary similarity score. This ensures that you're always sorting or filtering by the strongest search results, even if you're searching across different fields.

To filter or sort by the primary similarity score, use the following reference in your a!sortInfo() or a!queryFilter() function:

recordType!<Record Type Name>.searchResults.allSearchFields.similarityScore

For example, the expression below shows how you'd sort a query to the Case record type so only the most relevant cases are returned at the top. If there is no similarity score present, then sort by the case's created on date.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
a!localVariables(
  local!searchTerm: "software update",
  a!queryRecordType(
    recordType: recordType!SSTA Case,
    fields: {
      recordType!SSTA Case.fields.id,
      /* The title and description fields have smart search enabled  */
      recordType!SSTA Case.fields.title,
      recordType!SSTA Case.fields.description
    },
    filters: /* Use smart search to filter the query  */
    a!queryFilter(
      field: recordType!SSTA Case.fields.title,
      operator: "search",
      value: local!searchTerm
    ),
    pagingInfo: a!pagingInfo(
      startIndex: 1,
      batchSize: 100,
      /* Sort by primary similarity score since you're  */
      /* search across fields with smart search enabled */
      sort: {
        a!sortInfo(
          field: recordType!SSTA Case.searchResults.allSearchFields.similarityScore,
          ascending: false
        ),
        /* Add a secondary sort when there is no similarity score */
        a!sortInfo(
          field: recordType!SSTA Case.fields.createdOn,
          ascending: false
        )
      }
    )
  ).data
)

Sort results by document page number

When searching through PDF documents, you may want to sort results by the page where the match was found instead of by the similarity score. When you search on a Document field that contains PDFs, you can access the specific page where a match was found from the searchMatches property.

To sort by the page number where search results were found:

1. In an expression rule, create an a!queryRecordType() expression that searches for a phrase in a document.

   Tip:  It's best to only search through a single document to avoid returning a large set of data that could potentially impact performance.

   For example:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   42
   43
   44
   45
   46
   47
    a!queryRecordType(
      recordType: recordType!Case Knowledge Repository,
      fields: {
        recordType!Case Knowledge Repository.fields.id,
           
        /* Smart search is enabled on the documentSummary and document field */
        recordType!Case Knowledge Repository.fields.documentSummary,
        recordType!Case Knowledge Repository.fields.document,
           
        /* Returns the primary similarity score and source content*/
        recordType!Case Knowledge Repository.searchResults.allSearchFields,
           
        /*Returns multiple search matches for the document field being searched on */
        recordType!Case Knowledge Repository.searchResults.document.searchMatches
      },
      filters: a!queryLogicalExpression(
        operator: "AND",
        filters: {
          /* Filter by a single document */
          a!queryFilter(
            field: recordType!Case Knowledge Repository.fields.id,
            operator: "=",
            value: 1
          ),
          /* Search the contents of the document */
          a!queryFilter(
            field: recordType!Case Knowledge Repository.fields.document,
            operator: "search",
            value: "conversational experiences via agents"
          ),
          /* Only return matches with a primary similarity score of 0.5 or higher */
          /* Results will come back sorted by similarity score in descending order */
          a!queryFilter(
            field: recordType!Case Knowledge Repository.searchResults.document.similarityScore,
            operator: ">=",
            value: 0.5,
          )
        },
        ignoreFiltersWithEmptyValues: true
      ),
      pagingInfo: a!pagingInfo(
        startIndex: 1,
        batchSize: 100
      )
    /* Index directly into search matches */ 
    ).data
    [recordType!Case Knowledge Repository.searchResults.document.searchMatches]
   

2. Create another expression rule to call and structure the results of the original query using the todatasubset() function.

   In this example, we'll retrieve the matches found in the document and limit the results to only three. We'll also sort the results by page number in ascending order.

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
    a!localVariables(
      /* Call the original expression rule */
      local!myData: rule!searchSingleDocument,
      local!dataSubset: todatasubset(
        /* Extract the list of maps from the searchMatches field */
        arrayToPage: local!myData[1],
        pagingConfiguration: a!pagingInfo(
          startIndex: 1,
        /* Only return 3 items from the search */
          batchSize: 3,
          /* Sort by page in ascending order */
          sort: a!sortInfo(
            field: "page", 
            ascending: true
          )
        )
      ),
      local!dataSubset
    )
   

Then, when you run the expression above, the results would show all search results ordered based on the page number:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
"DataSubset": [
  {
    "startIndex": 1,
    "batchSize": 3,
    "totalCount": 5,
    "sort": {
      "field": "page",
      "ascending": true
    },
    "data": [
      {
        "similarityScore": 0.8,
        "sourceContent": "The insured is responsible for duty of care.",
        "page": 2
      },
      {
        "similarityScore": 0.9,
        "sourceContent": "Premium payments are due annually on the policy anniversary.",
        "page": 5
      },
      {
        "similarityScore": 0.6,
        "sourceContent": "In the event of loss, notify the insurer within 30 days.",
        "page": 15
      }
    ]
  }
]

Query search results for individual fields

You can use a!recordType.<Record Type Name>.searchResults.<Field Name> to reference the search results for a specific field with smart search enabled. For example, you can query the top similarity score and the list of match scores for a document field without pulling in the search results from other fields like description or title.

By retrieving only the necessary data, you can improve performance and reduce the amount of data transferred and processed, which is crucial when dealing with large documents and potentially numerous search matches. However, you should not use this reference when passing search results to a process model. Instead, we recommend using a!recordType.<Record Type Name>.searchResults.allSearchFields when passing results to a process model.

Consider only querying the search results from a specific field when displaying information in an interface using a!queryRecordType(). For example, the following query returns each contract's ID, its highest document similarity score, and a list of similarity scores for all individual matches:

1
2
3
4
5
6
7
8
9
10
11
a!queryRecordType(
  recordType: recordType!Contract,
  fields: {
    recordType!Contract.id,
    recordType!Contract.searchResults.document.searchMatches.similarityScore
  },
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    batchSize: 10
  )
).data

Learn more about this property.

Passing search results to a process model

When passing search results to a process model, you should pass all of the search results, not just one or two fields.

If you extract specific fields from the search results, like recordType!Case.searchResults.allSearchFields.similarityScore, before passing them to the process model, the associated search metadata may be lost. This can lead to issues where the process model cannot access the necessary search context, potentially causing errors downstream.

To maintain the complete search context:

• Pass the entire record, including the searchResults, to the process model. For example, recordType!Case.searchResults.allSearchFields.
• Avoid indexing into specific search result fields before passing the data.
• In the process model, store the incoming data into a variable for the appropriate record type.

By passing the full record, the process model retains access to all search-related metadata, such as similarity scores and source content, ensuring that any logic dependent on this information functions correctly.

Monitoring smart search status

Whether or not smart search is online depends on the index status of your record type. If the record type fails to index, then smart search will be offline.

You can monitor the index status of your record type in two places:

• The Smart Search tab on the record type's Sync History page. This view shows the indexing history specific to smart search for that record type.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

• The Record Sync Status tab in the Monitoring View. This view shows the latest smart search status for each record type that has smart search enabled.
  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Use these views to confirm that smart search is successfully indexing your data. If the status is Failed or Running, smart search results may be outdated or missing.

Indexing statuses

The following table lists all possible indexing statuses for smart search. Each status describes the current state of the index and, when applicable, the action you can take to resolve issues.

Indexing retries and failures

Smart search automatically retries failed indexing jobs to improve reliability, similar to how record types retry failed syncs.

If a row fails to index due to an error, Appian retries to index up to three times. If all retries fail, Appian marks the rows as failed and sets the overall indexing status to Failed.

Troubleshooting

This section helps you identify and resolve smart search issues.

Smart search is offline

If smart search is unavailable for your record type, it may be due to indexing failures.

Indexing may fail when:

• A sync failed on the record type. To resolve indexing issues and bring smart search back online in this instance, you must resolve the sync issue and sync the record type.

• The record type tries to index too many rows for a specific field type. Appian will index all rows of data in the field; however, if the field exceeds the row limit, indexing will fail.

• A document is an unsupported format or exceeds the maximum file size.

Whether smart search is still online depends on the indexing failure tolerance threshold configured in the record type. This setting determines how many rows can fail to index before smart search is taken offline.

If the number of rows that failed to index exceeds this threshold, smart search is taken offline for that record type and the indexing status will be set as Failed. A notification email will be sent to alert you and system administrators of the failure.

Unable to find relevant search results

While smart search understands the meaning and intent behind your terms, it's important to remember that smart search is not a conversational chatbot like data fabric chatbot. Smart search will only find the most relevant information based on your search terms.

You may not see the results you're expecting if you use smart search like a chatbot.

When using smart search:

• DO describe what you're looking for using a couple of key terms or a full phrase. The more context you provide, the more relevant your results will be.
  • Example: Search for terms like Paid time off, PTO, or vacation policy.
• DON'T attempt to have a conversation with the search.
  • Example: A search for What is our PTO policy? may find the Paid Time Off Guidelines document. However, it will not reply, "You have 15 days of PTO."

Feedback