Work with Smart Search Results

This page explains how to query, sort, and filter smart search results using the searchResults property of the record type. It also covers monitoring indexing status and troubleshooting common issues.

Refine smart search results

Smart search can return up to 5,000 results per search. To refine and organize smart search results, you can:

• Narrow or batch your data: Use a!smartSearch() or a!pagingInfo() to narrow the search results.
• Sort by similarity score: Sort results in descending order to show the most relevant or exact matches first. Sorting by similarity score is effective for both semantic and lexical searches.

• Filter by similarity score (semantic search only): Set a similarity score threshold in read-only grids or a!smartSearch() to only return results with a specific score.

  Note:  You must test semantic search with your data to determine which similarity scores represent a strong match. Because lexical scores are unbounded, do not filter them by similarity scores.

Query smart search results

You can view, sort, and filter smart search results using the searchResults property of the record type. You can query search results for a single field or for all fields that were searched on.

Query search results for multiple fields

When searching across multiple fields, use a!recordType.<Record Type Name>.searchResults.allSearchFields to retrieve information for the strongest match in the record type.

When you reference a!recordType.<Record Type Name>.searchResults.allSearchFields, the following properties are returned:

• sourceField: The field where the search term with the highest similarity score was found.
• similarityScore: The primary similarity score.
• sourceContent: The piece of text where the search term was found.
• page: (PDF documents only) The page number where the match was found.

To return only a specific property, index into it directly. For example, to return only the primary similarity score, use a!recordType.<Record Type Name>.searchResults.allSearchFields.similarityScore.

Example: Perform a lexical search on both the title and description field. In the query results, display the case id, title, description, and the primary search result properties.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
a!queryRecordType(
  recordType: recordType!Case,
  fields: {
    recordType!Case.fields.id,
    recordType!Case.fields.title,
    recordType!Case.fields.description,

    /* Returns similarityScore, sourceContent, and sourceField for the field with the strongest match */
    /* In this example, it could be title or description, because we are searching across those two fields */
    recordType!Case.searchResults.allSearchFields, 
  },
  /* Return results from either the title and description field */
  filters: a!queryLogicalExpression(
    operator: "OR",
    filters: {
      a!smartSearch(
      searchField: recordType!Case.fields.title,
      searchType: "lexical search",
      searchTerms: "ACME Auto"
    ),
      a!smartSearch(
      searchField: recordType!Case.fields.description,
      searchType: "lexical search",
      searchTerms: "Acme Auto"
    ),
   }
  ),
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    batchSize: 2,
    sort: {
      /* Sort by primary similarity score so the best matches are returned at the top */
      a!sortInfo(
        field: recordType!Case.searchResults.allSearchFields.similarityScore,
        ascending: false
      ),
      a!sortInfo(
        field: recordType!Case.fields.createdOn,
        ascending: false
      )
    }
  )
).data

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
data (List of Case)
Case
  id: 102
  title: "ACME Auto - Settlement Agreement"
  description: "Finalized documentation for the ACME Auto settlement regarding case #8829-B."
  searchResults.allSearchFields
    similarityScore: 1.678
    sourceContent: "ACME Auto - Settlement Agreement"
    sourceField: title
Case
  id: 405
  title: "Quarterly Fleet Review"
  description: "This report summarizes the performance of the ACME Auto fleet for Q3, noting a decrease in maintenance costs."
  searchResults.allSearchFields
    similarityScore: 1.1200
    sourceContent: "...performance of the ACME Auto fleet for Q3..."
    sourceField: description

Query search results for individual fields

When searching on a single field, use a!recordType.<Record Type Name>.searchResults.<Field Name> to reference its specific search results.

When you use a!recordType.<Record Type Name>.searchResults.<Field Name>, the following properties are returned:

• similarityScore: A number indicating how closely the content matches the search query.
• sourceContent: The piece of text where the search term was found.
• searchMatches: (Extra Long Text and Documents only) A list of maps that show where your search matched the field content. Each match includes a similarityScore, sourceContent, and page.

To return only a specific property, index into it directly. For example, to return only the similarity score, use a!recordType.<Record Type Name>.searchResults.allSearchFields.<Field Name>.similarityScore.

Example: Perform a semantic search on the document field (type Document) in the Contract record type. In the query output, display the primary key field, document name, and the similarity score and text where the match was found for each result.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
a!queryRecordType(
  recordType: recordType!Contract,
  fields: {
    recordType!Contract.fields.id,
    recordType!Contract.relationship.documentProperties.fields.name,
    /* Indexing into searchMatches to get the specific piece of */ 
    /* text where the term was found and the similarity score */
    recordType!Contract.searchResults.document.searchMatches.sourceContent,
    recordType!Contract.searchResults.document.searchMatches.similarityScore
  },
  filters: a!smartSearch(
    searchField: recordType!Contract.fields.document,
    searchType: "semantic search",
    searchTerms: "damage policy"
  )
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    batchSize: 2,
    /* Sort by the similarity score so the best matches are returned at the top */
    sort: {
      a!sortInfo(
       field:recordType!Contract.searchResults.document.searchMatches.similarityScore
       ascending: false
    ),
     a!sortInfo(
        field: recordType!Contract.fields.createdOn,
        ascending: false
     )
    }
  )
).data

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
data (List of Contracts)
Contract
  id: 23
  documentProperties:
    id: 1867
    name: Primary Lease Agreement.pdf
  searchResults.document
    sourceContent: "Landlord shall handle structural repairs, but Tenant is liable for all non-structural upkeep and minor fixes."
    similarityScore: 0.89
Contract
  id: 44
  documentProperties:
    id: 2041
    name: Commercial Rental Contract.pdf
  searchResults.document
    sourceContent: "Any damage resulting from Tenant's negligence shall be repaired promptly at the Tenant's own expense."
    similarityScore: 0.72

Sort smart search results

You can sort smart search results by similarityScore to surface the best matches, or by document page number to present results in reading order. You cannot sort by the sourceContent or sourceField property.

Sort by similarity score

In most scenarios, you should sort smart search results in descending order by similarity score to see the most relevant or exact matches first.

Depending on your search configuration, sort by:

• Primary similarity score: Use when searching across multiple fields. To reference the primary similarity score, enter a!recordType.<Record Type Name>.searchResults.allSearchFields.similarityScore.
• Similarity score for a single field: Use when searching by a single field.
  • For Text fields, enter a!recordType.<Record Type Name>.searchResults.<Field Name>.similarityScore.
  • For Extra Long Text fields or documents, enter a!recordType.<Record Type Name>.searchResults.<Field Name>.searchMatches.similarityScore.

Example: Sort by similarity score so that cases with the most exact search term are returned first. If there is no similarity score present (meaning no search has occurred), then sort by the case's created on date.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
a!localVariables(
  local!searchTerm: "software update",
  a!queryRecordType(
    recordType: recordType!Case,
    fields: {
      recordType!Case.fields.id,
      /* The title and description fields have smart search enabled  */
      recordType!Case.fields.title,
      recordType!Case.fields.description,
      recordType!Case.searchResults.allSearchFields.similarityScore
    },
    filters: a!queryLogicalExpression(
      operator: "OR",
      filters: {
        a!smartSearch(
         searchField: recordType!Case.fields.title,
         searchType: "lexical search",
         searchTerms: local!searchTerm
        ),
        a!smartSearch(
         searchField: recordType!Case.fields.description,
         searchType: "lexical search",
         searchTerms: local!searchTerm
        ),
      }
    ),
    pagingInfo: a!pagingInfo(
      startIndex: 1,
      batchSize: 100,
      /* Sort by primary similarity score since you're  */
      /* search across fields with smart search enabled */
      sort: {
        a!sortInfo(
          field: recordType!Case.searchResults.allSearchFields.similarityScore,
          ascending: false
        ),
        /* Add a secondary sort when there is no similarity score */
        a!sortInfo(
          field: recordType!Case.fields.createdOn,
          ascending: false
        )
      }
    )
  ).data
)

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
data (List of Case)
Case
  id: 78
  title: "Software update failed on production server"
  description: "Attempted a scheduled software update on the production server. The update process stalled at 85% and required a manual rollback."
  searchResults.allSearchFields
    similarityScore: 1.72
Case
  id: 203
  title: "Post software update connectivity loss"
  description: "After applying the latest software update, several workstations lost network connectivity for approximately 20 minutes."
  searchResults.allSearchFields
    similarityScore: 1.45

Sort smart search results by document page number

When searching through PDF documents, you can sort smart search results by page number instead of similarity score. Access the page where a match was found from the searchMatches property, then use todatasubset() to sort the matches by page.

To sort by the page number where search results were found:

1. In an expression rule, create an a!queryRecordType() expression that searches for a phrase in a document.

   Example:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
    a!queryRecordType(
      recordType: recordType!Case Knowledge Repository,
      fields: {
        recordType!Case Knowledge Repository.fields.id,
           
        /* Smart search is enabled on the documentSummary and document field */
        recordType!Case Knowledge Repository.fields.documentSummary,
        recordType!Case Knowledge Repository.fields.document,
           
        /* Returns the primary similarity score and source content*/
        recordType!Case Knowledge Repository.searchResults.allSearchFields,
           
        /*Returns multiple search matches for the document field being searched on */
        recordType!Case Knowledge Repository.searchResults.document.searchMatches
      },
      /* Search the contents of the document */
      filters: a!smartSearch(
        searchField: recordType!Case Knowledge Repository.fields.document,
        searchType: "semantic search",
        searchTerms: "conversational experiences via agents",
        /* Search by a single document */
        identifier: { 1 },
        /* Only return matches with a primary similarity score of 0.5 or higher */
        similarityScoreThreshold: 0.5
      ),
      pagingInfo: a!pagingInfo(
        startIndex: 1,
        batchSize: 100
      )
    /* Index directly into search matches */ 
    ).data
    [recordType!Case Knowledge Repository.searchResults.document.searchMatches]
   

2. Create another expression rule to call and structure the results of the original query using the todatasubset() function.

   Example: Retrieve the matches found in the document and limit the results to only three. Sort the results by page number in ascending order.

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
    a!localVariables(
      /* Call the original expression rule */
      local!myData: rule!searchSingleDocument,
      local!dataSubset: todatasubset(
        /* Extract the list of maps from the searchMatches field */
        arrayToPage: local!myData[1],
        pagingConfiguration: a!pagingInfo(
          startIndex: 1,
        /* Only return 3 items from the search */
          batchSize: 3,
          /* Sort by page in ascending order */
          sort: a!sortInfo(
            field: "page", 
            ascending: true
          )
        )
      ),
      local!dataSubset
    )
   

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
"DataSubset": [
  {
    "startIndex": 1,
    "batchSize": 3,
    "totalCount": 5,
    "sort": {
      "field": "page",
      "ascending": true
    },
    "data": [
      {
        "similarityScore": 0.8,
        "sourceContent": "The insured is responsible for duty of care.",
        "page": 2
      },
      {
        "similarityScore": 0.9,
        "sourceContent": "Premium payments are due annually on the policy anniversary.",
        "page": 5
      },
      {
        "similarityScore": 0.6,
        "sourceContent": "In the event of loss, notify the insurer within 30 days.",
        "page": 15
      }
    ]
  }
]

Filter smart search results by similarity score

When using semantic search, you can filter results by similarity score to exclude low-quality matches.

Example: Return employees whose Title or Job Description semantically matches "software developer" with a score of 0.4 or higher.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
a!queryRecordType(
  recordType: recordType!AT Employee,
  fields: {
    recordType!AT Employee.fields.id,
    recordType!AT Employee.fields.firstName,
    recordType!AT Employee.fields.lastName,
    recordType!AT Employee.fields.title,
    recordType!AT Employee.fields.jobDescription,
    recordType!AT Employee.searchResults.allSearchFields.similarityScore
  },
  filters: a!queryLogicalExpression(
    operator: "OR",
    filters: {
      a!smartSearch(
        searchField: recordType!AT Employee.fields.title,
        searchType: "semantic search",
        searchTerms: "software developer",
        similarityScoreThreshold: 0.4
      ),
      a!smartSearch(
        searchField: recordType!AT Employee.fields.jobDescription,
        searchType: "semantic search",
        searchTerms: "software developer",
        similarityScoreThreshold: 0.4
      )
    }
  ),
  pagingInfo: a!pagingInfo(
    startIndex: 1, 
    batchSize: 5,
    sort: a!sortInfo(
      field: recordType!AT Employee.searchResults.allSearchFields.similarityScore
      ascending: false
    )
  )
)

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
data (List of AT Employee)
AT Employee
  id: 12
  firstName: "Marcus"
  lastName: "Chen"
  title: "Senior Software Engineer"
  jobDescription: "Designs and develops enterprise applications using Java and Python. Leads code reviews and mentors junior developers."
  searchResults.allSearchFields
    similarityScore: 0.82
AT Employee
  id: 87
  firstName: "Priya"
  lastName: "Sharma"
  title: "Full Stack Developer"
  jobDescription: "Builds and maintains web applications using React and Node.js. Collaborates with product teams on feature development."
  searchResults.allSearchFields
    similarityScore: 0.71
AT Employee
  id: 45
  firstName: "David"
  lastName: "Okafor"
  title: "Application Developer II"
  jobDescription: "Develops internal tools and automates business processes. Writes unit tests and maintains CI/CD pipelines."
  searchResults.allSearchFields
    similarityScore: 0.54

Pass smart search results to a process model

When passing smart search results to a process model, pass the entire record including searchResults (for example, recordType!Case.searchResults.allSearchFields). Do not index into specific properties before starting the process, or the process model loses access to the search context.

By passing the full record, the process model retains access to all search-related metadata, such as similarity scores and source content, ensuring that any logic dependent on this information functions correctly.

Monitor smart search indexing status

Whether smart search is online depends on the index status of your record type and the indexing failure tolerance level.

You can monitor the smart search index status of your record type in two places:

• The Smart Search tab on the record type's Sync History page. This view shows the indexing history specific to smart search for that record type.
  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

• The Record Sync Status tab in the Monitor view. This view shows the latest smart search status for each record type that has smart search enabled.
  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Use these views to confirm that smart search is successfully indexing your data. If the smart search status is Failed or Running, smart search results may be outdated or missing.

Smart search indexing statuses

The following table lists all possible smart search indexing statuses. Each status describes the state of the smart search index and, when applicable, the action you can take to resolve issues.

Smart search indexing retries and failures

Smart search automatically retries failed indexing jobs to improve reliability, similar to how record types retry failed syncs.

If a row fails to index due to an error, Appian retries indexing up to three times. If all retries fail, Appian marks the rows as failed and sets the overall smart search indexing status to Failed.

Troubleshoot smart search

This section helps you identify and resolve common smart search issues.

Smart search is offline

If smart search is unavailable for your record type, the issue may be due to indexing failures.

Whether smart search remains online during an indexing failure depends on the indexing failure tolerance threshold configured in the record type. This setting determines how many rows can fail to index before smart search is taken offline.

If the tolerance threshold is exceeded:

• Smart search is taken offline for that record type and the indexing status is Failed.
• A notification email is sent to alert you and system administrators of the failure.

Common causes for smart search indexing failures

Smart search indexing may fail when:

• Sync failure: A sync failed on the record type. To resolve indexing issues and bring smart search back online, resolve the sync issue and sync the record type.
• Row limit exceeded: The record type tries to index too many rows for a specific field type. Appian indexes all rows of data in the field; however, if the field exceeds the row limit, indexing fails.
• Unsupported document format or size: A document is an unsupported format or exceeds the maximum file size.
• External document download failure: For documents stored in external systems, the download integration fails to retrieve a document. Verify that the integration is configured correctly and that the external system is accessible.

Unable to find relevant smart search results

Semantic search understands the meaning and intent behind your terms, but semantic search is not a conversational chatbot like data fabric chatbot. Semantic search finds the most relevant information based on your search terms, not answers to questions.

You may not see the results you expect if you use semantic search like a chatbot.

When using semantic search:

• DO describe what you are looking for using a couple of key terms or a full phrase. The more context you provide, the more relevant your results are.
  • Example: Search for terms like Paid time off, PTO, or vacation policy.
• DON'T attempt to have a conversation with the search.
  • Example: A search for What is our PTO policy? may find the Paid Time Off Guidelines document. However, semantic search does not reply, "You have 15 days of PTO."

Feedback