Configure Smart Search

What is smart search?

Smart search provides a tailored search experience for the Text fields, Extra Long Text fields, and documents in your synced record types. You can search documents regardless of whether they're stored in Appian or in an external system.

When you implement smart search, you can choose between two search capabilities:

• Semantic search: An AI-powered search that finds results based on meaning and context. For example, a search for "electricity outage" can return records containing "power failure" or "blackout," even if the exact keywords do not match.

• Lexical search: A full-text search that matches specific words and characters in your data. For example, a search for "Jane Smith" in a document will return all pages that reference this exact name.

Selecting the search capability that best fits your business data ensures users find information more intuitively.

Why use smart search?

When you use smart search in your applications, you can:

• Discover related data and patterns: Use semantic search to find information based on meaning and context. This allows you to identify similar records or recognize trends—such as duplicate cases—even when using different terminology.
• Perform exact matches: Use lexical search to find exact words, characters, or phrases within documents and Extra Long Text fields. This expands your search capabilities to include data types not supported by traditional keyword search.
• Search documents in any storage location: Find information in documents managed by a record type, whether they're stored in Appian or in an external system.
• Sort results by relevance: Use similarity scores to sort results by match quality. This ensures that the most relevant matches appear at the top of the list so users can find information faster.
• Integrate with other AI features to accelerate workflows: Combine smart search with AI skills to use the search results to automate decision making.

Choosing the right search method

Before implementing smart search, ensure it's the correct tool for your requirements. While smart search provides advanced search capabilities for synced record types, it is separate from traditional keyword search and filters.

Search options

To choose the right search method, you must understand the different options:

• Smart search: An advanced search capability for synced record types. Smart search includes semantic search (searching by meaning) and lexical search (searching for exact phrases). It is the only way to perform a semantic or full-text search on documents and Extra Long Text fields.

• Keyword search: The default search behavior for record lists and read-only grids. Keyword search looks for exact or partial character matches in Text, Number, User, or Group fields. It cannot search the contents of documents or Extra Long Text fields.

Compare search methods

Use this table to select the best method for your data and user requirements:

Combine search and filter methods

You can combine different search types and filters to create a more powerful user experience:

• Semantic search and filters: Find a concept within a specific subset (for example, cases with "connectivity issues" that are In Progress).
• Lexical search and filters: Find a specific phrase within a filtered set of documents (for example, "Section A-12" in documents tagged as Legal).
• Semantic search and lexical search: In a query, apply a semantic search and a lexical search to the same field to return results that match by meaning or by exact phrase.
• Keyword search and semantic search: In a read-only grid, the default search box can return rows matching an exact or partial term and rows matching a related concept.
• Keyword search and lexical search: In a read-only grid, the default search box can return rows matching an exact or partial term and rows matching an exact phrase in documents or Extra Long Text fields.

Where to use smart search

You can use smart search in the following locations to allow you and your users to find relevant information:

• The search box in read-only grids: Configure a read-only grid so users can leverage both keyword search and smart search in the search box. You can implement this search behavior using:
  • The default search available in the grid.
  • A custom search box configured with other components.
  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

  Note:  Smart search is not available for record lists. To use smart search in a grid, configure a read-only grid with smart search.

• a!queryRecordType() expressions: Use the "semantic search" or "lexical search" operators in a!queryFilter() to apply smart search logic to query results.

  For example, query all cases and use semantic search to identify similar cases based on case names and descriptions. Then, display the similar cases at the top in a card layout so users can explore similar cases and their resolution.

  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

• Process models. Use smart search to support automated decision-making. Combine it with AI skills to provide next-step suggestions without user input.
  
  

• Data fabric chatbots. Reference a record type with smart search enabled in the recordTypes parameter of a data fabric chatbot. This allows the chatbot to leverage semantic search so it can understand intent.

How smart search works

When you enable smart search on fields in your record type, Appian creates a dedicated search index. This index enables semantic and lexical search capabilities across your record data.

Once enabled, you can implement smart search in your applications so users can describe what they're looking for or search for an exact phrase.

Each search result includes information you can use to refine the user experience:

• Similarity score: Ranks how relevant the result is to the search query.
• Source content: Displays the specific text where the match was found.
• Page number: Identifies the specific page of a document where a match was found (returned for PDF files only).

You can use these details in your application using the searchResults property to filter, sort, or display the results.

Indexing

Indexing is the process of preparing synced data for searching. While indexing is separate from syncing data, indexing does depend on successful syncs.

Indexing automatically occurs on a record type when:

• You deploy the record type: Appian indexes all existing rows.
• You update smart search fields: Appian indexes all existing rows.
• A sync completes: Whether it's a full sync, incremental sync, or smart service sync, Appian only indexes data that changed since the last sync, which typically makes indexing faster and minimizes AI action consumption.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

If a sync fails and the record data is unavailable, smart search becomes unavailable because there is no data to index. To maintain search availability during indexing issues, configure an indexing failure tolerance level.

Similarity score

A similarity score is a number that indicates how closely a record matches your search. Every search result receives a similarity score, with higher scores representing more relevant or exact matches.

The similarity score range depends on the search capability you implement:

• Semantic search: Returns a score between 0 and 1. This score is not a percentage match. Instead, it's a mathematical value based on the semantic meaning of the content, not a keyword-by-keyword comparison.
• Lexical search score: Returns a score greater than 1. This score is unbounded, meaning it has no set upper limit, and the highest scores will vary across search terms.

Interpreting similarity score

For lexical search, the specific similarity score is less important than its rank relative to other results. A higher value simply indicates a stronger character-based match.

For semantic search, scores are relative to your specific data set. What qualifies as a strong or weak match depends entirely on your data and how you're using the semantic search.

Use the following ranges as a general starting point for testing your semantic search results:

• 1.0: Keyword match.
• 0.8 or greater: Strong match; the content is highly relevant to the intent.
• 0.6 – 0.79: Moderate match; the content is related but may contain tangential information.
• 0.4 – 0.59: Fair match; the content shares some conceptual overlap.
• 0.39 or below: Weak match; the results are likely irrelevant.
• null: No match found.

Example 1: Case record type

You enable semantic search on the description field of the Case record type, which contains narrative, sentence-based text about a support case.

When searching for customer reporting an incorrect invoice charge, you might find that a score of 0.8 accurately identifies direct complaints. However, a score of 0.2 might return irrelevant topics, such as a customer requesting an invoice copy. In this scenario, you would set your "Strong Match" threshold at 0.8.

Example 2: Order record type

You enable semantic search on the description field in the Order record type, which contains less structured text about customer orders, often mixed with product features and notes.

When searching for raincoats in black, you may find that relevant results return lower similarity scores. In this data set, you might determine that a "Strong Match" actually begins at 0.3 because the search successfully identifies relevant colors and items despite the unstructured text.

Reference similarity scores

You can reference the primary similarity score or the similarity score for a specific field in your expressions. When searching across multiple fields, use the primary similarity score to ensure you are sorting or filtering by the strongest match.

To reference similarity scores, use the following properties of the record type:

• Primary similarity score
  • Property: recordType!<Record Type Name>.searchResults.allSearchFields.similarityScore
  • Recommended when: Searching across multiple fields or finding the strongest match in a row.
• Individual field score
  • Property: recordType!<Record Type Name>.searchResults.<Field Name>.similarityScore
  • Recommended when: Searching a single field or displaying match quality for a specific column in a grid.

Primary similarity score

When you search across multiple fields, a row may have several search matches. The highest similarity score found in that row is called the primary similarity score. Use the primary similarity score to filter and sort data by the strongest search matches, regardless of which field contained the match.

For example, if smart search is enabled on the title, description, and name fields, a semantic search for "electrical outage" in a read-only grid evaluates all three fields. To ensure users see the most relevant results first, sort the grid by the primary similarity score.

Use similarity scores

Use the similarity score to sort or filter search results so it's easier to focus on the most relevant or exact information.

You can use similarity scores for:

• Sorting: Sort queries and grids by the primary similarity score in descending order. This ensures the most relevant results appear first and is effective for both semantic and lexical searches.
  
  

• Filtering: Use a similarity score threshold in a grid or use a!queryFilter() to filter results by a similarity score.

  Only filter by score for semantic search. Because lexical scores are unbounded and do not have a standard scale, filtering them by a specific similarity score may lead to inconsistent results.
  
  

• Visual Indicators: Display semantic match quality (e.g., "Strong," "Moderate," "Fair") using tags or icons. See an example displaying semantic match quality in a grid.

  Only use visual indicators for semantic search. Lexical search scores cannot be reliably mapped to a consistent qualitative scale.

Source content

The source content is the piece of text where the search term was found. You can reference the source content from the searchResults property.

Use the source content to:

• Validate match quality: Compare the source content against the similarity score during development to determine what constitutes a "Strong" versus "Weak" semantic match for your data. For example, if a semantic search for "server down" returns a score of 0.8 and the source content is "server went down last night," you can confirm the match is highly relevant.
• Extend workflows with AI skills: Pass the source content to an AI skill. The AI can then summarize the relevant paragraphs or format the text for display in your interface.

You cannot use the source content to filter or sort a query. Use the similarity score for these operations instead.

Search matches

Smart search can return multiple matches within a single Extra Long Text field or document. By default, smart search includes up to 10 matches per field or document, sorted by similarity score in descending order.

Search matches appear in the results of a query in the searchMatches property, and include the following:

• Similarity score.
• Source content.
• Page: The PDF page number where the match occurred (available for documents only).

Example: Searching for property damage in the Document field of a Rental Contract record type could return the following results:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
- data List of Contracts - 2 items
  - Contracts
    - id: 2 (Number (Integer))
    - docIds: 22411 - Primary Lease Agreement.pdf (Document)
    - searchResults Map
      - docIds Map
        - searchMatches List of Map - 3 items
          - Map
            - page: 12 (Number (Integer))
            - similarityScore: 0.8923412 (Number (Decimal))
            - sourceContent: "Landlord shall handle structural repairs, but Tenant is liable for all non-structural upkeep and minor fixes."
          - Map
            - page: 45 (Number (Integer))
            - similarityScore: 0.8451229 (Number (Decimal))
            - sourceContent: "Any damage resulting from Tenant's negligence shall be repaired promptly at the Tenant's own expense."
          - Map
            - page: 3 (Number (Integer))
            - similarityScore: 0.7654321 (Number (Decimal))
            - sourceContent: "The Premises must be returned to the Lessor in the same state of repair as existed at the Commencement Date."
  - Contracts
    - id: 3 (Number (Integer))
    - docIds: 22412 - Commercial Rental Contract.pdf (Document)
    - searchResults Map
      - docIds Map
        - searchMatches List of Map - 3 items
          - Map
            - page: 5 (Number (Integer))
            - similarityScore: 0.9123847 (Number (Decimal))
            - sourceContent: Occupant shall indemnify the Owner against all loss or damage to the Property arising from the Occupant’s use thereof."
          - Map
            - page: 14 (Number (Integer))
            - similarityScore: 0.8876543 (Number (Decimal))
            - sourceContent: "In the event of a casualty loss, the Lessee must provide written notice of the damage to the Lessor within 24 hours."
          - Map
            - page: 22 (Number (Integer))
            - similarityScore: 0.8234110 (Number (Decimal))
            - sourceContent: "The cost of restoring the unit to its original state, excluding reasonable deterioration, is the sole responsibility of the Tenant."

Learn how to filter or sort by page or source content.

Requirements for smart search

Review these requirements before you enable smart search.

Region availability

Smart search is available for environments in select regions. If your Appian environment isn't in a supported region, you can elect to send your data to a supported region. This doesn't change your environment's region. Contact Appian Support to learn more.

Supported record types

You can only enable smart search on synced record types with a single primary key field.

Supported fields

You can enable smart search on the following fields:

• Text
• Extra Long Text
• Document (for documents stored in Appian)
• Primary Key (only for documents stored in an external system)

Each record type supports a limited combination of fields that can be enabled for smart search:

• Up to three Text fields.
• One Text field and one Extra Long Text field.
• One Text field and one Document field (for Appian-stored documents).
• One Text field and one primary key field (for externally-stored documents).

Do not enable smart search on fields that store lookup data, like statusValue or priority. Instead, create a user filter to filter by the list of static values.

Row considerations

For each record type, smart search can search through:

• 10 million rows of Text.
• 3.75 million rows of Extra Long Text.
• 325,000 documents (stored in Appian or in external systems).

Fields that exceed these limits cause indexing to fail, which will take smart search offline. To prevent smart search from going offline, consider configuring a Limited indexing failure tolerance level when enabling smart search.

Supported document formats

Smart search can only search through documents of type:

• DOCX
• HTML
• MD
• PDF (not password protected)
• TXT

Smart search indexes the text content within your documents. While the system can process images containing readable text (such as scanned-text documents), it cannot index purely graphic or illustrative images that lack extractable text.

Documents containing only non-textual images cause indexing to fail, which takes smart search offline. Ensure your documents contain searchable text to avoid this error.

Supported document sizes

Depending on the document type, a document can have a maximum file size or page length:

• PDF (embedded-text) and DOCX files: 200MB.
• PDF (scanned-text): 500 scanned pages.
• TXT, MD, and HTML files: 10MB.

Files that exceed these size limits cause indexing to fail, which will take smart search offline. To prevent smart search from going offline, consider configuring a Limited indexing failure tolerance level when enabling smart search.

Usage considerations for smart search

Use these settings and best practices to optimize search behavior and maintain system reliability.

Refining search results

Smart search can return up to 5,000 results per search. To refine and organize the search results, you can:

• Filter or batch your data: Use a!queryFilter() or a!pagingInfo() to narrow the search results.
• Sort by similarity score: Sort results in descending order to show the most relevant or exact matches first. This is effective for both semantic and lexical searches.

• Filter by similarity score (Semantic search only): Set a similarity score threshold in grids or use a!queryFilter() to filter by a specific score.

  Note:  You must test semantic search with your data to determine which similarity scores represent a strong match. Because lexical scores are unbounded, do not filter them by similarity scores.

Indexing failure tolerance level

Configure an indexing failure tolerance level to keep smart search online even if some records fail to index. This prevents minor data issues, such as an oversized document or a single index error, from disabling smart search functionality for the entire record type.

Select a tolerance level on the Smart Search page of the record type:

Performance best practices

Search time increases as the number of rows and searchable fields grows. Generally, Text fields index the fastest. Extra Long Text fields index more slowly than Text fields, and documents may take the longest to index.

To ensure optimal performance:

• Only enable smart search for fields that are strictly necessary for your users.
• If you do not need to search the full content of an Extra Long Text field or document, consider extracting key metadata into a Text field and indexing that instead.

Smart search indexing will almost always take more time to complete than the sync itself, regardless of the field type. This is expected behavior because indexing involves additional processing beyond what a standard sync requires. The difference is most pronounced during first-time indexing. Subsequent indexing is typically much faster because only rows updated since the last index are processed.

For documents near the size limits, indexing time can increase further. We recommend validating indexing time in your environment if you plan to index a large number of large documents.

Enable smart search

Enabling smart search on a record fields provides you with access to both semantic and lexical search capabilities. You choose which specific search type to use when implementing smart search in your queries or interfaces.

To enable fields for smart search:

1. In the record type, go to Smart Search.
2. Choose a supported field combination.
3. Select an Indexing Failure Tolerance Level for the record type:
   • Strict: (Default) No tolerance for failures.
   • Limited: Some tolerance for failures.
   • Relaxed: Full tolerance for failures.
4. Click SAVE.

Saving the record type starts a full sync. Once the sync completes, the indexing process for smart search begins automatically. You can monitor the indexing status from the record type or the Monitor view.

Implement smart search

After you enable fields for smart search, you can implement semantic search and lexical search throughout your applications.

How you implement search depends on the component or function you are using. While smart search can be implemented using the a!queryFilter() function, some components—like the read-only grid—use specific parameters to enable smart search.

Use the following table to determine which search types are supported in each location:

Use smart search in the default grid search box

Supported search capability: Semantic search or lexical search

When you use a record type as the source for a read-only grid, you can configure the grid's default search box to apply semantic search or lexical search in addition to keyword search. This approach uses the smartSearchType, similarityScoreThreshold, and searchFields parameters on the read-only grid component.

For implementation steps, see Use smart search in the default grid search box on the Read-Only Grid page.

Use smart search in a custom search box

Supported search capability: Semantic search and lexical search

If you want to customize the grid's search box and its styling, you must build a custom search box. This approach involves using a separate input component—typically a text component—to capture user search terms and apply them as filters to your record data.

When you implement a custom search box:

• No grid search parameters: You cannot use the smartSearchType, similarityScoreThreshold, or searchFields parameters. Instead, use a!queryFilter() to define which fields to search on.
• Search scope: Smart search is only applied on the fields you define in a!queryFilter() expressions. To search on multiple fields, combine filters in an a!queryLogicalExpression() function.
• No keyword search: A custom search box does not automatically include keyword search like the default search box. Only the smart search type you specify in the a!queryFilter() operator is applied.
• No Extra Long Text fields: You cannot filter by Extra Long Text fields in a!recordData() within a grid. To search Extra Long Text fields in a grid, use the default search box with the searchFields parameter.

To implement smart search in a custom grid search box:

1. Create a local variable to store your search terms. For example, local!searchTerm.
2. Add a text component to your interface to serve as the search box. Set the value and saveInto parameters to your local variable (local!searchTerm).
3. In the read-only grid, use a!recordData() to configure the source using your record type with smart search enabled.
4. In the read-only grid, set the showSearchBox parameter to false.
5. In the filters parameter of a!recordData(), configure a!queryLogicalExpression() to determine how the searches are applied.
6. For each field you want to search on, add an a!queryFilter() with the following properties:
   • For field, reference a field enabled for smart search.
   • For operator, enter "semantic search" or "lexical search".
   • For value, reference your local variable (local!searchTerm).
   • For showWhen, use a!isNotNullOrEmpty() to only apply this filter when a search term is entered. For example, a!isNotNullOrEmpty(local!searchTerm).

   Tip:  You can apply both semantic search and lexical search on the same field by creating two a!queryFilter() expressions that reference the same field but use different operators.

7. (Optional) If you are using semantic search, add an additional a!queryFilter() outside of your "OR" expression to filter by similarity score. Do not filter by similarity score for lexical search.

See the Configure a Custom Search Box in a Read-Only Grid: Smart Search recipe for a full expression implementing smart search in a custom search box.

Use smart search in an a!queryRecordType() expression

Supported search capability: Semantic search and lexical search

You can implement smart search in a!queryRecordType() to retrieve data for an interface or process model.

When using a!queryRecordType():

• Use semantic search, lexical search, or both: Apply one or both search types in the same query by using the "semantic search" and "lexical search" operators in separate a!queryFilter() expressions. You can apply both operators to the same field or to different fields.
• Search multiple fields: Apply smart search to as many enabled fields as needed within a single query.
• Granular control: Sort, filter, and display results using the searchResults property of the record type.

To use smart search in a!queryRecordType():

1. Use a!queryRecordType() to build a query to your record type.
2. In the filters parameter, add a!queryFilter():
   • For field, reference a field enabled for smart search.
   • For operator, enter "semantic search" to search by meaning or "lexical search" to search for exact words or phrases.
   • For value, enter the text or local variable you want to search.
3. To search across multiple fields, wrap the filters in a!queryLogicalExpression() and specify how the filers are evaluated. In most cases, you'll set the operator to "OR" to return results with matches in any of the specified fields.
4. (Semantic search only) Add another a!queryFilter() outside of your "OR" expression to filter by similarity score. This restricts results to records that meet or exceed a specific match quality.
5. In the pagingInfo parameter, add a!sortInfo() to sort by similarity score in descending order so the most relevant results are first.

See the following examples to learn how to:

• Implement semantic search in a query.
• Implement lexical search in a query.
• Implement both semantic and lexical search in a query.
• Search across Appian-stored documents.
• Search across externally-stored documents.

Use smart search in a!relatedRecordData()

Supported search capability: Semantic search and lexical search

When querying one-to-many related record data, you can apply smart search in a!relatedRecordData() to filter the related records returned for each base record. This is useful when you want to find the most relevant related records based on meaning or exact phrasing.

For implementation steps and field reference requirements, see Using smart search in a!relatedRecordData.

Use semantic search in a data fabric chatbot

Supported search capability: Semantic search

To use semantic search in a data fabric chatbot, reference a record type with smart search enabled in the recordTypes parameter. This will automatically apply semantic search to the data fabric chat queries.

Search results

Smart search returns up to 5,000 results per search. You can view, sort, and filter these results using the searchResults property of the record type.

Query search results for multiple fields

When searching across multiple fields, use a!recordType.<Record Type Name>.searchResults.allSearchFields to retrieve information for the strongest match in the record type.

You can display all of the search result properties or index into the one of the following properties:

• sourceField: The field where the search term with the highest similarity score was found.
• similarityScore: The primary similarity score.
• sourceContent: The piece of text where the search term was found.
• page: (PDF documents only) The page number where the match was found.

Example: Perform a lexical search on both the title and description field. In the query results, display the case id, title, description, and the primary search result properties.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
a!queryRecordType(
  recordType: recordType!Case,
  fields: {
    recordType!Case.fields.id,
    recordType!Case.fields.title,
    recordType!Case.fields.description,

    /* Returns similarityScore, sourceContent, and sourceField for the field with the strongest match */
    /* In this example, it could be title or description, because we are searching across those two fields */
    recordType!Case.searchResults.allSearchFields, 
  },
  /* Return results from either the title and description field */
  filters: a!queryLogicalExpression(
    operator: "OR",
    filters: {
      a!queryFilter(
      field: recordType!Case.fields.title,
      operator: "lexical search",
      value: "ACME Auto"
    ),
      a!queryFilter(
      field: recordType!Case.fields.description,
      operator: "lexical search",
      value: "Acme Auto"
    ),
   }
  ),
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    batchSize: 2,
    sort: {
      /* Sort by primary similarity score so the best matches are returned at the top */
      a!sortInfo(
        field: recordType!Case.searchResults.allSearchFields.similarityScore,
        ascending: false
      ),
      a!sortInfo(
        field: recordType!Case.fields.createdOn,
        ascending: false
      )
    }
  )
).data

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
data (List of Case)
Case
  id: 102
  title: "ACME Auto - Settlement Agreement"
  description: "Finalized documentation for the ACME Auto settlement regarding case #8829-B."
  searchResults.allSearchFields
    similarityScore: 1.678
    sourceContent: "ACME Auto - Settlement Agreement"
    sourceField: title
Case
  id: 405
  title: "Quarterly Fleet Review"
  description: "This report summarizes the performance of the ACME Auto fleet for Q3, noting a decrease in maintenance costs."
  searchResults.allSearchFields
    similarityScore: 1.1200
    sourceContent: "...performance of the ACME Auto fleet for Q3..."
    sourceField: description

Query search results for individual fields

When searching on a single field, use a!recordType.<Record Type Name>.searchResults.<Field Name> to reference its specific search results.

From this property, you can index into the following information:

• similarityScore: A number indicating how closely the content matches the search query.
• sourceContent: The piece of text where the search term was found.
• searchMatches: (Extra Long Text and Documents only) A list of maps that show where your search matched the field content. Each match includes a similarityScore, sourceContent, page.

Example: Perform a semantic search on the document field (type Document) in the Contract record type. In the query output, display the primary key field, document name, and the similarity score and text where the match was found for each result.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
a!queryRecordType(
  recordType: recordType!Contract,
  fields: {
    recordType!Contract.fields.id,
    recordType!Contract.relationship.documentProperties.fields.name,
    /* Indexing into searchMatches to get the specific piece of */ 
    /* text where the term was found and the similarity score */
    recordType!Contract.searchResults.document.searchMatches.sourceContent,
    recordType!Contract.searchResults.document.searchMatches.similarityScore
  },
  filters: a!queryFilter(
    field: recordType!Contract.fields.document,
    operator: "semantic search",
    value: "damage policy"
  )
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    batchSize: 2,
    /* Sort by the similarity score so the best matches are returned at the top */
    sort: {
      a!sortInfo(
       field:recordType!Contract.searchResults.document.searchMatches.similarityScore
       ascending: false
    ),
     a!sortInfo(
        field: recordType!Contract.fields.createdOn,
        ascending: false
     )
    }
  )
).data

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
data (List of Contracts)
Contract
  id: 23
  documentProperties:
    id: 1867
    name: Primary Lease Agreement.pdf
  searchResults.document
    sourceContent: "Landlord shall handle structural repairs, but Tenant is liable for all non-structural upkeep and minor fixes."
    similarityScore: 0.89
Contract
  id: 44
  documentProperties:
    id: 2041
    name: Commercial Rental Contract.pdf
  searchResults.document
    sourceContent: "Any damage resulting from Tenant's negligence shall be repaired promptly at the Tenant's own expense."
    similarityScore: 0.72

Sort results by similarity score

In most scenarios, you should sort search results in descending order by similarity score to see the most relevant or exact matches first.

Depending on your search configuration, sort by:

• Primary similarity score when searching across multiple fields: a!recordType.<Record Type Name>.searchResults.allSearchFields.similarityScore

• Similarity score for the field when searching by a single field.

  • For Text fields: a!recordType.<Record Type Name>.searchResults.<Field Name>.similarityScore
  • For Extra Long Text fields or documents: a!recordType.<Record Type Name>.searchResults.<Field Name>.searchMatches.similarityScore

Example: Sort by similarity score so the most relevant cases return first. If there is no similarity score present (meaning no search has occurred), then sort by the case's created on date.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
a!localVariables(
  local!searchTerm: "software update",
  a!queryRecordType(
    recordType: recordType!Case,
    fields: {
      recordType!Case.fields.id,
      /* The title and description fields have smart search enabled  */
      recordType!Case.fields.title,
      recordType!Case.fields.description
    },
    filters: a!queryLogicalExpression(
      operator: "OR",
      filters: {
        a!queryFilter(
         field: recordType!Case.fields.title,
         operator: "lexical search",
         value: local!searchTerm
        ),
        a!queryFilter(
         field: recordType!Case.fields.description,
         operator: "lexical search",
         value: local!searchTerm
        ),
      }
    ),
    pagingInfo: a!pagingInfo(
      startIndex: 1,
      batchSize: 100,
      /* Sort by primary similarity score since you're  */
      /* search across fields with smart search enabled */
      sort: {
        a!sortInfo(
          field: recordType!Case.searchResults.allSearchFields.similarityScore,
          ascending: false
        ),
        /* Add a secondary sort when there is no similarity score */
        a!sortInfo(
          field: recordType!Case.fields.createdOn,
          ascending: false
        )
      }
    )
  ).data
)

Filter results by similarity score

When using semantic search, you can filter results by similarity score to exclude low-quality matches.

Example: Return employees whose Title or Job Description semantically matches "software developer" with a score of 0.4 or higher. Because multiple fields are being searched, the similarity score filter is applied using an outer "AND" operator to ensure the threshold applies to the entire search result.

Expression:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
a!queryRecordType(
  recordType: recordType!AT Employee,
  fields: {
    recordType!AT Employee.fields.id,
    recordType!AT Employee.fields.firstName,
    recordType!AT Employee.fields.lastName,
    recordType!AT Employee.fields.title,
    recordType!AT Employee.fields.jobDescription
  },
  filters: a!queryLogicalExpression(
    operator: "AND",
    /* Nested OR logic to search multiple fields */
    logicalExpressions: a!queryLogicalExpression(
      operator: "OR",
      filters: {
        a!queryFilter(
          field: recordType!AT Employee.fields.title,
          operator: "semantic search",
          value: "software developer"
        ),
        a!queryFilter(
          field: recordType!AT Employee.fields.firstName,
          operator: "semantic search",
          value: "software developer"
        ),

      }
    ),
    /* Outer AND logic to apply the similarity score threshold */
    filters: a!queryFilter(
      field: recordType!AT Employee.searchResults.allSearchFields.similarityScore,
      operator: ">=",
      value: 0.4
    ),

  ),
  pagingInfo: a!pagingInfo(startIndex: 1, batchSize: 5)
)

Sort results by document page number

When searching through PDF documents, you may want to sort results by the page where the match was found instead of their similarity score. When you search on a Document field that contains PDFs, you can access the specific page where a match was found from the searchMatches property.

To sort by the page number where search results were found:

1. In an expression rule, create an a!queryRecordType() expression that searches for a phrase in a document.

   Example:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   42
   43
   44
   45
   46
   47
    a!queryRecordType(
      recordType: recordType!Case Knowledge Repository,
      fields: {
        recordType!Case Knowledge Repository.fields.id,
           
        /* Smart search is enabled on the documentSummary and document field */
        recordType!Case Knowledge Repository.fields.documentSummary,
        recordType!Case Knowledge Repository.fields.document,
           
        /* Returns the primary similarity score and source content*/
        recordType!Case Knowledge Repository.searchResults.allSearchFields,
           
        /*Returns multiple search matches for the document field being searched on */
        recordType!Case Knowledge Repository.searchResults.document.searchMatches
      },
      filters: a!queryLogicalExpression(
        operator: "AND",
        filters: {
          /* Filter by a single document */
          a!queryFilter(
            field: recordType!Case Knowledge Repository.fields.id,
            operator: "=",
            value: 1
          ),
          /* Search the contents of the document */
          a!queryFilter(
            field: recordType!Case Knowledge Repository.fields.document,
            operator: "semantic search",
            value: "conversational experiences via agents"
          ),
          /* Only return matches with a primary similarity score of 0.5 or higher */
          /* Results will come back sorted by similarity score in descending order */
          a!queryFilter(
            field: recordType!Case Knowledge Repository.searchResults.document.similarityScore,
            operator: ">=",
            value: 0.5,
          )
        },
        ignoreFiltersWithEmptyValues: true
      ),
      pagingInfo: a!pagingInfo(
        startIndex: 1,
        batchSize: 100
      )
    /* Index directly into search matches */ 
    ).data
    [recordType!Case Knowledge Repository.searchResults.document.searchMatches]
   

2. Create another expression rule to call and structure the results of the original query using the todatasubset() function.

   Example: Retrieve the matches found in the document and limit the results to only three. Sort the results by page number in ascending order.

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
    a!localVariables(
      /* Call the original expression rule */
      local!myData: rule!searchSingleDocument,
      local!dataSubset: todatasubset(
        /* Extract the list of maps from the searchMatches field */
        arrayToPage: local!myData[1],
        pagingConfiguration: a!pagingInfo(
          startIndex: 1,
        /* Only return 3 items from the search */
          batchSize: 3,
          /* Sort by page in ascending order */
          sort: a!sortInfo(
            field: "page", 
            ascending: true
          )
        )
      ),
      local!dataSubset
    )
   

Sample output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
"DataSubset": [
  {
    "startIndex": 1,
    "batchSize": 3,
    "totalCount": 5,
    "sort": {
      "field": "page",
      "ascending": true
    },
    "data": [
      {
        "similarityScore": 0.8,
        "sourceContent": "The insured is responsible for duty of care.",
        "page": 2
      },
      {
        "similarityScore": 0.9,
        "sourceContent": "Premium payments are due annually on the policy anniversary.",
        "page": 5
      },
      {
        "similarityScore": 0.6,
        "sourceContent": "In the event of loss, notify the insurer within 30 days.",
        "page": 15
      }
    ]
  }
]

Pass search results to a process model

When your business logic requires a process model to act on search data (such as an automated "Review Related Cases" task), you should pass all of the search results. Do not index into specific properties (like similarityScore or sourceContent).

If you index into a specific search result before the process starts, the process model cannot access the necessary search context. This makes it impossible for the process to know which field triggered the match or what the original sourceContent was.

To maintain the complete search context in a process model:

• Pass the entire record, including the searchResults, to the process model. For example, recordType!Case.searchResults.allSearchFields.
• Avoid indexing into specific search result fields before passing the data.
• In the process model, store the incoming data into a variable for the appropriate record type.

By passing the full record, the process model retains access to all search-related metadata, such as similarity scores and source content, ensuring that any logic dependent on this information functions correctly.

Monitoring smart search status

Whether or not smart search is online depends on the index status of your record type and the indexing failure tolerance level.

You can monitor the index status of your record type in two places:

• The Smart Search tab on the record type's Sync History page. This view shows the indexing history specific to smart search for that record type.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

• The Record Sync Status tab in the Monitor view. This view shows the latest smart search status for each record type that has smart search enabled.
  

  Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Use these views to confirm that smart search is successfully indexing your data. If the status is Failed or Running, smart search results may be outdated or missing.

Indexing statuses

The following table lists all possible indexing statuses for smart search. Each status describes the current state of the index and, when applicable, the action you can take to resolve issues.

Indexing retries and failures

Smart search automatically retries failed indexing jobs to improve reliability, similar to how record types retry failed syncs.

If a row fails to index due to an error, Appian retries to index up to three times. If all retries fail, Appian marks the rows as failed and sets the overall indexing status to Failed.

Troubleshooting smart search

This section helps you identify and resolve smart search issues.

Smart search is offline

If smart search is unavailable for your record type, it may be due to indexing failures.

Whether smart search is still online during an indexing failure depends on the indexing failure tolerance threshold configured in the record type. This setting determines how many rows can fail to index before smart search is taken offline.

If the threshold is exceeded:

• Smart search is taken offline for that record type and the indexing status is Failed.
• A notification email will be sent to alert you and system administrators of the failure.

Common causes for indexing failures

Indexing may fail when:

• A sync failed on the record type. To resolve indexing issues and bring smart search back online in this instance, you must resolve the sync issue and sync the record type.

• The record type tries to index too many rows for a specific field type. Appian will index all rows of data in the field; however, if the field exceeds the row limit, indexing will fail.

• A document is an unsupported format or exceeds the maximum file size.

• For documents stored in external systems, the download integration fails to retrieve a document. Verify that the integration is configured correctly and that the external system is accessible.

Unable to find relevant search results

While semantic search understands the meaning and intent behind your terms, it's important to remember that semantic search is not a conversational chatbot like data fabric chatbot. Semantic search will only find the most relevant information based on your search terms.

You may not see the results you're expecting if you use semantic search like a chatbot.

When using semantic search:

• DO describe what you're looking for using a couple of key terms or a full phrase. The more context you provide, the more relevant your results will be.
  • Example: Search for terms like Paid time off, PTO, or vacation policy.
• DON'T attempt to have a conversation with the search.
  • Example: A search for What is our PTO policy? may find the Paid Time Off Guidelines document. However, it will not reply, "You have 15 days of PTO."

Feedback