About Smart Search

What is smart search?

Smart search provides a tailored search experience for the Text fields, Extra Long Text fields, and documents in your synced record types. You can search documents regardless of whether they're stored in Appian or in an external system.

When you implement smart search, you can choose between two search capabilities:

• Semantic search: An AI-powered search that finds results based on meaning and context. For example, a search for "electricity outage" can return records containing "power failure" or "blackout," even if the exact keywords do not match.

• Lexical search: A full-text search that matches specific words and characters in your data. For example, a search for "Jane Smith" in a document will return all pages that reference this exact name.

Selecting the search capability that best fits your business data ensures users find information more intuitively.

Why use smart search?

When you use smart search in your applications, you can:

• Discover related data and patterns: Use semantic search to find information based on meaning and context. This allows you to identify similar records or recognize trends—such as duplicate cases—even when using different terminology.
• Perform exact matches: Use lexical search to find exact words, characters, or phrases within documents and Extra Long Text fields. This expands your search capabilities to include data types not supported by traditional keyword search.
• Search documents in any storage location: Find information in documents managed by a record type, whether they're stored in Appian or in an external system.
• Sort results by relevance: Use similarity scores to sort results by match quality. This ensures that the most relevant matches appear at the top of the list so users can find information faster.
• Integrate with other AI features to accelerate workflows: Combine smart search with AI skills to use the search results to automate decision making.

Choosing the right search method

Before implementing smart search, ensure it's the correct tool for your requirements. While smart search provides advanced search capabilities for synced record types, it is separate from traditional keyword search and filters.

Search options

To choose the right search method, you must understand the different options:

• Smart search: An advanced search capability for synced record types. Smart search includes semantic search (searching by meaning) and lexical search (searching for exact phrases). It is the only way to perform a semantic or full-text search on documents and Extra Long Text fields.

• Keyword search: The default search behavior for record lists and read-only grids. Keyword search looks for exact or partial character matches in Text, Number, User, or Group fields. It cannot search the contents of documents or Extra Long Text fields.

Compare search methods

Use this table to select the best method for your data and user requirements:

Combine search and filter methods

You can combine different search types and filters to create a more powerful user experience:

• Semantic search and filters: Find a concept within a specific subset (for example, cases with "connectivity issues" that are In Progress).
• Lexical search and filters: Find a specific phrase within a filtered set of documents (for example, "Section A-12" in documents tagged as Legal).
• Semantic search and lexical search: In a query, combine semantic and lexical search to return results that match by meaning or by exact phrase.
• Keyword search and semantic search: In a read-only grid, the default search box can return rows matching an exact or partial term and rows matching a related concept.
• Keyword search and lexical search: In a read-only grid, the default search box can return rows matching an exact or partial term and rows matching an exact phrase in documents or Extra Long Text fields.

Requirements for smart search

Review these requirements before you enable smart search.

Region availability

Smart search is available for environments in select regions. If your Appian environment isn't in a supported region, you can elect to send your data to a supported region. This doesn't change your environment's region. Contact Appian Support to learn more.

Supported record types

You can enable smart search on any synced record type.

Smart search is not supported for unsynced record types and legacy record types.

Supported fields

You can enable smart search on the following fields:

• Text, including fields that are part of a composite primary key.
• Extra Long Text, including fields that are part of a composite primary key.
• Document (for documents stored in Appian)
• Primary Key (only for documents stored in an external system)

Each record type supports a limited combination of fields that can be enabled for smart search:

• Up to three Text fields.
• One Text field and one Extra Long Text field.
• One Text field and one Document field (for Appian-stored documents).
• One Text field and one primary key field (for externally-stored documents).

Row considerations

For each record type, smart search can search through:

• 10 million rows of Text.
• 3.75 million rows of Extra Long Text.
• 325,000 documents (stored in Appian or in external systems).

Fields that exceed these limits cause indexing to fail, which will take smart search offline. To prevent smart search from going offline, consider configuring a Limited indexing failure tolerance level when enabling smart search.

Supported document formats

Smart search can only search through documents of type:

• DOCX
• HTML
• MD
• PDF (not password protected)
• TXT

Smart search indexes the text content within your documents. While the system can process images containing readable text (such as scanned-text documents), it cannot index purely graphic or illustrative images that lack extractable text.

Documents containing only non-textual images cause indexing to fail, which takes smart search offline. Ensure your documents contain searchable text to avoid this error.

Supported document sizes

Depending on the document type, a document can have a maximum file size or page length:

• PDF (embedded-text) and DOCX files: 200MB.
• PDF (scanned-text): 500 scanned pages.
• TXT, MD, and HTML files: 10MB.

Files that exceed these size limits cause indexing to fail, which will take smart search offline. To prevent smart search from going offline, consider configuring a Limited indexing failure tolerance level when enabling smart search.

Where to use smart search

You can use smart search in the following locations to allow you and your users to find relevant information:

The search box in read-only grids

Configure the search box in a read-only grid so users can search by meaning and/or exact phrases.

You can implement smart search in a read-only grid using:

• The default search box available in the grid. This allows you to implement both smart search and keyword search.
• A custom search box configured with other components. This allows you to implement smart search in a custom-built search box.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

a!queryRecordType() expressions

Use a!smartSearch() in the filters parameter of a!queryRecordType() to apply smart search logic to query results.

For example, query all cases and use semantic search to identify similar cases based on case names and descriptions. Then, display the similar cases at the top in a card layout so users can explore similar cases and their resolution.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Process models

Use smart search to support automated decision-making. Combine it with AI skills to provide next-step suggestions without user input.

Data fabric chatbots

Reference a record type with smart search enabled in the recordTypes parameter of a data fabric chatbot. This allows the chatbot to leverage semantic search so it can understand intent.

How smart search works

When you enable smart search on fields in your record type, Appian creates a dedicated search index. This index enables semantic and lexical search capabilities across your record data.

Once enabled, you can implement smart search in your applications so users can describe what they're looking for or search for an exact phrase.

Each search result includes information you can use to refine the user experience:

• Similarity score: Ranks how relevant the result is to the search query.
• Source content: Displays the specific text where the match was found.
• Page number: Identifies the specific page of a document where a match was found (returned for PDF files only).

You can use these details in your application using the searchResults property to filter, sort, or display the results.

Indexing

Indexing is the process of preparing synced data for searching. While indexing is separate from syncing data, indexing does depend on successful syncs.

Indexing automatically occurs on a record type when:

• You deploy the record type: Appian indexes all existing rows.
• You update smart search fields: Appian indexes all existing rows.
• A sync completes: Whether it's a full sync, incremental sync, or smart service sync, Appian only indexes data that changed since the last sync, which typically makes indexing faster and minimizes AI action consumption.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

If a sync fails and the record data is unavailable, smart search becomes unavailable because there is no data to index. To maintain search availability during indexing issues, configure an indexing failure tolerance level.

Similarity score

A similarity score is a number that indicates how closely a record matches your search. Every search result receives a similarity score, with higher scores representing more relevant or exact matches.

The similarity score range depends on the search capability you implement:

• Semantic search: Returns a score between 0 and 1. This score is not a percentage match. Instead, it's a mathematical value based on the semantic meaning of the content, not a keyword-by-keyword comparison.
• Lexical search score: Returns a score greater than 1. This score is unbounded, meaning it has no set upper limit, and the highest scores will vary across search terms.

Interpreting similarity score

For lexical search, the specific similarity score is less important than its rank relative to other results. A higher value simply indicates a stronger character-based match.

For semantic search, scores are relative to your specific data set. What qualifies as a strong or weak match depends entirely on your data and how you're using the semantic search.

Use the following ranges as a general starting point for testing your semantic search results:

• 1.0: Keyword match.
• 0.8 or greater: Strong match; the content is highly relevant to the intent.
• 0.6 – 0.79: Moderate match; the content is related but may contain tangential information.
• 0.4 – 0.59: Fair match; the content shares some conceptual overlap.
• 0.39 or below: Weak match; the results are likely irrelevant.
• null: No match found.

Example 1: Case record type

You enable semantic search on the description field of the Case record type, which contains narrative, sentence-based text about a support case.

When searching for customer reporting an incorrect invoice charge, you might find that a score of 0.8 accurately identifies direct complaints. However, a score of 0.2 might return irrelevant topics, such as a customer requesting an invoice copy. In this scenario, you would set your "Strong Match" threshold at 0.8.

Example 2: Order record type

You enable semantic search on the description field in the Order record type, which contains less structured text about customer orders, often mixed with product features and notes.

When searching for raincoats in black, you may find that relevant results return lower similarity scores. In this data set, you might determine that a "Strong Match" actually begins at 0.3 because the search successfully identifies relevant colors and items despite the unstructured text.

Reference similarity scores

You can reference the primary similarity score or the similarity score for a specific field in your expressions. When searching across multiple fields, use the primary similarity score to ensure you are sorting or filtering by the strongest match.

To reference similarity scores, use the following properties of the record type:

• Primary similarity score
  • Property: recordType!<Record Type Name>.searchResults.allSearchFields.similarityScore
  • Recommended when: Searching across multiple fields or finding the strongest match in a row.
• Individual field score
  • Property: recordType!<Record Type Name>.searchResults.<Field Name>.similarityScore
  • Recommended when: Searching a single field or displaying match quality for a specific column in a grid.

Primary similarity score

When you search across multiple fields, a row may have several search matches. The highest similarity score found in that row is called the primary similarity score. Use the primary similarity score to filter and sort data by the strongest search matches, regardless of which field contained the match.

For example, if smart search is enabled on the title, description, and name fields, a semantic search for "electrical outage" in a read-only grid evaluates all three fields. To ensure users see the most relevant results first, sort the grid by the primary similarity score.

Use similarity scores

Use the similarity score to sort or filter search results so it's easier to focus on the most relevant or exact information.

You can use similarity scores for:

• Sorting: Sort queries and grids by the primary similarity score in descending order. This ensures the most relevant results appear first and is effective for both semantic and lexical searches.
  
  

• Filtering: Use the similarityScoreThreshold parameter in a!smartSearch() or read-only grids to set a minimum match quality for semantic search results.

  Only filter by score for semantic search. Because lexical scores are unbounded and do not have a standard scale, filtering them by a specific similarity score may lead to inconsistent results.
  
  

• Visual Indicators: Display semantic match quality (for example, "Strong," "Moderate," "Fair") using tags or icons. See an example displaying semantic match quality in a grid.

  Only use visual indicators for semantic search. Lexical search scores cannot be reliably mapped to a consistent qualitative scale.

Source content

The source content is the piece of text where the search term was found. You can reference the source content from the searchResults property.

Use the source content to:

• Validate match quality: Compare the source content against the similarity score during development to determine what constitutes a "Strong" versus "Weak" semantic match for your data. For example, if a semantic search for "server down" returns a score of 0.8 and the source content is "server went down last night," you can confirm the match is highly relevant.
• Extend workflows with AI skills: Pass the source content to an AI skill. The AI can then summarize the relevant paragraphs or format the text for display in your interface.

You cannot use the source content to filter or sort a query. Use the similarity score for these operations instead.

Search matches

Smart search can return multiple matches within a single Extra Long Text field or document. By default, smart search includes up to 10 matches per field or document, sorted by similarity score in descending order.

Search matches appear in the results of a query in the searchMatches property, and include the following:

• Similarity score.
• Source content.
• Page: The PDF page number where the match occurred (available for documents only).

Example: Searching for property damage in the Document field of a Rental Contract record type could return the following results:

1
2
3
4
5
6
7
8
9
10
11
12
- data List of Contracts
  - Contracts
    - id: 2 (Number (Integer))
    - docIds: 22411 - Primary Lease Agreement.pdf (Document)
    - searchResults Map
      - docIds Map
        - searchMatches List of Map - 3 items
          - Map
            - page: 12 (Number (Integer))
            - similarityScore: 0.8923412 (Number (Decimal Floating))
            - sourceContent: "Landlord shall handle structural repairs, but Tenant is liable for all non-structural upkeep and minor fixes."
          - ...

Learn how to filter or sort by page or source content.

Feedback