Working with Documents in AI Agents

Giving your AI agents the ability to read, understand, and act on information from documents is one of the most powerful capabilities you can configure. The document tools available to your AI agent directly affect its accuracy, speed, and AI action consumption.

This page explains how AI agents work with documents, the available document tools, and how to write instructions that guide your AI agent toward the right approach for your use case.

Understanding document tools

AI agents have access to multiple document processing strategies. You must first add a document or document folder as a resource and enable the system document tools your AI agent needs. Once enabled, the AI agent selects among the enabled system document tools and the DocCenter tool based on your instructions:

• Semantic Search on Document and Folder: Finds specific information across multiple documents in a folder by searching for relevant content.
• Read Full Document: Loads the entire document into the AI agent's context for deep reasoning.
• Query Document (Text): Executes dynamic prompts against document content for extraction or summarization.
• Query Visual Document: Evaluates both text and visual elements in scanned or image-heavy documents.
• Query Spreadsheet: Extracts structured data from spreadsheet files, always returning JSON output.
• DocCenter models: Process model tools that invoke DocCenter models for high-accuracy extraction and classification.

For supported file types and size limits for each tool, see the following table:

Tool	File type	Max file size	Page limit	Max image size
Query Document (text mode)	PDF	25 MB	100 pages	N/A
Query Document (text mode)	JPEG, PNG	10 MB	N/A	10k x 10k pixels
Query Document (text mode)	TIFF (single-page)	10 MB	1 page	10k x 10k pixels
Query Document (text mode)	TIFF (multi-page)	10 MB	100 pages	4k x 4k pixels
Query Document (text mode)	DOCX	4.5 MB	N/A	N/A
Query Document (text mode)	HTML, MD, TXT	4.5 MB	N/A	N/A
Query Visual Document	PDF	24 MB	100 pages	N/A
Query Visual Document	JPEG, PNG	3.75 MB	N/A	8k x 8k pixels
Query Visual Document	TIFF (single-page)	3.75 MB	1 page	4k x 4k pixels
Query Visual Document	TIFF (multi-page)	18 MB	20 pages	4k x 4k pixels
Query Spreadsheet	XLS, XLSX, XLSM	10 MB	N/A	N/A
Query Spreadsheet	CSV	10 MB	N/A	N/A
Read Full Document	PDF	25 MB	20 pages	10k x 10k pixels
Semantic Search	PDF, DOC, DOCX, HTML, MD, TXT	50 MB cumulative	N/A	N/A

Query Spreadsheet additional limits:

• Maximum 2 million cells per file.
• Maximum 40,000 rows and 200 columns per sheet.
• Maximum 100 sheets per workbook.

Semantic Search on Document and Folder tool

Semantic Search on Document and Folder finds specific answers within large document collections by intelligently searching for relevant content across multiple documents in a folder.

Best for

Your AI agent is likely to select the Semantic Search on Document and Folder tool for these scenarios:

• Question-answering and fact-checking across large document sets.
• Knowledge retrieval from documents that exceed the context window.
• High-volume or customer-facing agents where cost efficiency matters.

Key characteristics

The Semantic Search on Document and Folder tool has the following characteristics:

• Handles massive scale: thousands of pages across multiple documents.
• Cost-efficient: uses embeddings for search, only passing relevant snippets to the reasoning model.
• Indexes documents once: vectorizes and indexes documents at runtime, allowing reuse without re-reading.
• Fast and precise: delivers targeted answers without processing irrelevant content.

Supported document types

Semantic Search on Document and Folder supports the following text-based document formats:

• DOC (structured and unstructured)
• DOCX
• HTML
• MD
• PDF (not image-based or scanned)
• TXT

Limitations

The Semantic Search on Document and Folder tool has the following limitations:

• Scanned PDFs and image-based documents are not supported and will not return results. Use the Query Visual Document tool for scanned or image-heavy documents.
• Documents are indexed at runtime on first use, which may slow initial runs.
• Combined limit of 75 documents and 50 MB total size per agent run.
• Only searches direct child documents in folders; nested folders require separate tools.

Example use case

You're building a corporate helpdesk AI agent to answer employee questions based on 5,000 pages of internal documentation including HR policies, IT guides, and benefits information.

When an employee asks "What is the process for international travel approval?", your AI agent uses the Semantic Search on Document and Folder tool to query the entire knowledge base and retrieves the three most relevant paragraphs instantly.

Why not other tools?

The Read Full Document tool can't handle this volume. The Query Document (Text) tool isn't designed for open-ended question-answering across multiple documents.

Read Full Document tool

The Read Full Document tool loads the entire text of a document so your AI agent can understand it completely.

Your AI agent is likely to select the Read Full Document tool for these scenarios:

• Analysis that requires understanding how different parts of a document connect.
• Tasks that need a full understanding of tone, argument, or story.
• High-value analysis where the cost is worth the benefit.

The Read Full Document tool has the following characteristics:

• Complete view: Your AI agent understands how different sections relate to each other.
• Deep understanding: Supports complex analysis and comparison tasks.
• Document limits: Up to 20 pages for documents with images, 50 pages for text-only documents.
• Higher cost: Loads the entire document, which uses more AI actions.

How your AI agent selects this tool

Your AI agent is more likely to use the Read Full Document tool when it needs to compare sections, understand document structure, or perform analysis that requires the complete picture. To encourage this behavior, tell your AI agent to read the entire document in your Instructions.

Example use case

You're building a resume screening AI agent to review candidate resumes against your screening policy.

Your AI agent uses the Read Full Document tool to read both the candidate's resume and the screening policy. It can then evaluate whether the candidate meets all qualification criteria by considering the relationships between experience, education, and requirements.

Why not other tools?

The Semantic Search on Document and Folder tool might miss important evaluation criteria. The Query Document (Text) tool would require multiple steps to capture all the details in the resume and screening criteria.

Query Document (Text) tool

The Query Document (Text) tool asks specific questions about a document to extract, summarize, or classify information.

Your AI agent is likely to select the Query Document (Text) tool for these scenarios:

• Pulling data from documents with different formats.
• Summarization and classification tasks.
• Workflows where flexibility and speed matter more than perfect accuracy.

Key characteristics

The Query Document (Text) tool has the following characteristics:

• Flexible questions: your AI agent creates questions based on what it needs to find.
• Adapts easily: works with different document formats without setup.
• Efficient: returns clean, organized output instead of raw text.
• No setup required: easy to get started.
• Document limits: up to 100 pages for text, 20 pages for documents with images.

Example use case

You're building a sales order processing AI agent to handle PDF purchase orders from various customers and enter them into your CRM.

When a purchase order arrives, your AI agent uses the Query Document (Text) tool to ask "Extract the PO Number, Customer Name, Item SKUs, Quantities, and Total Amount into a JSON object." Your AI agent receives clean JSON and proceeds to update the CRM.

Why not other tools?

The Read Full Document tool would require your AI agent to search through all the text itself, which is less efficient. DocCenter might be more than you need if purchase order formats are fairly consistent and you don't require maximum accuracy.

Query Visual Document tool

The Query Visual Document tool is a multimodal document analysis tool that evaluates both text and visual elements in a document, including layout, hierarchy, grouping, and flow.

Best for

Your AI agent is likely to select the Query Visual Document tool for these scenarios:

• Scanned or image-heavy documents where the physical position of data defines its meaning.
• Forms with checkboxes, signature lines, or mixed-media layouts.
• Org charts, diagrams, and documents with connecting lines or arrows.
• Complex tables where row and column alignment conveys relationships.

Key characteristics

The Query Visual Document tool has the following characteristics:

• Visual understanding: Interprets layout, hierarchy, and spatial relationships between elements—not just text content.
• Multimodal analysis: Reads checkboxes, arrows, connecting lines, and other non-text elements.
• Scanned document support: Processes scanned documents and image-based PDFs that text-only tools cannot read.
• Flexible questions: Your AI agent creates prompts based on what it needs to find in the visual layout.

Limitations

The Query Visual Document tool has the following limitations:

• Higher AI action consumption compared to text-only tools due to multimodal processing.
• Image resolution affects accuracy—low-resolution scans may produce less reliable results.
• Does not support DOC, DOCX, HTML, or other text-only formats. Use the Query Document (Text) tool for those formats.

Example use case

You're building a compliance AI agent to analyze scanned corporate ownership charts. The charts contain hierarchical boxes connected by lines indicating ownership percentages.

When the AI agent receives a scanned ownership chart PDF, it uses the Query Visual Document tool to interpret the boxes, connecting lines, and percentage labels. The tool understands that a line from "Parent Corp" to "Subsidiary A" labeled "72%" represents a 72% ownership stake, even though this information is conveyed visually rather than as structured text.

Why not other tools?

The Query Document (Text) tool only reads text content. It cannot interpret visual relationships like connecting lines, checkbox states, or spatial hierarchies. The Read Full Document tool loads text but misses layout meaning. The Query Visual Document tool is the right fit when the physical arrangement of elements on the page carries meaning.

Query Spreadsheet tool

The Query Spreadsheet tool is a specialized extraction tool for structured data in spreadsheet files. It always returns output in JSON format, making it ideal for downstream processing.

Best for

Your AI agent is likely to select the Query Spreadsheet tool for these scenarios:

• Processing, searching, or extracting rows, columns, or cell values from spreadsheet files.
• Converting spreadsheet data into structured JSON for use in downstream tools or processes.
• Analyzing financial data, inventory lists, or any tabular information stored in Excel or CSV format.
• Querying specific sheets within large multi-sheet workbooks.

Key characteristics

The Query Spreadsheet tool has the following characteristics:

• Structured JSON output: Always returns data in JSON format for reliable downstream processing.
• Multi-sheet support: Handles workbooks with up to 100 sheets, querying specific sheets as needed.
• Large file handling: Supports files up to 10 MB with up to 2 million cells.
• Flexible queries: Your AI agent can ask targeted questions about specific rows, columns, or data patterns.

Limitations

The Query Spreadsheet tool has the following limitations:

• Does not support PDF, DOC, DOCX, or image-based formats. Use the Query Document (Text) tool or Query Visual Document tool for those formats.
• Formulas are evaluated before extraction—the tool reads cell values, not formulas.
• Extremely large workbooks approaching the 2 million cell limit may have longer processing times.

Example use case

You're building a financial reporting AI agent that processes monthly XLSM workbooks from multiple departments. Each workbook contains 15 sheets with budget data, actuals, and variance calculations.

When the AI agent receives a workbook, it uses the Query Spreadsheet tool to ask "Extract the Q2 summary totals from the 'Consolidated' sheet, including department name, budgeted amount, actual spend, and variance." The tool returns a clean JSON array that the AI agent passes to a process model for CRM updates.

Why not other tools?

The Query Document (Text) tool and Read Full Document tool cannot parse tabular structure from spreadsheet files. The Semantic Search on Document and Folder tool does not support spreadsheet formats. Only the Query Spreadsheet tool can reliably extract structured data from XLS, XLSX, XLSM, and CSV files.

DocCenter models

DocCenter lets you build models for your specific documents, helping you achieve higher extraction and classification accuracy.

DocCenter workflows can include confidence scores, quality metrics, and reconciliation steps that route documents for human review when extraction results are incomplete or uncertain. This makes DocCenter the preferred approach for regulated, financial, or legally sensitive use cases where human validation is required before downstream processing.

Use DocCenter models for:

• Regulatory, financial, or legally sensitive documents.
• Repeatable, high-stakes tasks where accuracy is critical.

DocCenter models have the following characteristics:

• Near-perfect accuracy: designed for scenarios where errors have serious consequences.
• Specialized: performs better than general-purpose extraction.
• Requires setup: you must create and train DocCenter models, then connect them through process model tools.

How to use DocCenter with AI agents

To use DocCenter models with your AI agent, complete the following steps:

1. Add the process model to your AI agent as a tool.
   • Use AIA Classification Run Model Version for classification.
   • Use AIA Extraction Run Model Version for extraction.
2. In your AI agent's Instructions, tell your AI agent when to use the tool.

Enforcing DocCenter usage inside an AI agent

To ensure predictable, high-accuracy extraction and classification, explicitly instruct your AI agent to route all such requests through your DocCenter process model tool. This prevents the AI agent from attempting extraction with system document tools.

In your AI agent's Instructions, include the following: "Use only the DocCenter tool for document classification or field extraction. Do not attempt extraction using other system document tools like Read Full Document, Semantic Search on Document and Folder, or Query Document (Text)."

When calling the DocCenter tool, provide the following inputs:

• document: the user-provided document or content reference
• docType: the expected document category if known (for example "invoice", "w2"); otherwise "unknown"
• modelId: for example, "invoice_v3" (change if a different profile is required)

The tool returns the following outputs:

• classification (text)
• fields (key value map with strings/numbers/dates)
• status (success/partial/failed)
• errorMessage (when status != success)

If the tool is unavailable or fails twice, instruct the AI agent to stop and reply with: handoff_required: <one-sentence reason>.

Example use case

You have a mortgage application and are building an AI agent to digitize application packets and extract dozens of fields from forms like the Fannie Mae 1003 with extreme precision for your underwriting system.

Your AI agent receives a 50-page application PDF. It calls the DocCenter tool specifically configured to handle mortgage documents. The tool returns a highly accurate, structured data set of all required fields.

Why not other tools?

The risk of extraction errors with the Query Document (Text) tool is too high for this use case. Regulatory requirements demand the accuracy that only DocCenter can provide.

How your AI agent chooses document tools

Once you've added a document or document folder as a resource and enabled the relevant system document tools, your AI agent selects the most appropriate enabled tool based on your instructions and the task at hand. See System document tools for the full list and how to enable them.

By default, among the enabled system document tools, AI agents favor the Semantic Search on Document and Folder tool because it's more cost-efficient. You can influence this choice by using your Instructions to guide the selection.

To override this default behavior and force the AI agent to use a specific tool, explicitly instruct it in your Instructions. For example, to ensure the AI agent uses the Read Full Document tool, include instructions like "Read the entire document to understand the full context" or "Load the complete document into your context."

Decision framework

Your need	Tool your AI agent may use	Why
Find specific facts in large document collections	Semantic Search on Document and Folder	Cost-efficient, handles scale, precise retrieval
Understand relationships between document sections	Read Full Document	Provides complete view for deep understanding
Extract data from varying document formats	Query Document (Text)	Flexible, no setup required
Analyze scanned documents or interpret visual layout	Query Visual Document	Understands spatial relationships, checkboxes, diagrams
Extract structured data from spreadsheets	Query Spreadsheet	Returns reliable JSON from Excel and CSV files
Maximum accuracy for mission-critical extraction	DocCenter	Built specifically for high-stakes scenarios

Cost, speed, and accuracy tradeoffs

Every document tool represents a tradeoff between three factors:

• Cost: AI action usage.
• Speed: How quickly your AI agent can process the document.
• Accuracy: How reliably your AI agent produces correct results.

Semantic Search on Document and Folder is often chosen for cost and speed by only passing relevant snippets to your AI agent. This makes it ideal for high-volume scenarios where you need good accuracy at scale.

Read Full Document prioritizes accuracy by giving your AI agent complete context, but uses more AI actions. Your AI agent may choose this tool when the analysis requires understanding the full document.

Query Document (Text) balances all three factors, providing good accuracy with moderate cost and speed. It's a flexible option for general-purpose extraction.

Query Visual Document provides multimodal analysis for documents where visual layout carries meaning. It uses more AI actions than text-only tools due to image processing, but delivers accuracy that text-only extraction cannot achieve for scanned or visually complex documents.

Query Spreadsheet is optimized for structured data extraction from spreadsheet files. It returns clean JSON output with moderate cost and high speed, making it the clear choice for any tabular data in Excel or CSV format.

DocCenter maximizes accuracy at the expense of setup time and configuration effort. Use this when errors are unacceptable.

Common misconceptions

"If a document fits, the Read Full Document tool is always best."

Not quite. While AI agents can handle long documents, fitting doesn't mean your AI agent will choose the Read Full Document tool. Here's why:

• Cost: The Read Full Document tool uses more AI actions. If your AI agent only needs to find a specific fact, it's more likely to use the Semantic Search on Document and Folder tool, which is far more cost-efficient.
• Precision: For finding specific details in large documents, AI agents can overlook information buried deep in the content. The Semantic Search on Document and Folder tool is more reliable for finding specific information.
• Task alignment: If your goal is data extraction, your AI agent is more likely to use the Query Document (Text) tool or DocCenter tools, which are built for this task and more efficient than loading the full document.

Best practices

Follow these guidelines to help your AI agent choose the right document tools:

• Write clear Instructions that describe your task. Your AI agent will often choose the Semantic Search on Document and Folder tool for most document-based tasks since it's cost-efficient and handles the majority of use cases well.
• Be explicit in your instructions when you want your AI agent to use the Read Full Document tool. Include phrases like "read the entire document" or "consider the full context" in your Instructions.
• Describe extraction tasks clearly so your AI agent knows when to use the Query Document (Text) tool for flexibility without the setup overhead of DocCenter models.
• Specify visual analysis needs when your documents contain checkboxes, diagrams, or scanned content. Instruct your AI agent to "analyze the visual layout" or "interpret the visual structure" to guide it toward the Query Visual Document tool.
• Indicate spreadsheet processing when working with Excel or CSV files. Include phrases like "extract data from the spreadsheet" or "return the results as JSON" to guide the AI agent toward the Query Spreadsheet tool.
• Use DocCenter tools for mission-critical scenarios where accuracy requirements exceed 95%. Add these as tools and specify when to use them in your Instructions.
• Test with representative documents to see which tools your AI agent chooses and adjust your instructions if needed.
• Monitor AI action usage in the Monitor tab to understand the cost implications of your AI agent's document tool choices.

Related resources

For more information about working with documents in AI agents:

• For document tool limitations and technical details, see AI Agent Reference.
• For guidance on creating DocCenter models, see the DocCenter documentation.
• For testing your document processing, see Monitor AI Agents.
• For troubleshooting document issues, see Troubleshooting AI Agents.

Feedback