Verification Models

Verification overview

Verification is the process of auditing a document against a set of business rules or compliance standards. Unlike extraction (which pulls structured data from documents) or classification (which categorizes documents by type), verification evaluates whether a document meets specific criteria defined by the developer.

In DocCenter, you can integrate verification into your document processing workflows using verification models. A verification model takes one or more target documents as input, evaluates them against rules you define, and outputs a structured record containing rule results (Pass or Fail) with detailed reasoning for each determination. Unlike classification and extraction models, which process a single document per instance, verification models can evaluate multiple documents in a single instance.

The Verification tab in DocCenter has two pages: Models and Instances.

• Models: Lists the verification models you've previously created. When you click on a model, you see model configurations and performance details.
• Instances: Lists the processes where verification models were executed.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

How verification models work

Verification models use an AI-powered agent to evaluate documents against developer-defined rules. The agent reads the target document, compares its content against rules written in natural language and any uploaded reference materials (such as policy manuals or procedure documents), and determines whether each rule passes or fails.

The verification workflow consists of three phases:

1. Rule definition: Define one or more rules using a structured natural language pattern. Each rule specifies what to verify, what constitutes a pass, and what constitutes a failure.
2. Document evaluation: When you run a verification model against a target document, the agent processes the document and evaluates each rule independently.
3. Results review: Review the verification results, including the pass/fail status for each rule and the agent's reasoning.

Verification in a document processing pipeline

Verification models are designed to work alongside classification and extraction models in a complete document processing pipeline:

1. Classify the document to determine its type.
2. Extract structured data from the document.
3. Verify the document against business rules or compliance standards.

Each step is independent. You can use verification models on their own or combine them with classification and extraction models depending on your workflow requirements.

Usage considerations

Supported file types

Verification models support the same file types as other DocCenter models:

• CSV
• DOCX
• EML
• GIF
• HTML
• JPG, JPEG
• MSG
• PDF
• PNG
• TIFF
• XLSX

Verification model capabilities

Verification models support the following:

• Reference documents for additional context: When you create a verification model, you can upload context (such as reference procedure manuals or policy documents) for the agent to use during evaluation.
• Confidence scoring: The agent returns a confidence level (Low, Medium, or High) alongside each rule result.
• Triggers and versioning: Verification models support triggers and versioning, just like classification and extraction models.

What verification models can't do

Verification models evaluate documents against rules, but they do not:

• Extract structured data: Verification models do not output key-value pairs. To extract data and verify the document, chain an extraction model and a verification model sequentially.
• Remediate documents: The agent flags compliance failures but does not edit, fix, or generate corrected text.
• Query external systems: Rules are evaluated strictly against the content in the uploaded documents. The agent does not call external databases or APIs during execution.
• Auto-generate test cases: Test cases must be selected manually or created through the + TEST CASE dialog. Verification models do not automatically generate test cases from a document set.
• Import or export test cases: Verification test cases remain linked to the model in DocCenter. You cannot import or export them as external files.
• Compare model versions side by side: A verification test run evaluates a model version against the expected results in a test suite. Verification does not provide a side-by-side view comparing two model versions.
• Support usage groups: Verification models do not support usage groups. Extraction and classification models do support usage groups.

Create and configure a verification model

1. In the Verification page group, click Models.
2. Click CREATE VERIFICATION MODEL.

3. Enter the Model Details:

   Field	Description
   Name	Name to uniquely identify the verification model.
   Description	Description of the model (optional).

4. Create Rules:

   • Describe Rule: Write verification rules using a natural language pattern to define what the agent checks.
     1. Provide a Rule Name.

     2. Add Instructions, which should have three components:

        Component	Purpose	Example
        VERIFY THAT	Defines the condition to check in the target document.	The document contains a signature on every page.
        PASS IF	Defines what constitutes a successful check.	Every page in the document contains at least one signature.
        FAIL IF	Defines what constitutes a failed check.	One or more pages are missing a signature.

        You can click Generate Instructions to populate these components automatically based on your rule name, or define them manually.

     3. Click ADD RULE to create additional rules the agent should follow during verification.
   • Generate from Document: Instead of writing every rule manually, you can let the agent analyze a document and suggest rules.
     1. Upload a document.
     2. The agent identifies checkable criteria from the reference document and proposes structured rules.
     3. Carefully review the rules the agent generates and make any necessary changes.
5. Add Context: Upload the reference materials the agent uses during rule evaluation. These documents help provide the agent with additional standards, policies, or procedures against to verify the target documents.
6. Click CREATE.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Run a verification model

To verify documents against your defined rules:

1. In the Verification page group, click Models.
2. Find and click the model you want to run.
3. Click RUN VERIFICATION.
4. Upload one or more files.
5. Click VERIFY.

The agent evaluates the target document against each rule and produces results.

Review verification results

After a verification model runs, view the results on the Instances tab:

1. In the Verification page group, click Instances.
2. Click the instance you want to review.

The instance detail view displays:

• Overall status: A summary of pass or fail results across all rules.
• The Rules the agent followed in the instance. For each rule, the following information appears:
  • Agent Result: Each rule with its determination (PASS or FAIL).
  • Agent Confidence: The agent's confidence level (LOW, MEDIUM, or HIGH) for each rule.
  • Override Value: If an instance has been reconciled, any manually changed results and justification comments appear in this column.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Reconcile verification results

If a rule result is incorrect, you can reconcile the instance to correct the determination:

1. Open the instance detail view.
2. Click RECONCILE INSTANCE.
3. Review each rule result and update the determination as needed.
4. Click SUBMIT to save the reconciled results.

Test a verification model

Test a verification model to confirm that rule changes produce the pass or fail outcomes you expect before you publish a new model version. If you have tested extraction or classification models, the workflow is the same.

A verification test case stores the expected pass or fail result for each rule, along with the input documents. When you run a test suite, DocCenter re-runs the agent against the test documents and compares the agent's rule-by-rule results against those expected results.

The Testing tab on a verification model has four sub-tabs: Test Cases, Test Case Runs, Test Suites, and Test Suite Runs.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Create a verification test case

Create a test case to define the expected pass or fail outcome for each rule for a given set of documents. You can create a test case directly from the Testing tab, or save a reconciled instance as a test case.

From the Testing tab:

1. Go to the model's Testing tab and select Test Cases.
2. Click + TEST CASE.
3. Enter a Test Case Name.
4. Upload one or more Input Documents.
5. For each rule under Expected Results, select Pass or Fail.
6. Click CREATE.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

From the Instances list:

1. In the Instances tab, click the options icon on the instance row.
2. Select Set as Test Case.

From an instance detail view:

1. Open the instance to view its results.
2. Click SET AS TEST CASE.

Create a verification test suite

Group verification test cases into a test suite when you want to target specific rules or document types rather than running every test case for the model:

1. Go to the model's Testing tab and select Test Suites.
2. Click + TEST SUITE.
3. In the New Test Suite dialog, select the test cases you want to include from the Available Test Cases panel on the left.
4. Enter a Name and an optional Description.
5. Click CREATE.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Run a verification test suite

Running a test suite re-evaluates its test cases against a verification model version, so you can confirm a rule change produced the expected results before publishing:

1. Go to the model's Testing tab and select Test Suites.
2. Find the test suite you want to run and click RUN TEST SUITE.
3. In the Run Against Version dropdown, select the model version to test against.
4. Click RUN TEST SUITE to start the run.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

You can also run test cases without creating a suite directly from the Test Cases tab:

• To run all test cases at once, click RUN ALL TEST CASES.
• To run a single test case, click the options icon next to the test case, select Run Test Case, choose a model version, and click RUN TEST CASE.

Review verification test run results

After a test suite run completes, go to the Test Suite Runs tab to review results.

The Test Suite Runs tab shows all test suite runs on the left, with the run date, initiated by, and overall accuracy. Select a run to see the individual test case results on the right — each test case shows its associated Instance, Documents, Accuracy, and Created date.

To review test case runs independently of a suite:

1. Go to the Test Case Runs tab.
2. The list shows all runs across all test cases, with columns: Test Case, Instance, Documents, Status, Accuracy, and Created.
3. Review the Accuracy value for each run. Accuracy is color-coded — green indicates high accuracy, orange indicates moderate, and red indicates low.
4. Click an Instance to open the verification instance and compare the agent's rule-by-rule results against the expected results.
5. Look for rules where the agent's result does not match the expected outcome. These discrepancies indicate that a rule change affected that document's outcome.

For definitions of the accuracy metrics, see Verification accuracy.

Integrate a verification model into a process

To run a verification model from your own process models, use the AIA_ENTRYPOINT_RunVerificationModel process model as a subprocess. This process model accepts the following parameters:

Parameter	Type	Description
documents	Document (List)	One or more documents to verify against the model's rules.
modelUuid	Text	The key that uniquely identifies the verification model to run.

To integrate verification into your workflow:

1. Open your process model in Appian Designer.
2. Add a subprocess activity node.
3. Select AIA_ENTRYPOINT_RunVerificationModel as the subprocess.
4. Map the required parameters from your process variables.

Feedback