Testing & Metrics

Before you implement an extraction or classification model, you want to be confident that it effectively serves its purpose. You can test these models directly within DocCenter and view metrics on performance. If you see an opportunity to improve, you can create a new version of the model and make adjustments.

• Extraction models: Test a model and create a new version.
• Classification models: Test a model and create a new version.
• Verification models: Test a model.

After determining the models that perform best for your use case, use DocCenter to view performance metrics.

Classification accuracy

Classification accuracy is measured for each model version.

Accuracy is the ratio of correct predictions to total predictions, expressed as a percentage. "Correct predictions" includes true positives (correctly predicting that the document is part of a certain category) and true negatives (correctly predicting that the document isn't part of a certain category).

To view model version accuracy:

1. Navigate to the Classification page group > Models page.
2. In the list, locate the model you're interested in.
3. Refer to the Accuracy column to view latest model version's accuracy.
4. To view accuracy for all versions of a model, click on the Model Name.

How is accuracy calculated for classification?

Classification accuracy is calculated using historical reconciliation information. If a classification category is changed during reconciliation, it is considered inaccurate. Overall accuracy is calculated using a ratio of accurate and inaccurate classifications.

For classification models configured to split packets into individual documents, accuracy is calculated based on two LLM decisions:

1. Categorization accuracy: Whether the model correctly identified the document type for each section.
2. Page range accuracy: Whether the model correctly identified the start and end pages for each document.

Both decisions are evaluated during reconciliation. If either the category or page range is corrected, the instance is considered inaccurate for that split. Overall model accuracy reflects the combined performance of both categorization and page range identification.

Example

You've created a model to recognize and classify two document types: invoices and purchase orders. You uploaded 35 test documents (all invoices), and the model positively identifies 27 of them as invoices and the remaining 8 as purchase orders. In reality, all 35 are invoices.

The model's accuracy is (27 / 35) * 100, or 77%.

Extraction accuracy

Extraction accuracy is measured and displayed at multiple levels:

• Individual fields
• Model version
• Extraction instance

To view model version accuracy:

1. Navigate to the Extraction page group > Models page.
2. In the list, locate the model you're interested in.
3. Refer to the Accuracy column to view the latest model version's accuracy.
4. To view accuracy for all fields in the latest version of a model, click on a Model Name.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

To view instance accuracy:

1. Navigate to the Extraction page group > Instances page.
2. In the list, locate the instance you're interested in.
3. Refer to the Accuracy column to view the instance's accuracy.
4. To view accuracy for all fields in an instance, click on the Instance Name.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

How is accuracy calculated for extraction?

Extraction accuracy is calculated using reconciliation information. If a field value is changed during reconciliation, it is considered inaccurate.

Level	Accuracy is calculated by…
Fields	dividing the number of times the field value wasn't changed during reconciliation by the number of total times the field was extracted for this model version.
Extraction instance	dividing the number of fields not changed during reconciliation by the number of total fields for this instance.
Model version	average extraction accuracy across all extraction instances associated with the model version.

The field data type also determines what's counted as "accurate":

• Normal field: A field is considered accurate if the extracted result is not changed, overridden, or cleared.
• List field: A list field is considered accurate if all elements were extracted correctly. If any were wrong, the field is considered inaccurate.
• Table field: A table field is calculated at the cell level. The accuracy of a table field is the number of correctly extracted cells divided by the total number of extracted cells.

Example

You've created a model to extract five fields from resumes:

• Name (Text)
• Email address (Text)
• Current title (Text)
• Current employer (Text)
• Skills (List)

During testing, you upload 20 resumes. All test documents have the five fields you're extracting:

The model is able to extract:

Field	Total extracted	Total correct extraction
Name	20	20
Email address	19	17
Current title	20	15
Current employer	15	15
Skills	20	15 (3 are missing a value, and 2 include incorrectly extracted values)

Here's how accuracy would be calculated for the document, fields, version, and instance:

Model version fields:

Field	Formula	Accuracy	Explanation
Name	20 / 20	100%	The field was extracted 20 times in 20 instances, and those 20 were correct.
Email address	17 / 20	85%	The field was extracted 19 times in 20 instances, but only 17 were correct.
Current title	15 / 20	75%	The field was extracted 20 times in 20 instances, but only 15 were correct.
Current employer	15 / 20	75%	The field was extracted 15 times in 20 instances, and those 15 were correct.
Skills	15 / 20	75%	The lists were extracted 20 times in 20 instances, but only 15 were entirely correct

Confidence score

The confidence score measures how sure the model is of its prediction. It is calculated for each prediction the model makes. For extraction models, confidence score is a reliability metric assigned to each word or line extracted from a document.

Most decisions (made by humans or computers) can vary in confidence. If someone asked you, "what's 2 + 2?" and you answer "4," your confidence in that answer is probably very high. But if someone asked you, "what did you eat for lunch last Tuesday?" you might not be so sure of your reply. "I think I had pizza" indicates you aren't very confident in your answer. Machine learning models apply the same confidence to their predictions. The confidence score indicates the system's certainty in the accuracy of the extracted value for a specific field.

How confidence scores are measured

During extraction, the model scans the document and identifies text at specific locations on the page. For each piece of text it identifies, it assigns a confidence score—a number between 0 and 100 that indicates the probability that the text was correctly read at that location. You can set a confidence threshold to define the minimum acceptable score; fields that fall below it are flagged for manual review.

Key characteristics of confidence scores:

• The score reflects certainty about what text was read and where on the page it was found.
• The score does not reflect semantic accuracy—it does not evaluate whether the extracted value makes logical sense in context.

Common factors that lower confidence scores:

• Poor scan quality or low-resolution images.
• Handwritten text.
• Overlapping or obscured text.
• Unusual fonts or complex formatting.

How to view confidence scores

To use confidence scores, enable Advanced Layout or Bounding Boxes during model creation. Then, after the model is created, configure a confidence threshold on the Straight Through Processing tab.

After running extraction, you can review low-confidence fields at the instance level on the Summary tab and field level on the reconciliation screen.

To view the count of fields below the confidence threshold:

1. After the extraction completes, go to the Instances tab.
2. Click the instance to open the Summary page.
3. In the Errors section, review how many fields and tables are below the confidence threshold (for example, 1 field and 1 table are below the confidence threshold).

To see individual confidence scores for fields that were flagged and correct the values, click Reconcile Fields. The reconcile fields screen applies these visual cues:

• Table-level flags: A count of extracted cells below the threshold (for example, 7 out of 32 extracted cells are below the confidence threshold).
• Field-level warnings: A yellow warning triangle with the specific confidence score (for example, Confidence: 63) displays directly below the data fields that fell below the configured threshold, allowing you to quickly identify and correct questionable values.

To resolve a confidence flag, compare the flagged value against the document preview on the right side of the screen. If the value is incorrect, click the highlighter icon next to the field and select the correct value from the document. See Reconcile Extraction Results for full instructions on completing reconciliation tasks.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

Example

You've created a model to extract seven fields from invoices:

• Company Name (Text)
• Bill To Address (Text)
• Bill To State (Text)
• Invoice Date (Text)
• Due Date (Text)
• Tax Amount (Text)
• Total (Text)

The model has Bounding Boxes and Advanced Layout enabled and a confidence threshold of 70. During testing, you upload a scanned invoice with poor print quality on some sections.

After extraction, you navigate to the Instances tab and see that the instance has a status of Extracted with Errors. You click the instance and go to the Summary page, then click Errors to see that two fields are below the confidence threshold.

You click Reconcile Fields to review the flagged fields. Only fields below the confidence threshold display a confidence score:

Field	Extracted Value	Confidence Score	Threshold	Flagged?
Company Name	Acme Corp	NA	70	No
Bill To Address	123 Main St	NA	70	No
Bill To State	CA	NA	70	No
Invoice Date	01/15/2026	NA	70	No
Due Date	02/15/2026	NA	70	No
Tax Amount	$47.50	65	70	Yes
Total	$547.50	65	70	Yes

The Tax Amount and Total fields display Confidence: 65 below their respective values. The remaining fields do not display confidence flags because they scored above the threshold. You can then compare the flagged values against the document preview and correct them during reconciliation.

See Reconcile Extraction Results for how to complete the reconciliation task.

How is confidence different from accuracy?

Confidence and accuracy are related but distinct concepts.

• Confidence is determined per field at extraction time. It indicates the probability that a specific extracted value is correct.
• Accuracy is an average determined from an entire data set after reconciliation. It measures how often the model's extractions were correct overall.

To understand the difference, consider the following example:

You've created a model to extract seven fields from invoices. You upload 35 test invoices and reconcile the results. During reconciliation, you correct any values that were extracted incorrectly. Of the 245 total field extractions (7 fields × 35 documents), 189 are correct without changes—meaning the extracted value was not changed, overridden, or cleared during reconciliation.

The model's accuracy is (189 / 245) × 100, or 77%. This percentage appears beside each field on the reconciliation screen, showing how reliable the model has been for that field across all reconciled instances.

Now look at confidence for a specific field in the same document. With a confidence threshold of 70, the model flags the total amount field on the reconciliation screen with Confidence: 60—it was extracted from a poorly scanned area and the model is less certain about the text it read. Fields above the threshold, such as the company name, do not display a confidence score.

A low confidence score doesn't guarantee the extracted value is wrong—it means the model is less certain about what it read. Confidence reflects the model's estimation of likelihood, not the actual result. In this case, a threshold of 70 flags the total amount for review (confidence 60 < threshold 70) while fields the model is more certain about pass through without a flag.

Verification accuracy

Verification accuracy measures how often the agent's pass or fail determination for each rule matches the expected result stored in a test case. It is calculated as the number of rule evaluations where the agent's result matched the expected result, divided by the total number of rule evaluations in the test run.

Example

You have a verification model that checks loan application documents against three rules:

• All required fields are completed.
• The applicant's signature is present on the final page.
• The declared income matches the supporting documentation.

You create a test suite with four test cases, each representing a different loan document where you have set the expected pass or fail result for all three rules. When you run the test suite, DocCenter evaluates all 12 rule determinations (three rules × four documents).

The results are:

Test case	Rule 1	Rule 2	Rule 3	Matched expected?
Document A	Pass	Pass	Pass	Yes, Yes, Yes
Document B	Pass	Fail	Pass	Yes, Yes, Yes
Document C	Pass	Pass	Fail	Yes, Yes, No
Document D	Fail	Pass	Pass	Yes, No, Yes

The agent matched the expected result for 10 of the 12 rule evaluations. The test run accuracy is (10 / 12) × 100, or 83%.

Reviewing the two mismatches shows which rules and documents need attention before you publish the updated model version.

To view accuracy for a test case:

1. Go to the model's Testing tab and select Test Cases.
2. Click a test case to open its detail page.
3. In the Test Case Runs table, review the Accuracy column for each run.

Test suite accuracy for extraction models

Test suite accuracy measures how well your extraction model performs against a specific subset of test cases. This allows you to track improvements for targeted document types or fields without running your entire test set. See extraction accuracy for the foundational accuracy calculation method.

For the equivalent metrics on verification models, see Verification accuracy and rule agreement.

How test suite accuracy is calculated

Test suite accuracy is calculated the same way as overall extraction accuracy, but only for the documents included in the test suite. You can see the accuracy metrics for test suites and for each instance within a test suite:

1. Instance: Dividing the number of fields not changed during reconciliation by the number of total fields for that instance.
2. Test suite run: The average accuracy of all instances within the test suite.

View test suite accuracy

To view accuracy metrics for a test suite run:

1. Once you are in the Testing tab, select Test Runs to see all the test suite runs.
2. Click on the test run you want to review.
3. View the accuracy metrics displayed for the overall test suite and for individual instances.

Use test suite instances accuracy to improve your model

Accuracy percentage for each instance in a test suite helps you:

• Focus refinements: Identify which fields or document types need improvement by running targeted test suites.
• Validate changes: After updating your prompt, run the same test suite to see if accuracy improved.
• Track progress: Monitor accuracy over time as you iterate on your model.

Example

You've created an extraction model for invoices and want to improve accuracy for a specific set of high-value invoices. You create a test suite called "High-Value Invoices" with 15 test cases. After running this test suite against your current model version, you see 85% accuracy. You then refine your prompt to better handle complex line items and create a new version. Running the same test suite against the new version shows 92% accuracy, indicating your changes improved performance for this specific document type.

Feedback