Process Handwriting

On This Page

If a person can't read certain handwriting, AI probably can't either. Poor handwriting and low-quality scans make accurate extraction difficult, but combining an advanced IDP tools AI skill and an unstructured document AI skill can improve results.

Note: If you're using plug-ins to convert documents from one format to another, refer to that plug-in's documentation in its AppMarket listing for more information.

Techniques for extracting handwriting

There are three main approaches you can use to extract handwritten text, each with its own strengths and limitations.

Feature	Description
Advanced IDP tools AI skill	Works well with structured documents and clear handwriting. May struggle with faded, blurred, or inconsistent handwriting.
Unstructured document AI skill	Uses deep learning models trained on both image and text. Can infer missing or unclear information using surrounding context.
Combination of both	If Advanced IDP tools AI skill alone is not reliable, an unstructured document AI skill can be used to double-check the results. If both methods produce the same result, we can be more confident in the extracted text. If results differ, use downstream reconciliation or human review.

Example

Consider this example where both the Advanced IDP tools AI skill and the unstructured document AI skill were used to extract data from a document. The two techniques produced different extraction results. The multimodal method misidentified the first name "Sofia" and extracted the email address as "sofa."

Field	Advanced IDP tools AI skill extracted	Unstructured document AI skill extracted
First Name	Sofia	Sofia
Email	`sofia@test.com`	`sofa@test.com`

Resolve differences

When the Advanced IDP Tools and unstructured document extraction AI skills each provide different results, you can use two approaches to resolve the differences:

Downstream AI reconciliation: You can use another AI skill to compare the extracted values and use context clues to choose the correct one. The AI skill can also cross-check the extracted text with other parts of the document.
Human-in-the-loop: Alternatively, discrepancies can be sent to a human reviewer. Use business rules to determine when manual review is necessary.

Sample prompt for AI reconciliation

Completed in: prompt builder

<instructions>

You will be given extracted results from two upstream models. Your job is to reconcile differences and output the final result.

For each field:

1. If all sources agree, say: "Consistent across all sources."
2. If they differ, list the values and explain the choice.
3. If one source is missing a value, indicate which one.
4. Use context from the document and look for additional references to verify accuracy.

</instructions>

Feedback

Was this page helpful?