Drive Thinking in AI Models

Goal

This recipe is a best practice for guiding your generative AI model to articulate its thinking before providing the final answer, which produces reliable and auditable results for complex tasks.

Instead of simply accepting an answer like "this invoice is fraudulent," you implement a pattern that records the AI's entire line of thinking. This approach transforms the AI from a black-box system into a transparent and auditable team member.

Use this recipe to:

• Establish a clear audit trail and build user trust by showing the explicit logic the AI model used to reach a conclusion.
• Improve the accuracy of final decisions for reasoning tasks, especially for those involving complex, multi-step problems.
• Reduce the risk of hallucination and ensure the AI follows instructions closely by constraining it to articulate its steps logically.

How it works

When designing prompts for generative AI, a fundamental best practice is to instruct the AI model to articulate its reasoning process before providing the final answer. Including a thinking key in the response provides a direct window into the model's logic for a specific transaction.

Implementing the thinking key in an AI model delivers the following primary benefits:

• Increased model accuracy: Requires the model to think step-by-step.
• Essential process transparency: Provides a clear, auditable trail of the model's logic.

This pattern is most beneficial for reasoning tasks, such as classification, analysis, and complex decision-making, where reliability and auditability are critical.

Implementation

Instruct the model to first lay out its reasoning and then derive the answer from that reasoning. This requires the model to follow a more logical path. When you place the thinking key before the output field in your requested structure, the model resolves its thoughts before committing to a decision. This approach significantly improves accuracy and reduces errors.

Test it out

Completed in: prompt builder and process model

1. Navigate to Appian Designer and open your prompt builder AI skill or create a new one.

2. Input the example prompt below. The prompt instructs the AI to structure its full reasoning process under the thinking key before delivering the final result.

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   <role>
   You are an expert contract analyst.
   </role>
   
   <instructions>
   1. First, analyze the provided contract clause step-by-step in a 'thinking' block. Note the parties involved, the effective date, and any payment terms.
   2. After your thinking process, determine if the clause contains non-standard payment terms.
   3. Respond in a valid JSON object with a 'thinking' key followed by an 'hasNonStandardTerms' key.
   </instructions>
    
   <clause>
   ...[contract text]...
   </clause>
   
   <response>
   {
     "thinking": "...",
     "hasNonStandardTerms": boolean
   } 
   </response>
   

3. When you execute the prompt, it returns the JSON output with a thinking key, similar to the example below. Example output:

   1
   2
   3
   4
   {
   "thinking": "Step 1: Review contract clauses. Parties involved: Acme Auto; Effective date: 10/31/2025; Payment terms: Payment due before services are rendered. Step 2: Compare these clauses to standard terms and note discrepancies. Step 3: Synthesize findings. The payment terms ask for payment before services are rendered, which is unusual.",
   "hasNonStandardTerms": true
   }
   

4. Next, open the Process Model and place the Execute Generative AI Skill smart service after the start form.
5. Configure the smart service to store the AI's response as a process variable such as pv!jsonOutput, so you can work with it more easily in later nodes.
6. Immediately after the smart service, use a Script Task and the a!fromJson() function to parse the pv!jsonOutput and cast the data to a record type.

   1
   2
   3
   4
   cast(
   recordType!Orders,
   a!fromJson(pv!jsonOutput)
   )
   

7. Finally, add and configure a Write Records smart service to save the data to the record type.
8. Configure the Write Records smart service to save the data to your record type by mapping:
   • The parsed boolean value to the record field. For example, hasNonStandardTerms.
   • The parsed thinking text to the separate audit trail field. For example, ai_analysis_notes.
   

   Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

After the AI Skill returns the JSON output, the process model parses it. Using a write records node, both the thinking text and the hasNonStandardTerms boolean are written to the database record. The database table would have columns like hasNonStandardTerms (boolean) and ai_analysis_notes (text).

On the Appian interface where a user reviews the contract analysis, both fields are queried and displayed. The hasNonStandardTerms value might drive a prominent visual indicator, such as a red NON-STANDARD tag. The ai_analysis_notes field is rendered in a read-only text box labeled AI Reasoning, giving the user immediate, full context for the tag. This enables the user to instantly validate the AI's conclusion and take appropriate action.

Advantages

Mandating a thinking component in your AI's response is a simple but powerful design pattern. This approach builds essential trust with users and leverages proven prompting techniques to make the AI more accurate, logical, and less prone to errors.

1. Improves model accuracy: This structure makes the AI more accurate by ensuring the following:
   • Reduces hallucination: Forcing the model to articulate steps makes it less likely to invent facts because each step logically follows the last. The model effectively self-corrects as it writes out its thought process.
   • Improved complex reasoning: The model dedicates resources to solving multi-step problems sequentially, rather than trying to compute the entire result at once.
   • Follows instructions more closely: The thinking key reserves space for longer explanations when the final output must be in a strict format. This separation prevents conversational text from appearing in the actual answer field, leading to cleaner, more reliable data outputs.
2. Boosts business transparency: This structure builds trust and creates a necessary audit trail through the following mechanisms:
   • Auditability: Creates a clear audit trail that shows exactly what information the AI used and the logical steps it took to arrive at a conclusion.
   • User empowerment: Surfacing this reasoning to an end user demystifies the AI's decision, allowing the user to address specific points immediately.
   • Faster course correction: If the AI makes an error, the user instantly sees where the logic went wrong. This feedback is critical for refining the process.

Feedback