Balance AI Performance and Cost

What is the AI design triangle?

When designing an AI-powered process in Appian, you must navigate a fundamental set of tradeoffs between three competing factors:

• Accuracy (quality)
• Speed (latency)
• AI action consumption

You can visualize these tradeoffs as the AI design triangle. The AI design triangle shows that you can't maximize accuracy, speed, and AI action consumption at the same time. Improving one area, such as speed, often requires a compromise in another, like accuracy. For example, a faster response might be less accurate, while a more accurate one consumes more AI actions and takes longer. A successful AI implementation depends on consciously choosing the right balance for your specific business problem.

Scroll or drag to pan · ⌘+/⌘− to zoom · Esc to close

The three factors in the AI design triangle

Every AI skill call in your process is a negotiation between the three factors in the AI design triangle: accuracy, speed, and AI action consumption.

Accuracy

In the AI design triangle, accuracy refers to the measure of how correct, reliable, and useful the AI's output is. Accuracy is often the main reason for using AI in the first place—to achieve a level of quality that traditional automation can't.

Influenced by: The clarity and detail of your prompt, and the richness of the context you provide.

Speed

In the AI design triangle, speed is the time it takes for an AI skill to execute and return a result. This key performance measure is often referred to as latency: the delay between a request and the corresponding response. Low latency is critical for a good user experience in synchronous processes or real-time interfaces.

Influenced by: The complexity of the task, the size of the AI model, and the number of sequential AI calls in your process design.

AI action consumption

An AI action is the unit of measure used to track and charge for the execution of Appian's AI features within your applications.

Influenced by: The length of your prompts, the size of the data you pass in, and the verbosity of the answer you request from the AI.

Common AI design triangle scenarios

The right balance for the AI design triangle depends entirely on your specific business need. The following scenarios describe common optimization goals and the tradeoffs they require.

Optimize for maximum accuracy

Optimizing for maximum accuracy is for mission-critical tasks where getting the right answer is non-negotiable.

• Use cases:
  • Analyzing a legal contract for risks.
  • Extracting critical data from a patient's medical history.
  • Making a final compliance determination.
• Design choices:
  • In the prompt, ask the AI to output its reasoning process.
  • Enable extended thinking for complex reasoning tasks.
  • Provide rich, detailed prompts with multiple high-quality examples.
• The tradeoff: This approach leads to higher AI action consumption and slow speed. The resulting process is powerful but expensive, and it's best suited for asynchronous workflows where a user isn't waiting for an immediate response.

Optimize for maximum speed

Optimizing for maximum speed is for real-time applications that need an instant response to ensure a good user experience.

• Use cases:
  • Real-time duplicate case detection as a user types.
  • Providing instant suggestions in a form.
  • Powering a responsive chatbot.
• Design choices:
  • Write concise prompts that ask a direct, simple question.
  • Limit the amount of context passed into the AI skill.
• The tradeoff: Speed often comes at the cost of accuracy. The goal is a good enough answer delivered instantly, not a perfect answer that takes ten seconds. The cost is typically low due to the use of smaller models.

Optimize for minimum AI action consumption

Optimizing for minimum AI action consumption is for high-volume, non-critical tasks where AI utilization is the primary concern.

• Use cases:
  • Performing a quick, low-stakes review of many emails.
  • Creating descriptive tags for documents to make them easier to search later.
  • Sentiment analysis on customer feedback.
• Design choices:
  • Minimize AI action use by writing short prompts and asking for very brief, structured answers. For example, a single word or a boolean.
• The tradeoff: This method almost always leads to lower accuracy. The model might miss details and make more mistakes, but the AI actions consumed per execution are minimized.

Choose your AI design triangle priority

Before you build an AI-powered process, make a conscious decision about which factor in the AI design triangle you're optimizing for.

1. Identify your primary driver: For this specific business problem, what is the single most important outcome? Is it near-perfect quality, an instant response time, or the lowest possible AI action consumption?
2. Define your good enough thresholds: For the other two factors, establish the minimum acceptable level. If speed is your driver, what is the lowest accuracy your business can tolerate? If accuracy is key, what is the longest your process can reasonably take? Define these as quantifiable numbers for easy validation.
3. Choose your design pattern: With your priorities defined, select the tools and techniques that align with your goal. High accuracy demands multi-step patterns. High speed demands simple prompts.
4. Test, measure, and iterate: In your development environment, measure all three variables. Log the AI action consumption, time the process execution, and have business users score the accuracy of the output. Adjust your design until you hit the right balance.

Effective AI process design is the art of making deliberate tradeoffs. By understanding the AI design triangle and consciously deciding which corner to prioritize for a given use case, you can build solutions that are not only technologically impressive but also practical, cost-effective, and perfectly aligned with the needs of your business.

Feedback