Free cookie consent management tool by TermsFeed

Estimating AI Skill Token Usage

On This Page

Tip:  Interface patterns give you an opportunity to explore different interface designs. Be sure to check out How to Adapt a Pattern for Your Application.

Goal

When designing applications with generative AI, it's important to estimate token consumption for each transaction. This topic shows you how to estimate and measure token usage for AI skills so you can make design choices that balance cost, speed, and accuracy.

How tokens are counted

It's important to remember that a single business workflow might involve multiple AI calls. For example, a process could make one call to classify a document, a second to extract data, and a third to summarize the findings. Your total estimate must account for every call.

The token count for a single AI call is the sum of its inputs and outputs:

Total Call Tokens = Input Tokens + Output Tokens

The input tokens are a combination of the static prompt you design in the AI skill and the dynamic data you pass in at runtime:

Input Tokens = Tokens from your static AI skill prompt + Tokens from your dynamic smart service input

Your total process estimate is the sum of all individual AI calls within that process.

How to estimate token usage

Use the following workflow to estimate token usage while designing and developing your application.

To estimate the token count for your process:

  1. Map your AI calls: In your process model, identify every Execute Generative AI Skill smart service node. You'll perform a separate estimation for each node.
  2. Estimate each component: For each AI call, estimate its three parts. As a rule of thumb, you can estimate that one token is approximately four characters of text. You can also use an online tokenizer tool for a more precise count.
    • Prompt Tokens: Copy the complete text from your AI Skill design object. Use the rule of thumb or an online tool to estimate its size.
    • Dynamic Input Tokens: Take a representative example of the runtime data, such as the text from a typical email or case notes. Use the rule of thumb or an online tool to estimate its size.
    • Output Tokens: Draft an example of a typical successful response from the AI, such as the desired JSON output. Estimate its token count. Don't underestimate this; detailed summaries or large JSON objects can consume significant tokens.
  3. Sum for a total process estimate: Add the estimates for every AI call in your workflow. The result is the estimated token consumption for a single, end-to-end transaction.

Measure actual token usage

While estimation is crucial for design, the Execute Generative AI Skill smart service returns the actual token count as node outputs for every transaction. You can find the exact numbers in the following smart service outputs:

  • Input Tokens (Number - Integer)
  • Output Tokens (Number - Integer)

Tip:  As a best practice, you should log these outputs to a database for every transaction. This turns your estimation into a precise measurement, allowing you to track actual costs, identify unexpectedly "expensive" prompts, and refine your future estimates with real-world data.

Appian displays AI token metrics on the Adoption page in Forum to ensure customers are aware of their AI usage across environments and Appian AI features.

Use your estimate to make design decisions

Understanding the token cost of a transaction empowers you to make critical design trade-offs.

For example, you might ask:

  • Is the added accuracy worth the cost? "My prompt is more accurate with five few-shot examples, but my estimation shows this adds 600 tokens per call. For a process that runs 10,000 times a month, that's an extra 6 million tokens. Is the accuracy gain worth the cost?"
  • Can I be more concise? "Does my summary really need to be five paragraphs long, or can I prompt for a three-sentence summary and save 200 output tokens per call?"

Note:  We recommends prioritizing accuracy over token cost, as the business value provided by a more accurate response often exceeds the monetary cost of the tokens.

Licensing and platform considerations

Keep these points in mind when managing your overall token usage:

  • Monthly allotment: Your allotted tokens are for a specific monthly period. Whether unused tokens roll over depends on your license tier.
  • Service continuity: Appian will not abruptly disable the AI service if you reach your token limit. Instead, we will notify your Appian Account Executive to discuss your usage.
  • Purchasing additional tokens: If you anticipate exceeding your limit, you can purchase additional tokens. Contact your Appian Account Executive for details.

Feedback