Build an AI Chat with Real-Time Streaming and Persistent History

On This Page

Tip: Interface patterns give you an opportunity to explore different interface designs. Be sure to check out How to Adapt a Pattern for Your Application.

Goal

Build an AI chat interface with real-time streaming responses and persistent conversation history using the Chat Field component.

This design pattern is not recommended for offline interfaces because reflecting immediate changes in an interface based on user interaction requires a connection to the server.

This recipe shows you how to build an AI-powered chat interface that streams responses in real time and saves conversation history to a record type. Users can ask questions, see answers appear as they're generated, and return later to view their previous conversations.

Use this recipe when you want to create a conversational AI experience that maintains context across sessions, such as an HR policy assistant, customer support chatbot, or internal knowledge base assistant.

Prerequisites

Before you begin, you need:

A record type to store chat messages with these fields:
- messageContent (Text) - The message text
- role (Text) - Either "USER" or "ASSISTANT"
- user (User) - The user who sent or received the message
- createdOn (Date and Time) - When the message was created
- Expression rules to query and format the chat history for the logged-in user, sorted by creation date.
Access to the Chat Component component and a!callLanguageModel() function.

Create supporting expression rules

You'll need two expression rules to manage chat history.

CFR_getChatHistoryByUserId
CFR_getFormattedChatHistoryByUser

Get chat history for display

Create an expression rule called CFR_getChatHistoryByUserId that queries the last 20 messages for the current user. This rule returns record data formatted for display in the chat component.

Example

a!queryRecordType(
  recordType: recordType!CFR Chat Message,
  filters: a!queryFilter(
    field: recordType!CFR Chat Message.fields.user,
    operator: "=",
    value: ri!username
  ),
  pagingInfo: a!pagingInfo(
    startIndex: 1,
    sort: a!sortInfo(
      field: recordType!CFR Chat Message.fields.createdOn,
      ascending: false()
    )
  )
).data

Format history for the language model

Create an expression rule called CFR_getFormattedChatHistoryByUser that returns the same messages formatted as a list of maps with role and messageContent keys. The AI model requires this specific format in its messages parameter.

Example

a!forEach(
  items: a!queryRecordType(
    recordType: recordType!CFR Chat Message,
    filters: a!queryFilter(
      field: recordType!CFR Chat Message.fields.user,
      operator: "=",
      value: ri!username
    ),
    pagingInfo: a!pagingInfo(
      startIndex: 1,
      sort: a!sortInfo(
        field: recordType!CFR Chat Message.fields.createdOn,
        ascending: false()
      )
    )
  ).data,
  expression: a!map(
    role: fv!item[recordType!CFR Chat Message.fields.role],
    messageContent: fv!item[recordType!CFR Chat Message.fields.messageContent]
  )
)

Set up the interface

This interface uses three key elements to create a persistent chat experience:

Chat Field component - Displays the conversation and handles user input
Language model integration - Generates AI responses using a!callLanguageModel()
Record type - Stores conversation history for retrieval across sessions

The interface loads previous messages when the page opens, displays new messages immediately as users type them, and saves the complete conversation to the database after each exchange.

To create the interface:

In your application, create a new interface object.
Copy and paste the following expression into the interface.

a!localVariables(
  /* Store the user's most recent message for database persistence */
  local!lastUserMessage,
  local!modelResponse,
  local!streamingUuid,
  
  /* Track the current streaming response from the language model */
  local!streamedData: a!map(done: true, text: ""),

  /* Load chat history from the record type */
  local!messages: rule!CFR_getChatHistoryByUserId(
    username: loggedInUser(), 
    maxResults: 20
  ),
  /* Fetch history in a format specific to the language model as a list of maps with role and content. */
  local!formattedHistory: rule!CFR_getFormattedChatHistoryByUser(username: loggedInUser()),
  a!chatField(
    title: "Promotion Policy Assistant",
    height: "EXTRA_TALL",
    
    /* Display all previous messages from the record type */
    messages: a!forEach(
      items: local!messages,
      expression: a!chatMessage(
        messageContent: fv!item[recordType!CFR Chat Message.fields.messageContent],
        role: fv!item[recordType!CFR Chat Message.fields.role]
      )
    ),

      /* Handle new user messages */
      saveInto: {
        /* Add the user's message to the display immediately */
        a!save(
          local!messages,
          append(
            local!messages,
            a!map(role: "USER", messageContent: save!value)
          )
        ),
        
        /* Store the message for later database write */
        a!save(local!lastUserMessage, save!value),
        
        /* Start the language model stream */
        a!save(
          local!streamingUuid,
          a!callLanguageModel(
            isStreaming: true,
            prompt: concat(
              "You are a helpful HR assistant. Answer using the search context and user memory below: ",
              save!value
            )
          )
        )
      },

      /* Track the active language model stream */
      streamingUuid: local!streamingUuid,
      
      /* Process streaming responses from the language model */
      streamingSaveInto: {
        a!localVariables(
          local!val,
          {
            a!save(local!val, save!value),
            if(
              local!val.done,
              /* The language model has finished responding */
              {
                /* Clear the streaming buffer */
                a!save(local!streamedData, a!map()),
                
                /* Add the complete response to the message list */
                a!save(
                  local!messages,
                  append(
                    local!messages,
                    a!map(
                      role: "ASSISTANT",
                      messageContent: local!val.text
                    )
                  )
                ),
                a!save(local!modelResponse, local!val.text),

                /* Save both messages to the database */
                a!writeRecords(
                  records: {
                    /* Save the user question */
                    "recordType!CFR Chat Message"(
                      "recordType!CFR Chat Message.fields.user": loggedInUser(),
                      "recordType!CFR Chat Message.fields.role": "USER",
                      "recordType!CFR Chat Message.fields.messageContent": local!lastUserMessage,
                      "recordType!CFR Chat Message.fields.createdOn": now()
                    ),
                    /* Save the AI response */
                    "recordType!CFR Chat Message"(
                      "recordType!CFR Chat Message.fields.user": loggedInUser(),
                      "recordType!CFR Chat Message.fields.role": "ASSISTANT",
                      "recordType!CFR Chat Message.fields.messageContent": local!val.text,
                      "recordType!CFR Chat Message.fields.createdOn": now()
                    )
                  }
                )
              },
              /* The stream is still sending text */
              {
                a!save(local!streamedData, local!val)
              }
            )
          }
        )
      }
    )
  )
)

How it works

The interface follows this sequence when a user sends a message:

The user's message appears immediately in the chat interface for instant feedback.
The message is stored in a local variable for later database persistence.
The language model begins streaming its response in real time.
As the response streams, the interface updates to show the text as it's generated.
When the response is complete, both the user's message and the AI's response are written to the record type in a single operation.
The next time the user opens the interface, their previous conversation loads automatically.

Key implementation details

Immediate UI updates: The interface adds the user's message to the local message list right away, so users see their message appear instantly without waiting for the database write.

Deferred database writes: The interface waits until the language model completes its response before writing to the database. This approach reduces unnecessary database operations during streaming and ensures both messages are saved together.

Streaming response handling: The streamingSaveInto parameter monitors the language model's response. When the done property is true, the interface knows the response is complete and triggers the database write.

Conversation pairing: By writing both the user's message and the AI's response in a single a!writeRecords() call, the interface maintains the correct chronological order of the conversation.

Tips and best practices

Limit the number of messages loaded from history to improve performance. The example loads the 20 most recent messages.
Consider adding error handling for cases where the language model call fails or times out.
Test the interface with various message lengths to ensure the streaming experience remains smooth.
Add a loading indicator or typing animation to show users when the AI is generating a response.
Implement conversation threading if users need to maintain multiple separate conversations.

Limitations

The interface loads a fixed number of previous messages. Users cannot scroll to view older conversations beyond this limit.
If multiple users share the same account, they will see each other's chat history.
The streaming response cannot be interrupted once started. Users must wait for the complete response before sending another message.

Feedback

Was this page helpful?