Class OpenAiPromptCacheBreakpoint

java.lang.Object
dev.langchain4j.model.openai.OpenAiPromptCacheBreakpoint
public class OpenAiPromptCacheBreakpoint extends Object
Marks the end of a cacheable prompt prefix for gpt-5.6 and later, which match a cache entry exactly at a breakpoint instead of falling back to a shorter unmarked prefix.

Use mark(ChatMessage) to mark a message. Since prompt caching is prefix-based, the prompt_cache_breakpoint is applied to the last content block of the marked message, so that everything up to and including that message forms the cached prefix: 

SystemMessage systemMessage = OpenAiPromptCacheBreakpoint.mark(SystemMessage.from(SHARED_INSTRUCTIONS));
Marking is also possible by hand, by putting MODE_EXPLICIT under the ATTRIBUTE_KEY attribute: 
SystemMessage systemMessage = SystemMessage.builder()
        .text(SHARED_INSTRUCTIONS)
        .attributes(Map.of(OpenAiPromptCacheBreakpoint.ATTRIBUTE_KEY,
                           OpenAiPromptCacheBreakpoint.MODE_EXPLICIT))
        .build();
Breakpoints can be placed on a SystemMessage, a UserMessage and a ToolExecutionResultMessage. AiMessage cannot carry one, because assistant output blocks are not among the block types OpenAI accepts a breakpoint on.

Each request supports up to four cache writes, one of which is consumed by OpenAiPromptCacheOptions.MODE_IMPLICIT.

Since:
1.21.0
See Also:

  • Field Details

    • ATTRIBUTE_KEY

      public static final String ATTRIBUTE_KEY
      The ChatMessage attribute key under which the breakpoint mode is stored. Do not change, it is part of the public API.
      See Also:

    • MODE_EXPLICIT

      public static final String MODE_EXPLICIT
      The only breakpoint mode currently accepted by OpenAI.
      See Also:

  • Method Details

    • mark

      public static <T extends ChatMessage> T mark(T message)
      Returns a copy of the given message marked as a prompt cache breakpoint, so that everything up to and including that message forms the cached prefix: 
      SystemMessage systemMessage = OpenAiPromptCacheBreakpoint.mark(SystemMessage.from(SHARED_INSTRUCTIONS));
      The given message is left untouched, and any other attributes it carries are preserved.
      Parameters:
      message - a SystemMessage, a UserMessage or a ToolExecutionResultMessage.
      Returns:
      a marked copy of the given message.
      Throws:
      UnsupportedFeatureException - if the message cannot carry a breakpoint, e.g. an AiMessage.
      Since:
      1.21.0

    • isMarked

      public static boolean isMarked(Map<String,Object> attributes)
      Whether the given attributes mark a message as a prompt cache breakpoint.
      Parameters:
      attributes - the ChatMessage attributes, may be null.
      Returns:
      true if a breakpoint should be emitted for this message.
      Throws:
      IllegalArgumentException - if the attribute is present but holds an unsupported value. OpenAI answers such a request with an HTTP 400.