Interface ToolErrorVisibleToLlm

All Known Implementing Classes:
LlmVisibleToolExecutionException, McpApplicationErrorException, ToolErrorVisibleToLlmException
public interface ToolErrorVisibleToLlm
Marks an exception whose content is meant to be seen by the LLM.

By default, when a tool throws an exception, LangChain4j decides what the LLM is told about it. By implementing this interface, you take that decision yourself for a particular exception: messageForLlm() is sent to the LLM as the result of the tool execution, and the LLM can react to it, for example by trying something else or by explaining the problem to the user. With ToolExecutionErrorHandler.failInvocationUnlessVisibleToLlm(), exceptions that do not implement it are treated as failures of the application and are not shown to the LLM.

Implement it on your own exception: 

public class OrderNotFoundException extends RuntimeException implements ToolErrorVisibleToLlm {

    private final String orderId;

    public OrderNotFoundException(String orderId) {
        super("Order " + orderId + " not found");
        this.orderId = orderId;
    }

    @Override
    public String messageForLlm() {
        return "There is no order with ID " + orderId + ". Ask the user to check the order number.";
    }
}

Or, when you do not want to declare an exception class, throw a ready-made one with from(String) or from(String, Throwable): 

@Tool("Returns the status of an order")
String orderStatus(String orderId) {
    try {
        return orderService.status(orderId);
    } catch (SQLException e) {
        // the LLM is told only what it needs to know; the cause is not sent to it
        log.warn("Could not read the order database", e);
        throw ToolErrorVisibleToLlm.from("The order database is temporarily unavailable.", e);
    }
}

The marker is looked for on the exception as it was thrown by the tool, and on the error the handler receives after LangChain4j has unwrapped its own wrappers. It is not searched for further down the cause chain: an exception that wraps a marked one is the last word on what the LLM should be told, so wrapping a marked exception deliberately hides it.

The handler used when no handler is configured on AiServices, as well as ToolExecutionErrorHandler.sendExceptionMessageToLlm(), ToolExecutionErrorHandler.failInvocationUnlessVisibleToLlm() and ToolExecutionErrorHandler.sendGenericMessageToLlmUnlessVisibleToLlm(String), send messageForLlm() to the LLM. ToolExecutionErrorHandler.failInvocation() fails the invocation regardless of this interface. A handler you write yourself decides for itself whether to look at it.

Frameworks that build AI Services themselves, such as Quarkus, may use a default handler of their own that does not look at this interface. There, configure one of the handlers above explicitly.

Note that ToolErrorVisibleToLlmException is a plain RuntimeException and not a ToolExecutionException: the latter is the wrapper LangChain4j puts around a failing tool, while this one is thrown by the tool itself, so catch (ToolExecutionException e) does not catch it.

Since:
1.21.0

  • Method Summary

    Modifier and Type
    Method
    Description
    static ToolErrorVisibleToLlmException
    from(String message)
    Creates an exception that carries the given message to the LLM.
    static ToolErrorVisibleToLlmException
    from(String message, Throwable cause)
    Creates an exception that carries the given message to the LLM, keeping cause so that the technical details are still available in your logs.
    String
    messageForLlm()
    The text that is sent to the LLM as the result of the failed tool execution.

  • Method Details

    • messageForLlm

      String messageForLlm()
      The text that is sent to the LLM as the result of the failed tool execution.

      Write it for the LLM, not for a log file: say what went wrong and, when it helps, what the LLM could do about it. Do not pass the message of another exception through (for example return cause.getMessage()): such messages are written for developers and often contain internal details that should not reach the LLM provider.

      Must not be blank. A blank text is ignored (and a warning is logged), and the exception is treated as if it did not implement this interface: a handler that fails the invocation fails it, and a handler that sends the message of the exception to the LLM sends that instead.

    • from

      static ToolErrorVisibleToLlmException from(String message)
      Creates an exception that carries the given message to the LLM.
      Parameters:
      message - the text to send to the LLM, written for the LLM. Must not be blank.
      Returns:
      the exception, to be thrown by the tool
      Throws:
      IllegalArgumentException - if message is blank

    • from

      static ToolErrorVisibleToLlmException from(String message, Throwable cause)
      Creates an exception that carries the given message to the LLM, keeping cause so that the technical details are still available in your logs.
      Parameters:
      message - the text to send to the LLM, written for the LLM. Must not be blank.
      cause - the original error. It is not sent to the LLM. The handlers you configure do not log it, since the AI Service invocation continues normally once the error is handled, but the handler used by synchronous AI Services when none is configured logs every tool failure, including this cause, at WARN level. Log it yourself, before throwing, if you need it in your own logs.
      Returns:
      the exception, to be thrown by the tool
      Throws:
      IllegalArgumentException - if message is blank