Class DecisionModelFilteringToolProvider

java.lang.Object
dev.langchain4j.service.tool.DecisionModelFilteringToolProvider
All Implemented Interfaces:
ToolProvider
@Experimental public class DecisionModelFilteringToolProvider extends Object implements ToolProvider
A ToolProvider that passes on only the tools of another tool provider that are relevant to the conversation, as decided by a DecisionModel. This keeps requests to the LLM small when a tool provider offers many tools, for example an MCP server: 
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .toolProvider(DecisionModelFilteringToolProvider.builder()
                .toolProvider(mcpToolProvider)
                .decisionModel(decisionModel)
                .build())
        .build();
Unlike a ToolSearchStrategy, the tools are selected before the first LLM call, so no tool search round trip is needed. By default, every tool whose probability of being useful reaches the minimum probability is passed on, and the selection is based on the system messages and the last 10 messages of the conversation, with tool calls and tool results (see DecisionModelFilteringToolProvider.Builder.maxMessages(Integer) and DecisionModelFilteringToolProvider.Builder.messageFilter(Predicate)), which helps with follow-up messages such as "do the same for Berlin". Long messages are shortened (see DecisionModelFilteringToolProvider.Builder.maxMessageLength(Integer)). The conversation is sent with every batch of tools (see DecisionModelFilteringToolProvider.Builder.maxToolsPerDecisionRequest(Integer)).

Only the tools of the wrapped tool provider are filtered: tools configured directly on the AI Service, tools with the SearchBehavior.ALWAYS_VISIBLE search behavior and the tools configured with DecisionModelFilteringToolProvider.Builder.alwaysInclude(String...) are always passed on. Tools that were already called in the conversation are also always passed on, since some LLM providers reject requests whose messages contain calls to tools that are not in the request. This is similar to a ToolSearchStrategy, whose previously found tools stay available. The previous messages, used both for this and for DecisionModelFilteringToolProvider.Builder.maxMessages(Integer), are only known if the caller passes them in ToolProviderRequest.messages(), as LangChain4j AI Services do.

If the wrapped tool provider is dynamic, the tools are selected again, with a call to the decision model, before each LLM call of the tool-calling loop. The decision model then also sees the tool calls and tool results of the previous rounds, so the selected tools can change from one LLM call to the next, which can also keep the LLM provider from reusing its prompt cache.

The tools are selected for the user message as the user sent it: in AI Services, before retrieved content (RAG) and output format instructions were added to it (see InvocationContext.originalUserMessage()). Otherwise, the retrieved documents rather than the question would decide which tools are selected. The previous messages are sent as they are stored in the chat memory, which by default includes the retrieved content.

If the user message has no text, all tools are passed on. If the decision model fails or refuses to answer, the DecisionModelFilteringToolProvider.FallbackStrategy applies: by default, all tools are passed on and a warning is logged.

Since:
1.21.0
See Also: