Class DecisionModelFilteringToolProvider

java.lang.Object
dev.langchain4j.service.tool.DecisionModelFilteringToolProvider
All Implemented Interfaces:
ToolProvider
@Experimental public class DecisionModelFilteringToolProvider extends Object implements ToolProvider
A ToolProvider that passes on only the tools of another tool provider that are relevant to the conversation, as decided by a DecisionModel. This keeps requests to the LLM small when a tool provider offers many tools, for example an MCP server: 
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .toolProvider(DecisionModelFilteringToolProvider.builder()
                .toolProvider(mcpToolProvider)
                .decisionModel(decisionModel)
                .build())
        .build();
Unlike a ToolSearchStrategy, the tools are selected before the first LLM call, so no tool search round trip is needed. By default, every tool whose probability of being useful reaches the minimum probability is passed on, and the selection is based on the last 3 messages of the conversation (see DecisionModelFilteringToolProvider.Builder.maxMessages(Integer)), which helps with follow-up messages such as "do the same for Berlin".

Only the tools of the wrapped tool provider are filtered: tools configured directly on the AI Service, tools with the SearchBehavior.ALWAYS_VISIBLE search behavior and the tools configured with DecisionModelFilteringToolProvider.Builder.alwaysInclude(String...) are always passed on. Tools that were already called in the conversation are also always passed on, since some LLM providers reject requests whose messages contain calls to tools that are not in the request. This is similar to a ToolSearchStrategy, whose previously found tools stay available. The previous messages, used both for this and for DecisionModelFilteringToolProvider.Builder.maxMessages(Integer), are only known if the caller passes them in ToolProviderRequest.messages(), as LangChain4j AI Services do.

If the wrapped tool provider is dynamic, the tools are selected again, with a call to the decision model, before each LLM call of the tool-calling loop. Since the messages sent to the decision model usually do not change within a tool-calling loop, this only helps if the tools of the wrapped provider change.

The tools are selected for the user message as the user sent it: in AI Services, before retrieved content (RAG) and output format instructions were added to it (see InvocationContext.originalUserMessage()). Otherwise, the retrieved documents rather than the question would decide which tools are selected. The previous messages are sent as they are stored in the chat memory, which by default includes the retrieved content.

If the user message has no text, all tools are passed on. If the decision model fails, the DecisionModelFilteringToolProvider.FallbackStrategy applies: by default, all tools are passed on and a warning is logged.

Since:
1.21.0
See Also: