Class DecisionModelChatModelRouter

java.lang.Object
dev.langchain4j.model.chat.router.DecisionModelChatModelRouter
All Implemented Interfaces:
ChatModelRouter
@Experimental public class DecisionModelChatModelRouter extends Object implements ChatModelRouter
A ChatModelRouter that uses a DecisionModel to select the route: the decision model chooses the route whose description fits the last user message best. 
ChatModel chatModel = RoutingChatModel.builder()
        .route("simple", "Greetings, short factual questions, simple lookups", smallModel)
        .route("complex", "Multi-step reasoning, code, analysis", largeModel)
        .router(new DecisionModelChatModelRouter(decisionModel))
        .defaultRoute("complex")
        .build();
A route without a description is described by its name.

The decision model receives the last messages of the request (3 by default, see DecisionModelChatModelRouter.Builder.maxMessages(Integer)), up to and including the last user message, as sent to the chat model, so that short follow-ups such as "yes, go ahead" are understood. In an AI Service, the last user message is the one after the prompt template and retrieved content were added to it. Content other than text is represented by a marker, such as [attached image], so that a route whose description mentions images can be chosen for it.

The router selects the default route of the routing chat model (ChatModelRoutingResult.defaultRoute()) when the request contains no user message, and when the probability of the chosen route is below DecisionModelChatModelRouter.Builder.minProbability(Double). A minimum probability requires a decision model that reports probabilities: otherwise the call fails with an IllegalStateException, whatever the DecisionModelChatModelRouter.FallbackStrategy. When the decision model fails, the DecisionModelChatModelRouter.FallbackStrategy applies: by default, the default route is used and a warning is logged. When only one route can handle the request (for example, the only route supporting a JSON schema response format), that route is selected without calling the decision model.

routeAsync(ChatModelRoutingRequest) uses DecisionModel.decideAsync(DecisionRequest), so it fails with an AsyncNotSupportedException if the decision model does not support asynchronous calls.

Since:
1.21.0