Class RoutingChatModel

java.lang.Object
dev.langchain4j.model.chat.router.RoutingChatModel
All Implemented Interfaces:
ChatModel
@Experimental public class RoutingChatModel extends Object implements ChatModel
A ChatModel that sends each request to one of several chat models, as decided by a ChatModelRouter. It can be used wherever a ChatModel is expected (AI Services, agents, RAG), for example to send simple requests to a small, cheap model and complex ones to a larger model: 
ChatModel chatModel = RoutingChatModel.builder()
        .route("simple", "Greetings, short factual questions, simple lookups", smallModel)
        .route("complex", "Multi-step reasoning, code, analysis", largeModel)
        .router(new DecisionModelChatModelRouter(decisionModel))
        .defaultRoute("complex")
        .build();
The selected model handles the request as if it was called directly: its default parameters and listeners apply. Since the request can go to models of different providers, set only common parameters on requests (ChatRequestParameters), not provider-specific ones.

The name of the selected route is:

The rounds of a tool-calling loop stay on the same model: a request that ends with tool results goes to the route stored in the AiMessage that requested the tools, without asking the router. This also works when the messages are stored in a persistent chat memory, as long as it keeps the attributes of the messages.

supportedCapabilities() returns the capabilities supported by at least one route. A request that needs a capability (such as a JSON schema response format) is only routed to the routes that declare it: the router only sees those routes, and the default route is replaced by the first of them if it does not declare the capability. If no route declares the capability, all routes remain candidates, and the selected model accepts or rejects the request itself, as when it is called directly.

The routing chat model has no default request parameters of its own: ChatModel.defaultRequestParameters() returns empty parameters, and the default parameters of the selected model apply to each request.

The asynchronous method (ChatModel.chatAsync(ChatRequest)) selects the route with ChatModelRouter.routeAsync(ChatModelRoutingRequest), so it never blocks. A router that does not implement it, such as a router written as a lambda, fails the call with an AsyncNotSupportedException.

Since:
1.21.0
See Also: