Interface ChatModelRouter

All Known Implementing Classes:

DecisionModelChatModelRouter

Functional Interface:

This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

@Experimental
@FunctionalInterface
public interface ChatModelRouter

Decides which route of a RoutingChatModel or RoutingStreamingChatModel handles a chat request.

A router can be a simple rule:

ChatModelRouter router = request -> ChatModelRoutingResult.route(
        request.chatRequest().messages().size() > 20 ? "complex" : "simple");

or use a model, such as DecisionModelChatModelRouter.

Since:

1.21.0

Method Summary

Modifier and Type	Method	Description
ChatModelRoutingResult	route(ChatModelRoutingRequest request)	Selects the route that handles the request.
default CompletableFuture<ChatModelRoutingResult>	routeAsync(ChatModelRoutingRequest request)	Non-blocking counterpart of route(ChatModelRoutingRequest), used by the asynchronous and streaming methods of the routing chat models.
default void	validate(List<ChatModelRoute> routes)	Checks that this router can route between the given routes, for example that every route has a description.

Method Details

route

ChatModelRoutingResult route(ChatModelRoutingRequest request)

Selects the route that handles the request.

Parameters:

request - the chat request to route and the routes that can handle it. Never null.

Returns:

one of the routes (ChatModelRoutingResult.route(String)), or the default route (ChatModelRoutingResult.defaultRoute()). Never null.

routeAsync

default CompletableFuture<ChatModelRoutingResult> routeAsync(ChatModelRoutingRequest request)

Non-blocking counterpart of route(ChatModelRoutingRequest), used by the asynchronous and streaming methods of the routing chat models. Routers that can route without blocking, such as routers that call a model asynchronously, override it.

The default returns a future failed with an AsyncNotSupportedException: the non-blocking calls of the routing chat models (chatAsync(...), and streaming to a Publisher) then fail. A router that decides without blocking, for example with simple rules, can implement this method by returning CompletableFuture.completedFuture(route(request)); a router that blocks, for example one that looks up the user in a database, can run route(ChatModelRoutingRequest) on an executor of its choice.

Parameters:

request - the chat request to route and the routes that can handle it. Never null.

Returns:

a future of the result, see route(ChatModelRoutingRequest).

validate

default void validate(List<ChatModelRoute> routes)

Checks that this router can route between the given routes, for example that every route has a description. Called when a RoutingChatModel or RoutingStreamingChatModel is created, so misconfigurations fail early. The default does nothing.

Parameters:

routes - all routes, in the order in which they were configured.

Throws:

IllegalArgumentException - if the router cannot route between these routes.