Class MockStreamingChatModel

java.lang.Object
dev.langchain4j.test.mock.MockStreamingChatModel
All Implemented Interfaces:
StreamingChatModel
@Experimental public class MockStreamingChatModel extends Object implements StreamingChatModel
A StreamingChatModel for tests: it needs no network, streams the answers the test tells it to, and records the requests it receives.

Script the answers step by step, one step per request. For example, a model that first calls a tool and then streams text: 

MockStreamingChatModel model = MockStreamingChatModel.thatCallsTool("cancelBooking", Map.of("bookingNumber", "MS-777"))
        .thenResponds("Booking MS-777 is cancelled.");
To control exactly which tokens are streamed, use thatStreams(String...). To test a stream that breaks, let a response fail after its tokens: 
MockStreamingChatModel model = MockStreamingChatModel.thatStreams("Hel", "lo")
        .andFails(new RuntimeException("connection reset"));
Every that... method creates a mock with its first step (one step per message for the message-list forms), and every then... method adds a step. Each step answers exactly one request. A request after the last step gets an IllegalStateException in onError that shows the request, so a test notices when the code calls the model more often than the script expects, unless the code under test handles the error itself. To answer every further request with the last step, end the script with thenRepeats(), for example MockStreamingChatModel.thatResponds("Hello!").thenRepeats(). A step can compute its answer from the request with thatResponds(Function).

An answer is streamed the way a provider streams it: first its thinking, then its text, then its tool calls, each tool call as partial tool calls followed by the complete tool call. Text, thinking and tool call arguments are streamed in word-like tokens, the way a tokenizer splits text before it encodes it: a word with the space before it, up to three digits of a number, a run of punctuation, or whitespace. So "Hello, world!" is streamed as "Hello", ",", " world" and "!", and "Order 12345" as "Order", " ", "123" and "45". Chinese and Japanese text, written without spaces between words, is streamed one character per token. The complete response reports what a provider reports: the model name of the request, and the finish reason FinishReason.TOOL_EXECUTION for an answer with tool calls, otherwise FinishReason.STOP. To report token usage in every response, use withTokenUsage(TokenUsage). To report anything else, for example FinishReason.LENGTH or a different token usage per step, give the whole response: thatResponds(ChatResponse).

By default, the mock reports no capabilities. To test code that checks the capabilities of a model, for example a RoutingStreamingChatModel that sends a request with a JSON schema response format only to the models that support it, use withSupportedCapabilities(Capability.RESPONSE_FORMAT_JSON_SCHEMA).

Like a provider, the mock produces its answer on its own thread. chat(request, handler) returns immediately and calls the handler on another thread, also for a failure, so a test has to wait for onCompleteResponse or onError, for example with a CompletableFuture. The Publisher returned by chat(request) produces its events on a new thread for each subscription and delivers each one only after the subscriber requested it, either on that thread or, for an event that was waiting, on the thread that requests it. Each subscription to that Publisher is one request: the mock records the request and takes the next step when the Publisher is subscribed to, not when chat(request) returns, so a retry that subscribes again gets the next step.

A mock remembers which steps were taken and which requests it received, so create a new one for each test. A mock shared between tests, for example as a bean, fails with a clear message as soon as its script runs out, unless the script ends with thenRepeats(). The steps are taken in the order the requests arrive. A mock keeps every request it receives, so it is not meant for load tests or other long runs.