Class JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder

java.lang.Object

dev.langchain4j.model.jitllm.JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder

Enclosing class:

JitLLMStreamingChatModel

public static final class JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder
extends Object

Builder for JitLLMStreamingChatModel.

Method Summary

Modifier and Type	Method	Description
JitLLMStreamingChatModel	build()	Builds the model.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	contextLength(Integer contextLength)	Sets the context window: the maximum number of tokens of the whole conversation (system message, history, tool definitions and the generated response).
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	defaultRequestParameters(ChatRequestParameters defaultRequestParameters)	Sets the parameters used for every request, unless the request sets them itself.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	listeners(List<ChatModelListener> listeners)	Sets the listeners notified about every request, response and error.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	maxTokens(Integer maxTokens)	Sets the maximum number of tokens to generate per response, including the thinking.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	modelPath(Path modelPath)	Sets the path to the model file in GGUF format.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	onGPU(Boolean onGPU)	Sets whether the model runs on a GPU (true) or on the CPU (false).
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	returnThinking(Boolean returnThinking)	Sets whether the thinking of reasoning models (the text between <think> and </think>) is returned in AiMessage.thinking() and streamed to StreamingChatResponseHandler.onPartialThinking(PartialThinking).
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	seed(Integer seed)	Sets the seed of the random sampling, to make responses reproducible.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	stopSequences(List<String> stopSequences)	Sets the sequences that end the response when they appear in the answer.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	temperature(Double temperature)	Sets the sampling temperature.
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	think(Boolean think)	Sets whether reasoning models think before answering: true enables thinking, false disables it, and when not set, the model's own default applies (Qwen 3, for example, thinks).
JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder	topP(Double topP)	Sets the nucleus sampling probability.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

modelPath

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder modelPath(Path modelPath)

Sets the path to the model file in GGUF format. Required.

Parameters:

modelPath - the path to the GGUF file

Returns:

this builder

contextLength

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder contextLength(Integer contextLength)

Sets the context window: the maximum number of tokens of the whole conversation (system message, history, tool definitions and the generated response). Memory for the whole window is reserved when the model is loaded. Default: 4096.

Parameters:

contextLength - the context window in tokens

Returns:

this builder

onGPU

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder onGPU(Boolean onGPU)

Sets whether the model runs on a GPU (true) or on the CPU (false).

Running on a GPU requires the JVM to be started through TornadoVM with -Duse.tornadovm=true, otherwise building the model fails with an IllegalStateException. The GPU backend (CUDA, OpenCL or Metal) is the one provided by the installed TornadoVM SDK.

Default: true if the JVM was started with -Duse.tornadovm=true, false otherwise.

Parameters:

onGPU - whether to run on a GPU

Returns:

this builder

temperature

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder temperature(Double temperature)

Sets the sampling temperature. Default: 0.1.

Parameters:

temperature - the temperature

Returns:

this builder

topP

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder topP(Double topP)

Sets the nucleus sampling probability. Default: 0.95.

Parameters:

topP - the nucleus sampling probability

Returns:

this builder

maxTokens

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder maxTokens(Integer maxTokens)

Sets the maximum number of tokens to generate per response, including the thinking. Default: 512.

Parameters:

maxTokens - the maximum number of tokens to generate

Returns:

this builder

stopSequences

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder stopSequences(List<String> stopSequences)

Sets the sequences that end the response when they appear in the answer. The response is cut before the sequence. The thinking of reasoning models is not checked.

Parameters:

stopSequences - the stop sequences

Returns:

this builder

seed

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder seed(Integer seed)

Sets the seed of the random sampling, to make responses reproducible. Default: a different seed for every request.

Parameters:

seed - the seed

Returns:

this builder

think

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder think(Boolean think)

Sets whether reasoning models think before answering: true enables thinking, false disables it, and when not set, the model's own default applies (Qwen 3, for example, thinks). Models that cannot think ignore this setting. Thinking improves answers to complex questions, but the thinking tokens count against maxTokens(Integer) and take time to generate.

Parameters:

think - whether the model thinks before answering

Returns:

this builder

returnThinking

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder returnThinking(Boolean returnThinking)

Sets whether the thinking of reasoning models (the text between <think> and </think>) is returned in AiMessage.thinking() and streamed to StreamingChatResponseHandler.onPartialThinking(PartialThinking). The thinking is never part of AiMessage.text(). Default: false.

Parameters:

returnThinking - whether to return the thinking

Returns:

this builder

defaultRequestParameters

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder defaultRequestParameters(ChatRequestParameters defaultRequestParameters)

Sets the parameters used for every request, unless the request sets them itself. Values set directly on this builder (for example temperature(Double)) take precedence.

Parameters:

defaultRequestParameters - the default request parameters

Returns:

this builder

listeners

public JitLLMStreamingChatModel.JitLLMStreamingChatModelBuilder listeners(List<ChatModelListener> listeners)

Sets the listeners notified about every request, response and error.

Parameters:

listeners - the listeners

Returns:

this builder

build

public JitLLMStreamingChatModel build()

Builds the model. This loads the model file, which can take a while for large models.

Returns:

the model