Class JitLLMChatModel.JitLLMChatModelBuilder

java.lang.Object
dev.langchain4j.model.jitllm.JitLLMChatModel.JitLLMChatModelBuilder
Enclosing class:
JitLLMChatModel
public static final class JitLLMChatModel.JitLLMChatModelBuilder extends Object
Builder for JitLLMChatModel.

  • Method Details

    • modelPath

      public JitLLMChatModel.JitLLMChatModelBuilder modelPath(Path modelPath)
      Sets the path to the model file in GGUF format. Required.
      Parameters:
      modelPath - the path to the GGUF file
      Returns:
      this builder

    • contextLength

      public JitLLMChatModel.JitLLMChatModelBuilder contextLength(Integer contextLength)
      Sets the context window: the maximum number of tokens of the whole conversation (system message, history, tool definitions and the generated response). Memory for the whole window is reserved when the model is loaded. Default: 4096.
      Parameters:
      contextLength - the context window in tokens
      Returns:
      this builder

    • onGPU

      public JitLLMChatModel.JitLLMChatModelBuilder onGPU(Boolean onGPU)
      Sets whether the model runs on a GPU (true) or on the CPU (false).

      Running on a GPU requires the JVM to be started through TornadoVM with -Duse.tornadovm=true, otherwise building the model fails with an IllegalStateException. The GPU backend (CUDA, OpenCL or Metal) is the one provided by the installed TornadoVM SDK.

      Default: true if the JVM was started with -Duse.tornadovm=true, false otherwise.

      Parameters:
      onGPU - whether to run on a GPU
      Returns:
      this builder

    • temperature

      public JitLLMChatModel.JitLLMChatModelBuilder temperature(Double temperature)
      Sets the sampling temperature. Default: 0.1.
      Parameters:
      temperature - the temperature
      Returns:
      this builder

    • topP

      public JitLLMChatModel.JitLLMChatModelBuilder topP(Double topP)
      Sets the nucleus sampling probability. Default: 0.95.
      Parameters:
      topP - the nucleus sampling probability
      Returns:
      this builder

    • maxTokens

      public JitLLMChatModel.JitLLMChatModelBuilder maxTokens(Integer maxTokens)
      Sets the maximum number of tokens to generate per response, including the thinking. Default: 512.
      Parameters:
      maxTokens - the maximum number of tokens to generate
      Returns:
      this builder

    • stopSequences

      public JitLLMChatModel.JitLLMChatModelBuilder stopSequences(List<String> stopSequences)
      Sets the sequences that end the response when they appear in the answer. The response is cut before the sequence. The thinking of reasoning models is not checked.
      Parameters:
      stopSequences - the stop sequences
      Returns:
      this builder

    • seed

      public JitLLMChatModel.JitLLMChatModelBuilder seed(Integer seed)
      Sets the seed of the random sampling, to make responses reproducible. Default: a different seed for every request.
      Parameters:
      seed - the seed
      Returns:
      this builder

    • think

      public JitLLMChatModel.JitLLMChatModelBuilder think(Boolean think)
      Sets whether reasoning models think before answering: true enables thinking, false disables it, and when not set, the model's own default applies (Qwen 3, for example, thinks). Models that cannot think ignore this setting. Thinking improves answers to complex questions, but the thinking tokens count against maxTokens(Integer) and take time to generate.
      Parameters:
      think - whether the model thinks before answering
      Returns:
      this builder

    • returnThinking

      public JitLLMChatModel.JitLLMChatModelBuilder returnThinking(Boolean returnThinking)
      Sets whether the thinking of reasoning models (the text between <think> and </think>) is returned in AiMessage.thinking(). The thinking is never part of AiMessage.text(). Default: false.
      Parameters:
      returnThinking - whether to return the thinking
      Returns:
      this builder

    • defaultRequestParameters

      public JitLLMChatModel.JitLLMChatModelBuilder defaultRequestParameters(ChatRequestParameters defaultRequestParameters)
      Sets the parameters used for every request, unless the request sets them itself. Values set directly on this builder (for example temperature(Double)) take precedence.
      Parameters:
      defaultRequestParameters - the default request parameters
      Returns:
      this builder

    • listeners

      public JitLLMChatModel.JitLLMChatModelBuilder listeners(List<ChatModelListener> listeners)
      Sets the listeners notified about every request, response and error.
      Parameters:
      listeners - the listeners
      Returns:
      this builder

    • build

      public JitLLMChatModel build()
      Builds the model. This loads the model file, which can take a while for large models.
      Returns:
      the model