Package dev.langchain4j.model.jlama

Class JlamaStreamingChatModel

java.lang.Object

dev.langchain4j.model.jlama.JlamaStreamingChatModel

All Implemented Interfaces:

StreamingChatLanguageModel

public class JlamaStreamingChatModel
extends Object
implements StreamingChatLanguageModel

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	JlamaStreamingChatModel.JlamaStreamingChatModelBuilder

Constructor Summary

Constructors

Constructor	Description
JlamaStreamingChatModel(Path modelCachePath, String modelName, String authToken, Integer threadCount, Boolean quantizeModelAtRuntime, Path workingDirectory, com.github.tjake.jlama.safetensors.DType workingQuantizedType, Float temperature, Integer maxTokens)

Method Summary

Modifier and Type	Method	Description
static JlamaStreamingChatModel.JlamaStreamingChatModelBuilder	builder()
void	generate(List<ChatMessage> messages, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a sequence of messages.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.langchain4j.model.chat.StreamingChatLanguageModel

generate, generate, generate, generate

Constructor Details

JlamaStreamingChatModel

public JlamaStreamingChatModel(Path modelCachePath,
 String modelName,
 String authToken,
 Integer threadCount,
 Boolean quantizeModelAtRuntime,
 Path workingDirectory,
 com.github.tjake.jlama.safetensors.DType workingQuantizedType,
 Float temperature,
 Integer maxTokens)

Method Details

builder

public static JlamaStreamingChatModel.JlamaStreamingChatModelBuilder builder()

generate

public void generate(List<ChatMessage> messages,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a sequence of messages. Typically, the sequence contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

handler - The handler for streaming the response.