Package dev.langchain4j.model.jlama

Class JlamaStreamingLanguageModel

java.lang.Object

dev.langchain4j.model.jlama.JlamaStreamingLanguageModel

All Implemented Interfaces:

StreamingLanguageModel

public class JlamaStreamingLanguageModel
extends Object
implements StreamingLanguageModel

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	JlamaStreamingLanguageModel.JlamaStreamingLanguageModelBuilder

Constructor Summary

Constructors

Constructor	Description
JlamaStreamingLanguageModel(Path modelCachePath, String modelName, String authToken, Integer threadCount, Boolean quantizeModelAtRuntime, Path workingDirectory, com.github.tjake.jlama.safetensors.DType workingQuantizedType, Float temperature, Integer maxTokens)

Method Summary

Modifier and Type	Method	Description
static JlamaStreamingLanguageModel.JlamaStreamingLanguageModelBuilder	builder()
void	generate(String prompt, StreamingResponseHandler<String> handler)	Generates a response from the model based on a prompt.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.langchain4j.model.language.StreamingLanguageModel

generate

Constructor Details

JlamaStreamingLanguageModel

public JlamaStreamingLanguageModel(Path modelCachePath,
 String modelName,
 String authToken,
 Integer threadCount,
 Boolean quantizeModelAtRuntime,
 Path workingDirectory,
 com.github.tjake.jlama.safetensors.DType workingQuantizedType,
 Float temperature,
 Integer maxTokens)

Method Details

builder

public static JlamaStreamingLanguageModel.JlamaStreamingLanguageModelBuilder builder()

generate

public void generate(String prompt,
 StreamingResponseHandler<String> handler)

Description copied from interface: StreamingLanguageModel

Generates a response from the model based on a prompt.

Specified by:

generate in interface StreamingLanguageModel

Parameters:

prompt - The prompt.

handler - The handler for streaming the response.