Package dev.langchain4j.model.localai

Class LocalAiStreamingChatModel

java.lang.Object

dev.langchain4j.model.localai.LocalAiStreamingChatModel

All Implemented Interfaces:

StreamingChatLanguageModel

public class LocalAiStreamingChatModel
extends Object
implements StreamingChatLanguageModel

See LocalAI documentation for more details.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	LocalAiStreamingChatModel.LocalAiStreamingChatModelBuilder

Constructor Summary

Constructors

Constructor	Description
LocalAiStreamingChatModel(String baseUrl, String modelName, Double temperature, Double topP, Integer maxTokens, Duration timeout, Boolean logRequests, Boolean logResponses)

Method Summary

Modifier and Type	Method	Description
static LocalAiStreamingChatModel.LocalAiStreamingChatModelBuilder	builder()
void	chat(ChatRequest chatRequest, StreamingChatResponseHandler handler)	This is the main API to interact with the chat model.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.langchain4j.model.chat.StreamingChatLanguageModel

chat, chat, defaultRequestParameters, doChat, listeners, provider, supportedCapabilities

Constructor Details

LocalAiStreamingChatModel

public LocalAiStreamingChatModel(String baseUrl,
 String modelName,
 Double temperature,
 Double topP,
 Integer maxTokens,
 Duration timeout,
 Boolean logRequests,
 Boolean logResponses)

Method Details

chat

public void chat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

Description copied from interface: StreamingChatLanguageModel

This is the main API to interact with the chat model.

A temporary default implementation of this method is necessary until all StreamingChatLanguageModel implementations adopt it. It should be removed once that occurs.

Specified by:

chat in interface StreamingChatLanguageModel

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

handler - a StreamingChatResponseHandler that will handle streaming response from the LLM

builder

public static LocalAiStreamingChatModel.LocalAiStreamingChatModelBuilder builder()