Interface StreamingLanguageModel

All Known Implementing Classes:
AzureOpenAiStreamingLanguageModel, DisabledStreamingLanguageModel, JlamaStreamingLanguageModel, LocalAiStreamingLanguageModel, MistralAiStreamingFimModel, OllamaStreamingLanguageModel, OpenAiStreamingLanguageModel
public interface StreamingLanguageModel
Represents a language model that has a simple text interface (as opposed to a chat interface) and can stream a response one token at a time. It is recommended to use the StreamingChatModel instead, as it offers more features.

  • Method Details

    • generate

      void generate(String prompt, StreamingResponseHandler<String> handler)
      Generates a response from the model based on a prompt.
      Parameters:
      prompt - The prompt.
      handler - The handler for streaming the response.

    • generate

      default void generate(Prompt prompt, StreamingResponseHandler<String> handler)
      Generates a response from the model based on a prompt.
      Parameters:
      prompt - The prompt.
      handler - The handler for streaming the response.