Class GPULlama3ResponseParser.StreamingParser

java.lang.Object
dev.langchain4j.model.gpullama3.GPULlama3ResponseParser.StreamingParser
Enclosing class:
GPULlama3ResponseParser
public static class GPULlama3ResponseParser.StreamingParser extends Object
Parser for handling streaming responses with real-time thinking content separation.

This parser detects thinking content as tokens are generated and routes it to the appropriate handler methods (onPartialThinking vs onPartialResponse). The thinking tags are preserved and streamed as part of the thinking content.

  • Constructor Details

    • StreamingParser

      public StreamingParser(StreamingChatResponseHandler handler, org.beehive.gpullama3.model.Model model)
      Creates a new streaming parser.
      Parameters:
      handler - the streaming response handler
      model - the GPULlama3 model instance for token decoding

  • Method Details

    • onToken

      public void onToken(int tokenId)
      Processes each token as it's generated by the model.
      Parameters:
      tokenId - the token ID generated by the model