Package dev.langchain4j.model.dashscope
Class QwenTokenizer
java.lang.Object
dev.langchain4j.model.dashscope.QwenTokenizer
- All Implemented Interfaces:
Tokenizer
-
Nested Class Summary
Nested Classes -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionbuilder()
int
estimateTokenCountInMessage
(ChatMessage message) Estimates the count of tokens in the given message.int
estimateTokenCountInMessages
(Iterable<ChatMessage> messages) Estimates the count of tokens in the given messages.int
Estimates the count of tokens in the given text.int
estimateTokenCountInToolExecutionRequests
(Iterable<ToolExecutionRequest> toolExecutionRequests) Estimates the count of tokens in the given tool execution requests.int
estimateTokenCountInToolSpecifications
(Iterable<ToolSpecification> toolSpecifications) Estimates the count of tokens in the given tool specifications.static boolean
isBlank
(CharSequence cs) Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface dev.langchain4j.model.Tokenizer
estimateTokenCountInForcefulToolExecutionRequest, estimateTokenCountInForcefulToolSpecification, estimateTokenCountInTools, estimateTokenCountInTools
-
Constructor Details
-
QwenTokenizer
-
-
Method Details
-
estimateTokenCountInText
Description copied from interface:Tokenizer
Estimates the count of tokens in the given text.- Specified by:
estimateTokenCountInText
in interfaceTokenizer
- Parameters:
text
- the text.- Returns:
- the estimated count of tokens.
-
estimateTokenCountInMessage
Description copied from interface:Tokenizer
Estimates the count of tokens in the given message.- Specified by:
estimateTokenCountInMessage
in interfaceTokenizer
- Parameters:
message
- the message.- Returns:
- the estimated count of tokens.
-
estimateTokenCountInMessages
Description copied from interface:Tokenizer
Estimates the count of tokens in the given messages.- Specified by:
estimateTokenCountInMessages
in interfaceTokenizer
- Parameters:
messages
- the messages.- Returns:
- the estimated count of tokens.
-
estimateTokenCountInToolSpecifications
Description copied from interface:Tokenizer
Estimates the count of tokens in the given tool specifications.- Specified by:
estimateTokenCountInToolSpecifications
in interfaceTokenizer
- Parameters:
toolSpecifications
- the tool specifications.- Returns:
- the estimated count of tokens.
-
estimateTokenCountInToolExecutionRequests
public int estimateTokenCountInToolExecutionRequests(Iterable<ToolExecutionRequest> toolExecutionRequests) Description copied from interface:Tokenizer
Estimates the count of tokens in the given tool execution requests.- Specified by:
estimateTokenCountInToolExecutionRequests
in interfaceTokenizer
- Parameters:
toolExecutionRequests
- the tool execution request.- Returns:
- the estimated count of tokens.
-
isBlank
-
builder
-