Class ToolService

java.lang.Object

dev.langchain4j.service.tool.ToolService

public class ToolService
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	ToolService.CombinedToolResults	The results of a round of tool executions, keeping successful and failed tools together so the caller can compensate the tools that did succeed before failing the invocation with the tool error.
static final record	ToolService.CompensableToolExecution
static final record	ToolService.NextChatRequest
static final record	ToolService.ToolResultsOutcome

Constructor Summary

Constructors

Constructor	Description
ToolService()

Method Summary

Modifier and Type	Method	Description
Consumer<ToolExecution>	afterToolExecution()
void	afterToolExecution(Consumer<ToolExecution> afterToolExecution)
ToolExecutionResult	applyToolHallucinationStrategy(ToolExecutionRequest toolRequest)
ToolArgumentsErrorHandler	argumentsErrorHandler()
void	argumentsErrorHandler(ToolArgumentsErrorHandler handler)
ToolArgumentsErrorHandler	asyncArgumentsErrorHandler()	The ToolArgumentsErrorHandler for the asynchronous AI Service modes (CompletableFuture, reactive Flow.Publisher): a user-configured handler if present, otherwise the async default, which sends the parsing error to the LLM so it can retry with corrected arguments.
ToolExecutionErrorHandler	asyncExecutionErrorHandler()	The ToolExecutionErrorHandler for the asynchronous AI Service modes (CompletableFuture, reactive Flow.Publisher): a user-configured handler if present, otherwise the async default, which fails the AI Service invocation (rather than hiding the error from the developer by sending it to the LLM).
Consumer<BeforeToolExecution>	beforeToolExecution()
void	beforeToolExecution(Consumer<BeforeToolExecution> beforeToolExecution)
String	collectCompensableRound(List<ToolExecutionRequest> toolExecutionRequests, Map<ToolExecutionRequest, ToolExecutionResult> toolResults, List<ToolService.CompensableToolExecution> compensableExecutions, InvocationContext invocationContext)	Records one completed round's successful compensable tool executions into the shared accumulator and returns the first errored tool's name (or null).
static CompletableFuture<ToolService.CombinedToolResults>	combineToolResultsCollectingErrors(Map<ToolExecutionRequest, CompletableFuture<ToolExecutionResult>> futures)	Combines a set of in-flight (possibly already-started) tool executions into a single future of their results, keyed and ordered by request.
void	compensateIfNeeded(List<ToolExecutionRequest> toolExecutionRequests, Map<ToolExecutionRequest, ToolExecutionResult> toolResults, List<ToolExecutionResultMessage> resultMessages, List<ToolService.CompensableToolExecution> compensableExecutions, boolean anyToolErrored, ChatMemory chatMemory, List<ChatMessage> messages, InvocationContext invocationContext, AiServiceListenerRegistrar listenerRegistrar)	Synchronous tool compensation, used by the synchronous / TokenStream modes.
CompletableFuture<Void>	compensateIfNeededAsync(List<ToolExecutionRequest> toolExecutionRequests, Map<ToolExecutionRequest, ToolExecutionResult> toolResults, List<ToolExecutionResultMessage> resultMessages, List<ToolService.CompensableToolExecution> compensableExecutions, boolean anyToolErrored, ChatMemory chatMemory, List<ChatMessage> messages, InvocationContext invocationContext, CompensationReason reason, AiServiceListenerRegistrar listenerRegistrar, BiConsumer<ToolExecution, CompensationReason> streamEmitter)	Non-blocking counterpart of compensateIfNeeded(List, Map, List, List, boolean, ChatMemory, List, InvocationContext, AiServiceListenerRegistrar), used by the CompletableFuture and reactive modes.
CompletableFuture<Void>	compensateIfNeededAsync(List<ToolExecutionRequest> toolExecutionRequests, Map<ToolExecutionRequest, ToolExecutionResult> toolResults, List<ToolExecutionResultMessage> resultMessages, List<ToolService.CompensableToolExecution> compensableExecutions, boolean anyToolErrored, ChatMemory chatMemory, List<ChatMessage> messages, InvocationContext invocationContext, CompensationReason reason, AiServiceListenerRegistrar listenerRegistrar, BiConsumer<ToolExecution, CompensationReason> streamEmitter, boolean alreadyCollected, String preCollectedFailedToolName)	As compensateIfNeededAsync(List, Map, List, List, boolean, ChatMemory, List, InvocationContext, CompensationReason, AiServiceListenerRegistrar, BiConsumer), but when alreadyCollected is true this round's compensable tools have already been recorded into compensableExecutions (via collectCompensableRound(List, Map, List, InvocationContext)) and preCollectedFailedToolName carries the collect's result; the collection step is then skipped.
CompletableFuture<Void>	compensateOnCancellationAsync(List<ToolExecutionRequest> currentRoundRequests, Map<ToolExecutionRequest, ToolExecutionResult> currentRoundResults, List<ToolService.CompensableToolExecution> compensableExecutions, ChatMemory chatMemory, List<ChatMessage> messages, InvocationContext invocationContext, AiServiceListenerRegistrar listenerRegistrar, BiConsumer<ToolExecution, CompensationReason> streamEmitter)	Rolls back, on cancellation, every compensable tool that had already executed successfully - across all rounds - implementing the "drain the round, then roll back" contract for the asynchronous and reactive modes.
void	compensateOnToolErrors(boolean compensateOnToolErrors)
ToolServiceContext	createContext(InvocationContext invocationContext, UserMessage userMessage, List<ChatMessage> messages)
Executor	effectiveToolExecutor()	Resolves the Executor that the asynchronous AI Service modes run tools on: the one set via executeToolsConcurrently(Executor) / executeToolsConcurrently(), or the default virtual-thread executor when none was set.
ToolServiceResult	executeInferenceAndToolsLoop(AiServiceContext context, Object memoryId, ChatResponse chatResponse, ChatRequestParameters parameters, List<ChatMessage> messages, ChatMemory chatMemory, InvocationContext invocationContext, ToolServiceContext toolServiceContext, boolean isReturnTypeResult)
ToolServiceResult	executeInferenceAndToolsLoop(AiServiceContext context, Object memoryId, ChatResponse chatResponse, ChatRequestParameters parameters, List<ChatMessage> messages, ChatMemory chatMemory, InvocationContext invocationContext, ToolServiceContext toolServiceContext, Function<ChatRequest, ChatResponse> chatModelInvoker)
CompletableFuture<ToolServiceResult>	executeInferenceAndToolsLoopAsync(AiServiceContext context, Object memoryId, ChatResponse chatResponse, ChatRequestParameters parameters, List<ChatMessage> messages, ChatMemory chatMemory, InvocationContext invocationContext, ToolServiceContext toolServiceContext, CompletableFuture<?> cancellation)	Non-blocking counterpart of executeInferenceAndToolsLoop(AiServiceContext, Object, ChatResponse, ChatRequestParameters, List, ChatMemory, InvocationContext, ToolServiceContext, boolean).
CompletableFuture<ToolServiceResult>	executeInferenceAndToolsLoopAsync(AiServiceContext context, Object memoryId, ChatResponse chatResponse, ChatRequestParameters parameters, List<ChatMessage> messages, ChatMemory chatMemory, InvocationContext invocationContext, ToolServiceContext toolServiceContext, CompletableFuture<?> cancellation, Function<ChatRequest, CompletableFuture<ChatResponse>> chatModelInvoker)	As executeInferenceAndToolsLoopAsync(AiServiceContext, Object, ChatResponse, ChatRequestParameters, List, ChatMemory, InvocationContext, ToolServiceContext, CompletableFuture), but with a caller-supplied asynchronous model invoker.
ToolExecutionResult	executeTool(InvocationContext invocationContext, Map<String, ToolExecutor> toolExecutors, ToolExecutionRequest toolRequest, Consumer<BeforeToolExecution> externalBeforeToolExecution, Consumer<ToolExecution> externalAfterToolExecution)
void	executeToolsConcurrently()
void	executeToolsConcurrently(Executor executor)
static ToolExecutionResult	executeWithErrorHandling(ToolExecutionRequest toolRequest, ToolExecutor toolExecutor, InvocationContext invocationContext, ToolArgumentsErrorHandler argumentsErrorHandler, ToolExecutionErrorHandler executionErrorHandler)
ToolExecutionErrorHandler	executionErrorHandler()
void	executionErrorHandler(ToolExecutionErrorHandler handler)
Executor	executor()
static List<AiServiceTool>	findTools(Object objectWithTools)	Scans the given object for @Tool-annotated methods and returns a list of AiServiceTools.
void	hallucinatedToolNameStrategy(Function<ToolExecutionRequest, ToolExecutionResultMessage> toolHallucinationStrategy)
boolean	isImmediateTool(String toolName)	Deprecated. use returnBehavior(String) instead
int	maxSequentialToolsInvocations()	Deprecated. Use maxToolCallingRoundTrips() instead.
void	maxSequentialToolsInvocations(int maxSequentialToolsInvocations)	Deprecated. Use maxToolCallingRoundTrips(int) instead.
int	maxToolCallingRoundTrips()
void	maxToolCallingRoundTrips(int maxToolCallingRoundTrips)
List<ToolService.CompensableToolExecution>	newCompensableExecutionsAccumulator()	Creates the accumulator the compensableExecutions are collected into across tool-calling rounds, or null when compensation is disabled.
void	onCompensableToolExecution(BiConsumer<ToolExecution, Consumer<ToolExecution>> onCompensableToolExecution)
void	onToolExecutionError(Consumer<InvocationContext> onToolExecutionError)
CompletionStage<List<ChatMessage>>	persistToolResultsAndResolveMessages(AiServiceContext context, ChatMemory chatMemory, List<ChatMessage> accumulator, List<ToolExecutionResultMessage> resultMessages, InvocationContext invocationContext)	Persists the tool-result messages and resolves the messages to send in the next request — the memory-touching counterpart to processToolResults(AiServiceContext, List, Map, List, InvocationContext, ToolServiceContext), kept out of that shared bookkeeping so that each AI Service mode can perform the memory I/O on the appropriate thread (synchronously for the sync/TokenStream modes, composed for the CompletableFuture/reactive modes).
List<ChatMessage>	persistToolResultsAndResolveMessagesSync(AiServiceContext context, ChatMemory chatMemory, List<ChatMessage> accumulator, List<ToolExecutionResultMessage> resultMessages, InvocationContext invocationContext)	Blocking counterpart of persistToolResultsAndResolveMessages(AiServiceContext, ChatMemory, List, List, InvocationContext) for the synchronous and TokenStream AI Service modes, which use the synchronous ChatMemory methods (so a memory backed by a blocking store that only implements the synchronous methods keeps working).
ToolService.NextChatRequest	prepareNextChatRequest(AiServiceContext context, Object memoryId, List<ChatMessage> messages, InvocationContext invocationContext, ToolServiceContext toolServiceContext, Map<ToolExecutionRequest, ToolExecutionResult> toolResults, ChatRequestParameters parameters)	Builds the next round's ChatRequest, shared by every AI Service mode: given the already-resolved messages to send (see resolveMessagesForNextRequest(List, AiServiceContext, InvocationContext)), it refreshes dynamic tool providers, folds in any tools found by tool search, overrides the tool specifications on the carried-forward parameters, applies the chat-request transformer, and fires the AiServiceRequestIssuedEvent.
ToolService.ToolResultsOutcome	processToolResults(AiServiceContext context, List<ToolExecutionRequest> toolExecutionRequests, Map<ToolExecutionRequest, ToolExecutionResult> toolResults, List<ToolExecution> toolExecutions, InvocationContext invocationContext, ToolServiceContext toolServiceContext)	Per-round bookkeeping shared by every AI Service mode (sync, CompletableFuture, TokenStream, reactive Flow.Publisher): for each executed tool it records a ToolExecution, fires the ToolExecutedEvent, collects the tool-result message (in request order), and accumulates anyToolErrored + the per-tool ReturnBehaviors.
static ToolServiceContext	refreshDynamicProviders(ToolServiceContext toolServiceContext, List<ChatMessage> messages, InvocationContext invocationContext)	Re-evaluates dynamic tool providers and returns an updated ToolServiceContext with any newly provided tool specifications and executors.
static List<ChatMessage>	resolveMessagesForNextRequest(List<ChatMessage> memoryMessages, AiServiceContext context, InvocationContext invocationContext)	Resolves the messages to send in the next request from the current memory view, honoring storeRetrievedContentInChatMemory (when false, the last user message is replaced with the original, un-augmented one so retrieved content is not persisted across rounds).
ReturnBehavior	returnBehavior(String toolName)	Returns the effective ReturnBehavior for the given tool, as configured on this service.
static boolean	shouldReturnImmediately(boolean anyToolErrored, List<ReturnBehavior> returnBehaviors)
CompletableFuture<ToolExecutionResult>	startTool(ToolExecutionRequest toolRequest, Map<String, ToolExecutor> toolExecutors, InvocationContext invocationContext, Consumer<BeforeToolExecution> externalBeforeToolExecution, Consumer<ToolExecution> externalAfterToolExecution, Executor executor)	Initiates a single tool execution on the given executor, returning a future of its result.
Map<String, ToolExecutor>	toolExecutors()
ToolProvider	toolProvider()	Deprecated. use toolProviders() instead
void	toolProvider(ToolProvider toolProvider)
Set<ToolProvider>	toolProviders()
void	toolProviders(Collection<ToolProvider> toolProviders)
void	tools(Collection<Object> objectsWithTools)
void	tools(List<AiServiceTool> tools)
void	tools(Map<ToolSpecification, ToolExecutor> tools)
void	tools(Map<ToolSpecification, ToolExecutor> tools, Set<String> immediateReturnToolNames)
void	toolSearchStrategy(ToolSearchStrategy toolSearchStrategy)
List<ToolSpecification>	toolSpecifications()

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ToolService

public ToolService()

Method Details

hallucinatedToolNameStrategy

public void hallucinatedToolNameStrategy(Function<ToolExecutionRequest, ToolExecutionResultMessage> toolHallucinationStrategy)

toolProvider

public void toolProvider(ToolProvider toolProvider)

toolProviders

public void toolProviders(Collection<ToolProvider> toolProviders)

Since:

1.12.0

tools

public void tools(Map<ToolSpecification, ToolExecutor> tools)

tools

public void tools(Map<ToolSpecification, ToolExecutor> tools,
 Set<String> immediateReturnToolNames)

tools

public void tools(Collection<Object> objectsWithTools)

tools

public void tools(List<AiServiceTool> tools)

Since:

1.14.0

findTools

public static List<AiServiceTool> findTools(Object objectWithTools)

Scans the given object for @Tool-annotated methods and returns a list of AiServiceTools.

Parameters:

objectWithTools - an object containing @Tool-annotated methods

Returns:

list of resolved tools

Throws:

IllegalConfigurationException - if the object has no @Tool-annotated methods

Since:

1.13.0

compensateOnToolErrors

public void compensateOnToolErrors(boolean compensateOnToolErrors)

executeToolsConcurrently

public void executeToolsConcurrently()

Since:

1.4.0

executeToolsConcurrently

public void executeToolsConcurrently(Executor executor)

Since:

1.4.0

effectiveToolExecutor

public Executor effectiveToolExecutor()

Resolves the Executor that the asynchronous AI Service modes run tools on: the one set via executeToolsConcurrently(Executor) / executeToolsConcurrently(), or the default virtual-thread executor when none was set. Never null, so async tools are always offloaded and never block the model-response thread. Pass a single-threaded executor to run them serially.

Since:

1.20.0

maxToolCallingRoundTrips

public void maxToolCallingRoundTrips(int maxToolCallingRoundTrips)

maxToolCallingRoundTrips

public int maxToolCallingRoundTrips()

maxSequentialToolsInvocations

@Deprecated(since="1.15.0")
public void maxSequentialToolsInvocations(int maxSequentialToolsInvocations)

Deprecated.

Use maxToolCallingRoundTrips(int) instead.

maxSequentialToolsInvocations

@Deprecated(since="1.15.0")
public int maxSequentialToolsInvocations()

Deprecated.

Use maxToolCallingRoundTrips() instead.

beforeToolExecution

public void beforeToolExecution(Consumer<BeforeToolExecution> beforeToolExecution)

Since:

1.11.0

beforeToolExecution

public Consumer<BeforeToolExecution> beforeToolExecution()

Since:

1.17.0

afterToolExecution

public void afterToolExecution(Consumer<ToolExecution> afterToolExecution)

Since:

1.11.0

afterToolExecution

public Consumer<ToolExecution> afterToolExecution()

Since:

1.17.0

onCompensableToolExecution

public void onCompensableToolExecution(BiConsumer<ToolExecution, Consumer<ToolExecution>> onCompensableToolExecution)

onToolExecutionError

public void onToolExecutionError(Consumer<InvocationContext> onToolExecutionError)

argumentsErrorHandler

public void argumentsErrorHandler(ToolArgumentsErrorHandler handler)

Since:

1.4.0

argumentsErrorHandler

public ToolArgumentsErrorHandler argumentsErrorHandler()

Since:

1.4.0

executionErrorHandler

public void executionErrorHandler(ToolExecutionErrorHandler handler)

Since:

1.4.0

executionErrorHandler

public ToolExecutionErrorHandler executionErrorHandler()

Since:

1.4.0

asyncArgumentsErrorHandler

public ToolArgumentsErrorHandler asyncArgumentsErrorHandler()

The ToolArgumentsErrorHandler for the asynchronous AI Service modes (CompletableFuture, reactive Flow.Publisher): a user-configured handler if present, otherwise the async default, which sends the parsing error to the LLM so it can retry with corrected arguments.

Since:

1.20.0

asyncExecutionErrorHandler

public ToolExecutionErrorHandler asyncExecutionErrorHandler()

The ToolExecutionErrorHandler for the asynchronous AI Service modes (CompletableFuture, reactive Flow.Publisher): a user-configured handler if present, otherwise the async default, which fails the AI Service invocation (rather than hiding the error from the developer by sending it to the LLM).

Since:

1.20.0

toolSearchStrategy

public void toolSearchStrategy(ToolSearchStrategy toolSearchStrategy)

Since:

1.12.0

createContext

public ToolServiceContext createContext(InvocationContext invocationContext,
 UserMessage userMessage,
 List<ChatMessage> messages)

executeInferenceAndToolsLoop

public ToolServiceResult executeInferenceAndToolsLoop(AiServiceContext context,
 Object memoryId,
 ChatResponse chatResponse,
 ChatRequestParameters parameters,
 List<ChatMessage> messages,
 ChatMemory chatMemory,
 InvocationContext invocationContext,
 ToolServiceContext toolServiceContext,
 boolean isReturnTypeResult)

executeInferenceAndToolsLoop

public ToolServiceResult executeInferenceAndToolsLoop(AiServiceContext context,
 Object memoryId,
 ChatResponse chatResponse,
 ChatRequestParameters parameters,
 List<ChatMessage> messages,
 ChatMemory chatMemory,
 InvocationContext invocationContext,
 ToolServiceContext toolServiceContext,
 Function<ChatRequest, ChatResponse> chatModelInvoker)

executeInferenceAndToolsLoopAsync

public CompletableFuture<ToolServiceResult> executeInferenceAndToolsLoopAsync(AiServiceContext context,
 Object memoryId,
 ChatResponse chatResponse,
 ChatRequestParameters parameters,
 List<ChatMessage> messages,
 ChatMemory chatMemory,
 InvocationContext invocationContext,
 ToolServiceContext toolServiceContext,
 CompletableFuture<?> cancellation)

Non-blocking counterpart of executeInferenceAndToolsLoop(AiServiceContext, Object, ChatResponse, ChatRequestParameters, List, ChatMemory, InvocationContext, ToolServiceContext, boolean).

Re-invokes the model via ChatModel.chatAsync(ChatRequest) and composes tool executions without blocking: no thread waits while a model response is in flight. When tools are configured to execute concurrently, their results are composed (not joined) as well. When no tool executor is configured, tools run on the thread that delivered the model response.

The cancellation future lets the caller stop the loop: once it is cancelled, no further model call or tool execution is initiated, and any in-flight model call is cancelled. A tool execution that has already started is not interrupted — it runs to completion and its result is simply discarded (Java cannot safely interrupt arbitrary tool code; this is a deliberate best-effort contract).

Since:

1.20.0

executeInferenceAndToolsLoopAsync

public CompletableFuture<ToolServiceResult> executeInferenceAndToolsLoopAsync(AiServiceContext context,
 Object memoryId,
 ChatResponse chatResponse,
 ChatRequestParameters parameters,
 List<ChatMessage> messages,
 ChatMemory chatMemory,
 InvocationContext invocationContext,
 ToolServiceContext toolServiceContext,
 CompletableFuture<?> cancellation,
 Function<ChatRequest, CompletableFuture<ChatResponse>> chatModelInvoker)

As executeInferenceAndToolsLoopAsync(AiServiceContext, Object, ChatResponse, ChatRequestParameters, List, ChatMemory, InvocationContext, ToolServiceContext, CompletableFuture), but with a caller-supplied asynchronous model invoker. Lets a streaming-only AI Service drive the same loop by bridging its StreamingChatModel to a CompletableFuture<ChatResponse>, instead of requiring a ChatModel.

Since:

1.20.0

processToolResults

public ToolService.ToolResultsOutcome processToolResults(AiServiceContext context,
 List<ToolExecutionRequest> toolExecutionRequests,
 Map<ToolExecutionRequest, ToolExecutionResult> toolResults,
 List<ToolExecution> toolExecutions,
 InvocationContext invocationContext,
 ToolServiceContext toolServiceContext)

Per-round bookkeeping shared by every AI Service mode (sync, CompletableFuture, TokenStream, reactive Flow.Publisher): for each executed tool it records a ToolExecution, fires the ToolExecutedEvent, collects the tool-result message (in request order), and accumulates anyToolErrored + the per-tool ReturnBehaviors. This bookkeeping is intentionally free of memory I/O — persisting the collected result messages (synchronously or composed) is left to each mode via persistToolResultsAndResolveMessages(AiServiceContext, ChatMemory, List, List, InvocationContext) / persistToolResultsAndResolveMessagesSync(AiServiceContext, ChatMemory, List, List, InvocationContext), so memory work happens on the appropriate thread. The tool execution itself and the delivery of intermediate responses are likewise left to each mode.

Since:

1.20.0

persistToolResultsAndResolveMessages

public CompletionStage<List<ChatMessage>> persistToolResultsAndResolveMessages(AiServiceContext context,
 ChatMemory chatMemory,
 List<ChatMessage> accumulator,
 List<ToolExecutionResultMessage> resultMessages,
 InvocationContext invocationContext)

Persists the tool-result messages and resolves the messages to send in the next request — the memory-touching counterpart to processToolResults(AiServiceContext, List, Map, List, InvocationContext, ToolServiceContext), kept out of that shared bookkeeping so that each AI Service mode can perform the memory I/O on the appropriate thread (synchronously for the sync/TokenStream modes, composed for the CompletableFuture/reactive modes).

When chatMemory is null, the result messages are appended to accumulator and that list is returned. When present, each result message is added to memory (sequentially, never concurrently, so the underlying read-modify-write stays ordered) and the resolved memory view is returned.

Since:

1.20.0

persistToolResultsAndResolveMessagesSync

public List<ChatMessage> persistToolResultsAndResolveMessagesSync(AiServiceContext context,
 ChatMemory chatMemory,
 List<ChatMessage> accumulator,
 List<ToolExecutionResultMessage> resultMessages,
 InvocationContext invocationContext)

Blocking counterpart of persistToolResultsAndResolveMessages(AiServiceContext, ChatMemory, List, List, InvocationContext) for the synchronous and TokenStream AI Service modes, which use the synchronous ChatMemory methods (so a memory backed by a blocking store that only implements the synchronous methods keeps working).

Since:

1.20.0

resolveMessagesForNextRequest

public static List<ChatMessage> resolveMessagesForNextRequest(List<ChatMessage> memoryMessages,
 AiServiceContext context,
 InvocationContext invocationContext)

Resolves the messages to send in the next request from the current memory view, honoring storeRetrievedContentInChatMemory (when false, the last user message is replaced with the original, un-augmented one so retrieved content is not persisted across rounds).

Since:

1.20.0

prepareNextChatRequest

public ToolService.NextChatRequest prepareNextChatRequest(AiServiceContext context,
 Object memoryId,
 List<ChatMessage> messages,
 InvocationContext invocationContext,
 ToolServiceContext toolServiceContext,
 Map<ToolExecutionRequest, ToolExecutionResult> toolResults,
 ChatRequestParameters parameters)

Builds the next round's ChatRequest, shared by every AI Service mode: given the already-resolved messages to send (see resolveMessagesForNextRequest(List, AiServiceContext, InvocationContext)), it refreshes dynamic tool providers, folds in any tools found by tool search, overrides the tool specifications on the carried-forward parameters, applies the chat-request transformer, and fires the AiServiceRequestIssuedEvent. Reading the messages from memory is left to each mode (synchronously or composed) so this method performs no memory I/O. Each mode supplies how the returned request is actually dispatched.

Since:

1.20.0

compensateIfNeeded

public void compensateIfNeeded(List<ToolExecutionRequest> toolExecutionRequests,
 Map<ToolExecutionRequest, ToolExecutionResult> toolResults,
 List<ToolExecutionResultMessage> resultMessages,
 List<ToolService.CompensableToolExecution> compensableExecutions,
 boolean anyToolErrored,
 ChatMemory chatMemory,
 List<ChatMessage> messages,
 InvocationContext invocationContext,
 AiServiceListenerRegistrar listenerRegistrar)

Synchronous tool compensation, used by the synchronous / TokenStream modes. Collects this round's successful compensable tools (accumulated across rounds) and, if any tool in this round errored, rolls them all back — running their compensating actions, rewriting the chat memory (synchronously), and rewriting this round's resultMessages in place — before the result messages are persisted. A compensating action that returns a CompletableFuture is awaited. Returns immediately when compensation is disabled.

Since:

1.20.0

compensateIfNeededAsync

public CompletableFuture<Void> compensateIfNeededAsync(List<ToolExecutionRequest> toolExecutionRequests,
 Map<ToolExecutionRequest, ToolExecutionResult> toolResults,
 List<ToolExecutionResultMessage> resultMessages,
 List<ToolService.CompensableToolExecution> compensableExecutions,
 boolean anyToolErrored,
 ChatMemory chatMemory,
 List<ChatMessage> messages,
 InvocationContext invocationContext,
 CompensationReason reason,
 AiServiceListenerRegistrar listenerRegistrar,
 BiConsumer<ToolExecution, CompensationReason> streamEmitter)

Non-blocking counterpart of compensateIfNeeded(List, Map, List, List, boolean, ChatMemory, List, InvocationContext, AiServiceListenerRegistrar), used by the CompletableFuture and reactive modes. The compensating actions and the chat-memory rewrite run without blocking the model-delivery thread: a compensating action that performs blocking I/O should return a CompletableFuture, and the rewrite uses ChatMemory.setAsync(List).

Since:

1.20.0

compensateIfNeededAsync

public CompletableFuture<Void> compensateIfNeededAsync(List<ToolExecutionRequest> toolExecutionRequests,
 Map<ToolExecutionRequest, ToolExecutionResult> toolResults,
 List<ToolExecutionResultMessage> resultMessages,
 List<ToolService.CompensableToolExecution> compensableExecutions,
 boolean anyToolErrored,
 ChatMemory chatMemory,
 List<ChatMessage> messages,
 InvocationContext invocationContext,
 CompensationReason reason,
 AiServiceListenerRegistrar listenerRegistrar,
 BiConsumer<ToolExecution, CompensationReason> streamEmitter,
 boolean alreadyCollected,
 String preCollectedFailedToolName)

As compensateIfNeededAsync(List, Map, List, List, boolean, ChatMemory, List, InvocationContext, CompensationReason, AiServiceListenerRegistrar, BiConsumer), but when alreadyCollected is true this round's compensable tools have already been recorded into compensableExecutions (via collectCompensableRound(List, Map, List, InvocationContext)) and preCollectedFailedToolName carries the collect's result; the collection step is then skipped. The reactive streaming path uses this so a round's compensable tools are collected before the inflight-round handoff, closing the window where a concurrent cancellation could snapshot the accumulator before the collection.

Since:

1.20.0

compensateOnCancellationAsync

public CompletableFuture<Void> compensateOnCancellationAsync(List<ToolExecutionRequest> currentRoundRequests,
 Map<ToolExecutionRequest, ToolExecutionResult> currentRoundResults,
 List<ToolService.CompensableToolExecution> compensableExecutions,
 ChatMemory chatMemory,
 List<ChatMessage> messages,
 InvocationContext invocationContext,
 AiServiceListenerRegistrar listenerRegistrar,
 BiConsumer<ToolExecution, CompensationReason> streamEmitter)

Rolls back, on cancellation, every compensable tool that had already executed successfully - across all rounds - implementing the "drain the round, then roll back" contract for the asynchronous and reactive modes. If the current (drained) round's requests/results are supplied, its successful compensable tools are recorded first so they roll back together with those accumulated from earlier rounds. Runs the compensating actions, rewrites the chat memory, clears the accumulator, and fires a ToolCompensatedEvent per rolled-back tool (reason CompensationReason.INVOCATION_CANCELLED). A no-op when compensation is disabled or nothing is pending.

Since:

1.20.0

collectCompensableRound

public String collectCompensableRound(List<ToolExecutionRequest> toolExecutionRequests,
 Map<ToolExecutionRequest, ToolExecutionResult> toolResults,
 List<ToolService.CompensableToolExecution> compensableExecutions,
 InvocationContext invocationContext)

Records one completed round's successful compensable tool executions into the shared accumulator and returns the first errored tool's name (or null). The caller MUST hold the monitor of accumulator across this call together with whatever "round is complete" handoff it performs (e.g. clearing the inflight round), so that a concurrent cancellation - which reads that handoff and snapshots the accumulator under the same monitor - observes this round's tools as present once it sees the round released. Used by the reactive streaming path; the result messages are derived the same way as the cancellation path (toResultMessage(ToolExecutionRequest, ToolExecutionResult)).

Since:

1.20.0

newCompensableExecutionsAccumulator

public List<ToolService.CompensableToolExecution> newCompensableExecutionsAccumulator()

Creates the accumulator the compensableExecutions are collected into across tool-calling rounds, or null when compensation is disabled.

Since:

1.20.0

shouldReturnImmediately

public static boolean shouldReturnImmediately(boolean anyToolErrored,
 List<ReturnBehavior> returnBehaviors)

refreshDynamicProviders

public static ToolServiceContext refreshDynamicProviders(ToolServiceContext toolServiceContext,
 List<ChatMessage> messages,
 InvocationContext invocationContext)

Re-evaluates dynamic tool providers and returns an updated ToolServiceContext with any newly provided tool specifications and executors.

Non-dynamic providers are not re-called — their tools remain unchanged. Tools returned by dynamic providers are only added, never removed: once a tool is present in the context, it stays for the remainder of the AI service invocation.

Since:

1.13.0

startTool

public CompletableFuture<ToolExecutionResult> startTool(ToolExecutionRequest toolRequest,
 Map<String, ToolExecutor> toolExecutors,
 InvocationContext invocationContext,
 Consumer<BeforeToolExecution> externalBeforeToolExecution,
 Consumer<ToolExecution> externalAfterToolExecution,
 Executor executor)

Initiates a single tool execution on the given executor, returning a future of its result. Used by the non-blocking streaming AI Service to start a tool as soon as its CompleteToolCall arrives (rather than waiting for the whole model response), so that concurrent tools overlap each other and the tail of the model stream.

Since:

1.20.0

combineToolResultsCollectingErrors

public static CompletableFuture<ToolService.CombinedToolResults> combineToolResultsCollectingErrors(Map<ToolExecutionRequest, CompletableFuture<ToolExecutionResult>> futures)

Combines a set of in-flight (possibly already-started) tool executions into a single future of their results, keyed and ordered by request. Unlike a short-circuiting combine, it does not stop on the first tool failure: it waits for all tool executions to settle and returns every result (a failed tool becomes an error result), together with the first failure in request order. This lets the asynchronous AI Service modes, whose default ToolExecutionErrorHandler rethrows execution errors, still run the compensating actions of the tools that succeeded before failing the invocation with the tool error. The given map's iteration order determines the result order, so pass a LinkedHashMap in request order.

Since:

1.20.0

executeTool

public ToolExecutionResult executeTool(InvocationContext invocationContext,
 Map<String, ToolExecutor> toolExecutors,
 ToolExecutionRequest toolRequest,
 Consumer<BeforeToolExecution> externalBeforeToolExecution,
 Consumer<ToolExecution> externalAfterToolExecution)

executeWithErrorHandling

public static ToolExecutionResult executeWithErrorHandling(ToolExecutionRequest toolRequest,
 ToolExecutor toolExecutor,
 InvocationContext invocationContext,
 ToolArgumentsErrorHandler argumentsErrorHandler,
 ToolExecutionErrorHandler executionErrorHandler)

applyToolHallucinationStrategy

public ToolExecutionResult applyToolHallucinationStrategy(ToolExecutionRequest toolRequest)

toolSpecifications

public List<ToolSpecification> toolSpecifications()

toolExecutors

public Map<String, ToolExecutor> toolExecutors()

executor

public Executor executor()

Since:

1.4.0

toolProviders

public Set<ToolProvider> toolProviders()

Since:

1.12.0

toolProvider

@Deprecated(since="1.12.0")
public ToolProvider toolProvider()

Deprecated.

use toolProviders() instead

returnBehavior

public ReturnBehavior returnBehavior(String toolName)

Returns the effective ReturnBehavior for the given tool, as configured on this service. Unknown tools and tools without an explicit behavior default to ReturnBehavior.TO_LLM.

Since:

1.14.0

isImmediateTool

@Deprecated(since="1.14.0")
public boolean isImmediateTool(String toolName)

Deprecated.

use returnBehavior(String) instead