Interface ExecutorProvider

@Experimental
public interface ExecutorProvider

SPI for supplying the Executor that LangChain4j uses to offload blocking work and run its asynchronous continuations — concurrent tool execution, blocking RAG / retrieval and embedding-store calls, moderation, in-process embedding models, retry backoff scheduling, and similar.

Why implement this

LangChain4j hops threads at these offload points and does not propagate thread-local context (tracing spans, SLF4J MDC, security, CDI / request scope, …) across them by default. To make the context of the thread that calls an AI Service follow the work that the call offloads, implement captureContext().

A context-propagating executor (a MicroProfile ManagedExecutor, a Spring TaskDecorator, OpenTelemetry Context.taskWrapping(executor), Micrometer ContextSnapshot.wrap(executor), …) is not enough on its own: it captures the context of the thread that submits a task, and in a non-blocking AI Service call most work is submitted after the model has answered, from the thread that delivered the answer. Both can be combined: a task then runs with the context captured by captureContext(), applied inside whatever the executor sets up.

Executor, not ExecutorService

The return type is the minimal Executor: LangChain4j only needs to run tasks and never owns the executor's lifecycle (it will not call shutdown()). This accepts the widest range of host executors — every managed / context-propagating pool (Quarkus ManagedExecutor, Jakarta ManagedExecutorService, Guava ListeningExecutorService, a Spring TaskExecutor, …) is an Executor. Every offload point is driven through this single Executor; components that need a Future handle (to cancel or time-bound a task) obtain it via CompletableFuture.supplyAsync(..., executor) rather than requiring an ExecutorService.

Discovery and precedence

A provider is registered either by implementing this interface and declaring it for ServiceLoader (the standard way; frameworks typically do this for you), or programmatically via set(ExecutorProvider) (a convenience for tests and non-DI applications). The effective executor is resolved as:

1. a provider set programmatically via set(ExecutorProvider);
2. the first ExecutorProvider found on the classpath via ServiceLoader;
3. a built-in default: a virtual-thread-per-task executor on Java 21 or later, falling back to an unbounded platform-thread pool on Java 17-20.

Component-level executors (a retriever's, an AI Service's concurrent-tool executor, a transport's own executor(...) builder option, …) take precedence over this global default when explicitly set.

Contract

executor() is called at each offload, so it must return a shared, long-lived executor rather than create a new one per call. captureContext() has its own rules, see there.

This SPI intentionally exposes a single, global executor. Should per-purpose executors (e.g. separate pools for CPU-bound vs. blocking work) ever be needed, they can be added here as default methods that fall back to executor(), without breaking existing implementations.

Since:

1.20.0

Method Summary

Modifier and Type	Method	Description
default CapturedContext	captureContext()	Captures the context of the calling thread (tracing span, SLF4J MDC, security context, …), so that it can be restored in tasks that are submitted later, possibly from another thread.
Executor	executor()
static ExecutorProvider	get()
static void	set(ExecutorProvider provider)	Registers a process-wide ExecutorProvider programmatically, taking precedence over any ServiceLoader-discovered provider and the built-in default.

Method Details

executor

Executor executor()

Returns:

the shared Executor LangChain4j should offload blocking work and asynchronous continuations onto, or null to keep the built-in default (for a provider that only implements captureContext()).

captureContext

default CapturedContext captureContext()

Captures the context of the calling thread (tracing span, SLF4J MDC, security context, …), so that it can be restored in tasks that are submitted later, possibly from another thread.

LangChain4j calls this method once per AI Service invocation, in every mode, on the thread that calls the AI Service method (for methods returning a Flow.Publisher, Mono or Flux: when the method is called, not when the result is subscribed). It applies the returned CapturedContext to the tasks that the invocation offloads to an executor (tools, blocking RAG stages, moderation), whichever thread submits them. Work that runs without being offloaded (guardrails, chat memory, listeners, streaming callbacks) is not wrapped, and neither is work offloaded outside an AI Service invocation (retry backoff, threads that a model integration starts on its own, parallel and asynchronous agents of langchain4j-agentic).

This applies to AI Services created with AiServices. A framework that builds its own AI Service implementation decides itself how it propagates context.

Implementations must follow these rules:

• This method is called on every invocation, possibly on an event-loop thread: it must be cheap and must not block.
• The returned CapturedContext may wrap several tasks that run concurrently, and may be kept for as long as the invocation: it must not hold per-task state.
• A wrapped task must restore the previous context of the thread it runs on when it completes, normally or exceptionally. Tasks run on pooled threads, so a context left behind leaks into unrelated tasks, including another user's security context.

For example, with Micrometer context propagation, whose ContextSnapshot follows these rules:

private static final ContextSnapshotFactory CONTEXT_SNAPSHOT_FACTORY = ContextSnapshotFactory.builder().build();

public CapturedContext captureContext() {
    return CONTEXT_SNAPSHOT_FACTORY.captureAll()::wrap;
}

The default implementation captures nothing and returns CapturedContext.NONE.

Returns:

the context captured by this call. Must not be null.

Since:

1.23.0

set

static void set(ExecutorProvider provider)

Registers a process-wide ExecutorProvider programmatically, taking precedence over any ServiceLoader-discovered provider and the built-in default. This is a convenience for tests and non-DI applications; framework integrations typically register via ServiceLoader instead.

Example — run LangChain4j offloads on an application executor:

ExecutorProvider.set(() -> applicationExecutor);

To make the caller's context (tracing span, MDC, …) follow the offloaded work, register a provider that implements captureContext(); a lambda cannot.

Parameters:

provider - the provider to register, or null to clear a previously-set one (falling back to the ServiceLoader provider, then the built-in default).

Since:

1.20.0

get

static ExecutorProvider get()

Returns:

the ExecutorProvider previously registered via set(ExecutorProvider), or null if none was set (in which case a ServiceLoader provider or the built-in default is in effect). Does not return the ServiceLoader-discovered provider.

Since:

1.20.0