Class AiServiceInvocation

java.lang.Object
dev.langchain4j.test.service.AiServiceInvocation
@Experimental public final class AiServiceInvocation extends Object
What happened during one AI Service invocation: what it produced, the tools it executed, the sources it retrieved, the calls to models made during it (its own calls to its chat model and the tokens they used, and the calls of the components it runs), how it ended, and the AI Service invocations that its work started (for example its tools, guardrails or content retrievers).

Captured with AiServiceCapture (see AiServiceCapture.invocation()). Checked with LangChain4jAssertions.assertThat(...).

The tool assertions mark the tool executions they check, so that AiServiceInvocationAssert.hasNoMoreToolExecutions() can tell which ones were not checked. These marks are kept in the invocation, so do not share a capture between tests: capture in each test.

  • Method Summary

    Modifier and Type
    Method
    Description
    List<ChatModelCall>
    chatModelCalls()
    Returns the calls to chat models made during this invocation (see modelCalls()), those of the AI Service to its own chat model and those of the components it runs.
    List<DecisionModelCall>
    decisionModelCalls()
    Returns the calls to decision models made during this invocation (see modelCalls()).
    List<EmbeddingModelCall>
    embeddingModelCalls()
    Returns the calls to embedding models made during this invocation (see modelCalls()), for example to embed the query of a retrieval.
    Throwable
    error()
    Returns the error the invocation failed with, or null if it did not fail.
    FinishReason
    finishReason()
    Returns why the final response finished, or null if the model did not report it.
    ChatRequest
    firstRequest()
    Returns the first request that the chat model of the AI Service answered: the request of the first of the ownChatModelCalls() that returned a response.
    InvocationContext
    invocationContext()
    Returns the context of the invocation: its id, the AI Service interface and method, the method arguments and more.
    boolean
    isCancelled()
    Returns whether the invocation was cancelled before it completed.
    ChatRequest
    lastRequest()
    Returns the last request that the chat model of the AI Service answered: the request of the last of the ownChatModelCalls() that returned a response.
    ChatResponse
    lastResponse()
    Returns the last response of the chat model of the AI Service.
    List<ModelCall>
    modelCalls()
    Returns the calls to models made during this invocation, in the order they started: those of the AI Service to its own chat model (see ownChatModelCalls()) and those of the components it runs, for example a query transformer, an embedding model used for retrieval, a scoring model, a moderation model or a judge in a guardrail.
    List<ModerationModelCall>
    moderationModelCalls()
    Returns the calls to moderation models made during this invocation (see modelCalls()).
    List<AiServiceInvocation>
    nestedInvocations()
    Returns the AI Service invocations started by the work of this invocation (for example by its tools, guardrails or content retrievers), in the order they started.
    Object
    output()
    Returns what the AI Service method produced: for a method that returns a value directly, that value, for example a String or the object of a method with structured output; for a method that returns a Result, a CompletableFuture or another asynchronous value, the value inside it; for a streaming method (one that returns a TokenStream or a publisher such as a Flux), the text of the final response.
    String
    outputText()
    Returns the text produced by the AI Service method: the answer of a method that returns a String (also inside a Result or a CompletableFuture), or the text of the final response of a streaming method (see output()).
    List<ChatModelCall>
    ownChatModelCalls()
    Returns the calls of the AI Service to its own chat model, in the order they started: for example one call, then one more after each round of tool executions, or after an output guardrail asked the model again.
    List<ScoringModelCall>
    scoringModelCalls()
    Returns the calls to scoring models made during this invocation (see modelCalls()), for example to re-rank the retrieved contents.
    List<Content>
    sources()
    Returns the contents retrieved for retrieval-augmented generation (RAG) during this invocation, empty if nothing was retrieved.
    TokenUsage
    tokenUsage()
    Returns the tokens used, summed over the calls that the AI Service itself made to its chat model during the invocation (for example, before and after executing tools).
    List<ToolExecution>
    toolExecutions()
    Returns the executed tools, in the order of execution, also those whose execution failed (see ToolExecution.hasFailed()).
    String
    toString()
     

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Method Details

    • invocationContext

      public InvocationContext invocationContext()
      Returns the context of the invocation: its id, the AI Service interface and method, the method arguments and more.

    • output

      public Object output()
      Returns what the AI Service method produced:
      • for a method that returns a value directly, that value, for example a String or the object of a method with structured output;
      • for a method that returns a Result, a CompletableFuture or another asynchronous value, the value inside it;
      • for a streaming method (one that returns a TokenStream or a publisher such as a Flux), the text of the final response. Text that the model streamed in earlier responses, before it called tools, is not included: to check everything that was streamed, collect it with onPartialResponse(...), or see AiServiceCapture.lambdaOutputItems() when the lambda returns a publisher.
      Throws:
      AssertionError - if the invocation failed or was cancelled, or if the AI Service method returns void

    • outputText

      public String outputText()
      Returns the text produced by the AI Service method: the answer of a method that returns a String (also inside a Result or a CompletableFuture), or the text of the final response of a streaming method (see output()). null if such a method produced no text, for example a streaming method whose final response has only tool calls.
      Throws:
      AssertionError - if the invocation failed or was cancelled, or if the method does not produce text, for example a method with structured output (use output() then) or a method that returns nothing

    • firstRequest

      public ChatRequest firstRequest()
      Returns the first request that the chat model of the AI Service answered: the request of the first of the ownChatModelCalls() that returned a response. It holds the messages the AI Service built for the model (for example the system message, the messages from the chat memory and the user message, with the retrieved contents) and the tools it offered, with the default request parameters of the chat model (for example the model name or the temperature set on the model).
      Throws:
      AssertionError - if the chat model did not answer any request (for example when the invocation failed before), or if it did not notify its listeners (see ownChatModelCalls())

    • lastRequest

      public ChatRequest lastRequest()
      Returns the last request that the chat model of the AI Service answered: the request of the last of the ownChatModelCalls() that returned a response. When the AI Service executed tools, it holds the whole conversation that led to the final answer: the messages of the first request, the tool calls of the model and the tool results.
      Throws:
      AssertionError - if the chat model did not answer any request (for example when the invocation failed before), or if it did not notify its listeners (see ownChatModelCalls())

    • lastResponse

      public ChatResponse lastResponse()
      Returns the last response of the chat model of the AI Service. When the AI Service executed tools, this is the response after the last tool results, usually the final answer. Its token usage is that of this call only, also for streaming methods; see tokenUsage() for all the calls.
      Throws:
      AssertionError - if the chat model did not answer any request (for example when the invocation failed before), or if it did not notify its listeners (see ownChatModelCalls())

    • toolExecutions

      public List<ToolExecution> toolExecutions()
      Returns the executed tools, in the order of execution, also those whose execution failed (see ToolExecution.hasFailed()). The tool executions of nested invocations are not included.
      Throws:
      AssertionError - if the AI Service did not report whether its tool executions failed (see ToolExecutedEvent.isError()): ToolExecution.hasFailed() would then be false for every execution

    • sources

      public List<Content> sources()
      Returns the contents retrieved for retrieval-augmented generation (RAG) during this invocation, empty if nothing was retrieved.
      Throws:
      AssertionError - if the AI Service has no retrieval augmentor, or did not report what it retrieved

    • tokenUsage

      public TokenUsage tokenUsage()
      Returns the tokens used, summed over the calls that the AI Service itself made to its chat model during the invocation (for example, before and after executing tools). Models called by other components are not included, for example by a query transformer, a query router, a guardrail or an embedding model, and neither are the tokens of nested invocations. null if the model did not report them, or if no call of the chat model was captured (see ownChatModelCalls()).

    • finishReason

      public FinishReason finishReason()
      Returns why the final response finished, or null if the model did not report it.

    • error

      public Throwable error()
      Returns the error the invocation failed with, or null if it did not fail.

    • isCancelled

      public boolean isCancelled()
      Returns whether the invocation was cancelled before it completed.

    • modelCalls

      public List<ModelCall> modelCalls()
      Returns the calls to models made during this invocation, in the order they started: those of the AI Service to its own chat model (see ownChatModelCalls()) and those of the components it runs, for example a query transformer, an embedding model used for retrieval, a scoring model, a moderation model or a judge in a guardrail. The calls of nested invocations are not included, and neither are the calls of models that bypass the listeners LangChain4j notifies (see ModelCall).

    • ownChatModelCalls

      public List<ChatModelCall> ownChatModelCalls()
      Returns the calls of the AI Service to its own chat model, in the order they started: for example one call, then one more after each round of tool executions, or after an output guardrail asked the model again. Each request is the one the chat model received, with its default request parameters. Calls that failed are included (see ModelCall.hasFailed()).

      They are captured from the listeners of the chat model, so the list is empty for a chat model that does not notify its listeners, for example one that overrides chat(...) instead of implementing doChat(...).

    • chatModelCalls

      public List<ChatModelCall> chatModelCalls()
      Returns the calls to chat models made during this invocation (see modelCalls()), those of the AI Service to its own chat model and those of the components it runs.

    • embeddingModelCalls

      public List<EmbeddingModelCall> embeddingModelCalls()
      Returns the calls to embedding models made during this invocation (see modelCalls()), for example to embed the query of a retrieval.

    • scoringModelCalls

      public List<ScoringModelCall> scoringModelCalls()
      Returns the calls to scoring models made during this invocation (see modelCalls()), for example to re-rank the retrieved contents.

    • moderationModelCalls

      public List<ModerationModelCall> moderationModelCalls()
      Returns the calls to moderation models made during this invocation (see modelCalls()).

    • decisionModelCalls

      public List<DecisionModelCall> decisionModelCalls()
      Returns the calls to decision models made during this invocation (see modelCalls()).

    • nestedInvocations

      public List<AiServiceInvocation> nestedInvocations()
      Returns the AI Service invocations started by the work of this invocation (for example by its tools, guardrails or content retrievers), in the order they started. An integration that does not report which invocation started another one (see InvocationContext.parentInvocationId()) has no nested invocations: the invocations it starts on the capture thread are captured as separate ones.

    • toString

      public String toString()
      Overrides:
      toString in class Object