Interface ChatModelListener

All Known Implementing Classes:
MicrometerMetricsChatModelListener, ObservationChatModelListener
public interface ChatModelListener
A ChatModel listener that listens for requests, responses and errors.

Threading — these callbacks must not block. They are always invoked synchronously, on the model's own threads: on the synchronous API (chat) the caller's thread; on the asynchronous and reactive APIs (chatAsync and the streaming chat), onRequest(ChatModelRequestContext) runs on the thread that starts the call and onResponse(ChatModelResponseContext)/onError(ChatModelErrorContext) on the transport's I/O worker thread that reads the response (for HTTP models, the JDK HTTP client's HttpClient-* workers). Blocking in a callback — a synchronous database write, a synchronous HTTP call to an observability backend, synchronous file logging — stalls that worker and, under concurrency, degrades throughput for all in-flight calls, exactly as blocking in a reactive onNext would. Keep these methods non-blocking: record metrics or start/stop tracing spans (as the built-in Micrometer and observation listeners do); if a callback must perform blocking I/O, offload it to your own executor.

  • Method Summary

    Modifier and Type
    Method
    Description
    default void
    onError(ChatModelErrorContext errorContext)
    This method is called when an error occurs during interaction with the model.
    default void
    onRequest(ChatModelRequestContext requestContext)
    This method is called before the request is sent to the model.
    default void
    onResponse(ChatModelResponseContext responseContext)
    This method is called after the response is received from the model.

  • Method Details

    • onRequest

      default void onRequest(ChatModelRequestContext requestContext)
      This method is called before the request is sent to the model.
      Parameters:
      requestContext - The request context. It contains the ChatRequest and attributes. The attributes can be used to pass data between methods of this listener or between multiple listeners.

    • onResponse

      default void onResponse(ChatModelResponseContext responseContext)
      This method is called after the response is received from the model.
      Parameters:
      responseContext - The response context. It contains ChatResponse, corresponding ChatRequest and attributes. The attributes can be used to pass data between methods of this listener or between multiple listeners.

    • onError

      default void onError(ChatModelErrorContext errorContext)
      This method is called when an error occurs during interaction with the model.
      Parameters:
      errorContext - The error context. It contains the error, corresponding ChatRequest, partial ChatResponse (if available) and attributes. The attributes can be used to pass data between methods of this listener or between multiple listeners.