Class JudgedTextAssert

java.lang.Object
org.assertj.core.api.AbstractAssert<JudgedTextAssert, String>
dev.langchain4j.test.judge.JudgedTextAssert
All Implemented Interfaces:
org.assertj.core.api.Assert<JudgedTextAssert, String>, org.assertj.core.api.Descriptable<JudgedTextAssert>, org.assertj.core.api.ExtensionPoints<JudgedTextAssert, String>
@Experimental public final class JudgedTextAssert extends org.assertj.core.api.AbstractAssert<JudgedTextAssert, String>
Assertions on a text, checked by a Judge. Created by Judge.assertThat(String). Call given(String) or given(List), and basedOn(String...) or basedOnContents(Collection), if needed, before satisfies(String...), which can be called several times. Like other AssertJ assertions, it supports as(...) descriptions, and soft assertions through softly.check(() -> judge.assertThat(answer).satisfies(...)).

  • Field Summary

    Fields inherited from class org.assertj.core.api.AbstractAssert

    actual, info, myself, objects, throwUnsupportedExceptionOnEquals

  • Method Summary

    Modifier and Type
    Method
    Description
    JudgedTextAssert
    basedOn(String... sources)
    Gives the judge the sources the text should be based on, for example retrieved documents, so that it can check criteria such as "contains only claims that are stated in the sources".
    JudgedTextAssert
    basedOn(Collection<String> sources)
    Gives the judge the sources the text should be based on, for example retrieved documents, so that it can check criteria such as "contains only claims that are stated in the sources".
    JudgedTextAssert
    basedOnContents(Collection<? extends Content> sources)
    Gives the judge the sources the text should be based on, for example the sources retrieved by an AI Service (result.sources()), so that it can check criteria such as "contains only claims that are stated in the sources".
    JudgedTextAssert
    given(String userMessage)
    Gives the judge the text of the user message that the text responds to (for example userMessage.singleText()), so that it understands criteria that refer to the question.
    JudgedTextAssert
    given(List<? extends ChatMessage> conversation)
    Gives the judge the conversation that the text answers, so that it understands criteria that refer to it, for example "uses only facts from the tool results".
    JudgedTextAssert
    satisfies(JudgeCriterion... criteria)
    Asserts that the text satisfies every given criterion, according to the judge.
    JudgedTextAssert
    satisfies(String... criteria)
    Asserts that the text satisfies every given criterion, according to the judge.

    Methods inherited from class org.assertj.core.api.AbstractAssert

    actual, areEqual, asInstanceOf, asList, assertionError, asString, describedAs, descriptionText, doesNotHave, doesNotHaveSameClassAs, doesNotHaveSameHashCodeAs, doesNotHaveToString, doesNotHaveToString, doesNotMatch, doesNotMatch, equals, extracting, extracting, failure, failureWithActualExpected, failWithActualExpectedAndMessage, failWithMessage, getWritableAssertionInfo, has, hashCode, hasSameClassAs, hasSameHashCodeAs, hasToString, hasToString, inBinary, inHexadecimal, is, isElementOfCustomAssert, isEqualTo, isExactlyInstanceOf, isIn, isIn, isInstanceOf, isInstanceOfAny, isInstanceOfSatisfying, isNot, isNotEqualTo, isNotExactlyInstanceOf, isNotIn, isNotIn, isNotInstanceOf, isNotInstanceOfAny, isNotNull, isNotOfAnyClassIn, isNotSameAs, isNull, isOfAnyClassIn, isSameAs, matches, matches, newListAssertInstance, overridingErrorMessage, overridingErrorMessage, satisfies, satisfies, satisfies, satisfiesAnyOf, satisfiesAnyOf, satisfiesAnyOfForProxy, satisfiesForProxy, setCustomRepresentation, setDescriptionConsumer, setPrintAssertionsDescription, throwAssertionError, usingComparator, usingComparator, usingDefaultComparator, usingEquals, usingEquals, usingRecursiveAssertion, usingRecursiveAssertion, usingRecursiveComparison, usingRecursiveComparison, withFailMessage, withFailMessage, withRepresentation, withThreadDumpOnError

    Methods inherited from class Object

    clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.assertj.core.api.Descriptable

    as, as, as, describedAs, describedAs

  • Method Details

    • given

      public JudgedTextAssert given(String userMessage)
      Gives the judge the text of the user message that the text responds to (for example userMessage.singleText()), so that it understands criteria that refer to the question. Only the text is judged. Replaces a user message or conversation given before. Must be called before satisfies(...).
      Throws:
      IllegalStateException - if called after satisfies(...)

    • given

      public JudgedTextAssert given(List<? extends ChatMessage> conversation)
      Gives the judge the conversation that the text answers, so that it understands criteria that refer to it, for example "uses only facts from the tool results". For the answer of an AI Service, these are the messages of the last request it sent to the chat model: the system message, the messages from the chat memory, the user message with the retrieved contents, and the tool calls and their results. The requests can be collected by a ChatModelListener on the chat model of the AI Service (see ChatRequestAssert): 
      String answer = assistant.chat("Cancel booking MS-777");

Judge.from(judgeModel)
        .assertThat(answer)
        .given(requests.get(requests.size() - 1).messages())
        .satisfies("confirms what the cancellation tool returned");
      Only the text is judged. Images and other media of the conversation (audio, video, PDF files) are sent to the judge model too, so it must support them (see ChatModelJudge). Replaces a user message or conversation given before. Must be called before satisfies(...). A judge backed by a DecisionModel receives the conversation as messages, with the text as the last assistant message; it cannot combine a conversation with sources given by basedOn(...).
      Throws:
      IllegalStateException - if called after satisfies(...)

    • basedOn

      public JudgedTextAssert basedOn(String... sources)
      Gives the judge the sources the text should be based on, for example retrieved documents, so that it can check criteria such as "contains only claims that are stated in the sources". Must be called before satisfies(...).
      Throws:
      IllegalStateException - if called after satisfies(...)

    • basedOn

      public JudgedTextAssert basedOn(Collection<String> sources)
      Gives the judge the sources the text should be based on, for example retrieved documents, so that it can check criteria such as "contains only claims that are stated in the sources". Must be called before satisfies(...).
      Throws:
      IllegalStateException - if called after satisfies(...)

    • basedOnContents

      public JudgedTextAssert basedOnContents(Collection<? extends Content> sources)
      Gives the judge the sources the text should be based on, for example the sources retrieved by an AI Service (result.sources()), so that it can check criteria such as "contains only claims that are stated in the sources". Only the text of each source is given to the judge, and only the sources retrieved for this invocation: in a conversation, the answer can also be based on sources retrieved for earlier messages. Must be called before satisfies(...).
      Throws:
      IllegalStateException - if called after satisfies(...)

    • satisfies

      public JudgedTextAssert satisfies(String... criteria)
      Asserts that the text satisfies every given criterion, according to the judge. A criterion the judge is not sure about counts as not satisfied. Each call sends the text to the judge model. Without this call, nothing is judged. A lambda, as in satisfies(text -> ...), goes to AssertJ's own satisfies instead, which does not call the judge.
      Parameters:
      criteria - criteria written in natural language, for example "is polite" or "does not mention prices".
      Throws:
      AssertionError - listing each criterion that is not satisfied, with the judge's reasoning or probability.
      JudgeException - if the judge model returned a response that could not be understood, or refused to judge, or if the judge did not return one verdict per criterion. Exceptions thrown by the judge model are not caught.
      IllegalArgumentException - if the judge cannot judge this request, for example a DecisionModelJudge given both a conversation and sources; nothing is sent to the judge model then.

    • satisfies

      public JudgedTextAssert satisfies(JudgeCriterion... criteria)
      Asserts that the text satisfies every given criterion, according to the judge. Use this method instead of satisfies(String...) when a criterion defines when it passes and when it fails.
      Throws:
      AssertionError - listing each criterion that is not satisfied, with the judge's reasoning or probability.
      JudgeException - if the judge model returned a response that could not be understood, or refused to judge, or if the judge did not return one verdict per criterion. Exceptions thrown by the judge model are not caught.
      IllegalArgumentException - if the judge cannot judge this request, for example a DecisionModelJudge given both a conversation and sources; nothing is sent to the judge model then.