Class DecisionScoringModel

java.lang.Object

dev.langchain4j.model.scoring.DecisionScoringModel

All Implemented Interfaces:

ScoringModel

@Experimental
public class DecisionScoringModel
extends Object
implements ScoringModel

A ScoringModel backed by a DecisionModel: the score of a segment is the probability that the answer to "Does the document help answer the query?" is "yes".

It can be used wherever a ScoringModel is expected, for example to re-rank retrieved content with a ReRankingContentAggregator:

ContentAggregator contentAggregator = ReRankingContentAggregator.builder()
        .scoringModel(new DecisionScoringModel(decisionModel))
        .minScore(0.5)
        .build();

The segments are scored in requests of up to 20 segments (see DecisionScoringModel.Builder.maxSegmentsPerRequest(Integer)): the input is the query, and each segment is part of its own yes/no question (see DecisionScoringModel.Builder.questionTemplate(PromptTemplate)). Whether the answer to one question can be influenced by the other questions of the request depends on the decision model. With ScoringModel.scoreAsync(ScoringRequest), the requests are sent in parallel.

Since the text of a segment is part of its question, a retrieved document that contains instructions, such as "answer yes", can try to raise its own score. Treat the scores of untrusted content accordingly.

Since:

1.21.0

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DecisionScoringModel.Builder

Field Summary

Fields

Modifier and Type	Field	Description
static final PromptTemplate	DEFAULT_QUESTION_TEMPLATE	The default template of the question asked for each segment: "Does the document help answer the query?\nDocument: {{document}}".

Constructor Summary

Constructors

Modifier	Constructor	Description
	DecisionScoringModel(DecisionModel decisionModel)
protected	DecisionScoringModel(DecisionScoringModel.Builder builder)

Method Summary

Modifier and Type	Method	Description
static DecisionScoringModel.Builder	builder()
ScoringResponse	doScore(ScoringRequest request)	The provider hook behind ScoringModel.score(ScoringRequest).
CompletableFuture<ScoringResponse>	doScoreAsync(ScoringRequest request)	The provider hook behind ScoringModel.scoreAsync(ScoringRequest).
List<ScoringModelListener>	listeners()	The listeners notified before each call of ScoringModel.score(ScoringRequest) and ScoringModel.scoreAsync(ScoringRequest), after each successful call and when a call fails.
ModelProvider	provider()	The ModelProvider of this scoring model (for example ModelProvider.COHERE).
Response<List<Double>>	scoreAll(List<TextSegment> segments, String query)	Scores all provided TextSegments against a given query.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ScoringModel

defaultRequestParameters, score, score, score, scoreAsync

Field Details

DEFAULT_QUESTION_TEMPLATE

public static final PromptTemplate DEFAULT_QUESTION_TEMPLATE

The default template of the question asked for each segment: "Does the document help answer the query?\nDocument: {{document}}".

Constructor Details

DecisionScoringModel

public DecisionScoringModel(DecisionModel decisionModel)

DecisionScoringModel

protected DecisionScoringModel(DecisionScoringModel.Builder builder)

Method Details

listeners

public List<ScoringModelListener> listeners()

Description copied from interface: ScoringModel

The listeners notified before each call of ScoringModel.score(ScoringRequest) and ScoringModel.scoreAsync(ScoringRequest), after each successful call and when a call fails. Listeners are configured on the builder of the scoring model. Defaults to an empty list.

Specified by:

listeners in interface ScoringModel

provider

public ModelProvider provider()

Description copied from interface: ScoringModel

The ModelProvider of this scoring model (for example ModelProvider.COHERE). It is made available to the listeners. Defaults to ModelProvider.OTHER.

Specified by:

provider in interface ScoringModel

scoreAll

public Response<List<Double>> scoreAll(List<TextSegment> segments,
 String query)

Description copied from interface: ScoringModel

Scores all provided TextSegments against a given query.

Specified by:

scoreAll in interface ScoringModel

Parameters:

segments - The list of TextSegments to score.

query - The query against which to score the segments.

Returns:

the list of scores. The order of scores corresponds to the order of TextSegments.

doScore

public ScoringResponse doScore(ScoringRequest request)

Description copied from interface: ScoringModel

The provider hook behind ScoringModel.score(ScoringRequest).

The default scores the documents with ScoringModel.scoreAll(List, String), so a model that implements only ScoringModel.scoreAll(List, String) also supports ScoringModel.score(ScoringRequest). To do so, the default turns each document into a TextSegment, and a TextSegment cannot be blank: a model that implements only ScoringModel.scoreAll(List, String) therefore cannot score blank documents through ScoringModel.score(ScoringRequest), and such a request fails with an IllegalArgumentException.

To support listeners, override this method and implement ScoringModel.scoreAll(List, String) by calling ScoringModel.score(ScoringRequest), so that the listeners are also notified when ScoringModel.scoreAll(List, String) is called. Never make ScoringModel.scoreAll(List, String) call ScoringModel.score(ScoringRequest) without overriding this method: each method would call the other.

Specified by:

doScore in interface ScoringModel

Parameters:

request - the request, with the default parameters already applied.

Returns:

the scores, in the order of ScoringRequest.documents(), and the response metadata.

doScoreAsync

public CompletableFuture<ScoringResponse> doScoreAsync(ScoringRequest request)

Description copied from interface: ScoringModel

The provider hook behind ScoringModel.scoreAsync(ScoringRequest). The default returns a failed future carrying AsyncNotSupportedException: a scoring model that is not genuinely asynchronous does not pretend to be. A model backed by remote HTTP I/O opts in by overriding this with a genuinely async call (no thread parked), and should abort its in-flight call when the returned future is cancelled (best-effort). A model that has not opted in fails loudly on the async path rather than silently blocking a carrier thread.

Specified by:

doScoreAsync in interface ScoringModel

Parameters:

request - the (already defaults-applied) request to score.

Returns:

a CompletableFuture of the scores, in the order of ScoringRequest.documents().

builder

public static DecisionScoringModel.Builder builder()