Skip to main content

OpenAI Image Models

note

This is the documentation for the OpenAI integration, that uses a custom Java implementation of the OpenAI REST API, that works best with Quarkus (as it uses the Quarkus REST client) and Spring (as it uses Spring's RestClient).

LangChain4j provides 3 different integrations with OpenAI for generating images, and this is #1 :

  • OpenAI uses a custom Java implementation of the OpenAI REST API, that works best with Quarkus (as it uses the Quarkus REST client) and Spring (as it uses Spring's RestClient).

  • OpenAI Official SDK uses the official OpenAI Java SDK.

  • Azure OpenAI uses the Azure SDK from Microsoft, and works best if you are using the Microsoft Java stack, including advanced Azure authentication mechanisms.

Maven Dependency​

Plain Java​

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-open-ai</artifactId>
    <version>1.22.0</version>
</dependency>

Spring Boot​

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-open-ai-spring-boot4-starter</artifactId>
    <version>1.22.0-beta32</version>
</dependency>
note

This starter requires Spring Boot 4. On Spring Boot 3, use langchain4j-open-ai-spring-boot-starter instead. See Spring Boot Integration for details.

note

OpenAI has deprecated the DALL·E 2 and DALL·E 3 models in favor of the new GPT image models (gpt-image-1, gpt-image-1-mini, gpt-image-1.5, gpt-image-2, chatgpt-image-latest). The DALL·E-specific parameters style and response-format are no longer supported by the API and have been removed. New parameters background, output-format, output-compression, and moderation are available instead.

Creating OpenAiImageModel​

Plain Java​

ImageModel model = OpenAiImageModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-image-1")
        .build();

Spring Boot​

Add to the application.properties:

# Mandatory properties:
langchain4j.open-ai.image-model.api-key=${OPENAI_API_KEY}
langchain4j.open-ai.image-model.model-name=gpt-image-1

# Optional properties:
langchain4j.open-ai.image-model.background=...
langchain4j.open-ai.image-model.base-url=...
langchain4j.open-ai.image-model.custom-headers=...
langchain4j.open-ai.image-model.log-requests=...
langchain4j.open-ai.image-model.log-responses=...
langchain4j.open-ai.image-model.max-retries=...
langchain4j.open-ai.image-model.moderation=...
langchain4j.open-ai.image-model.organization-id=...
langchain4j.open-ai.image-model.output-compression=...
langchain4j.open-ai.image-model.output-format=...
langchain4j.open-ai.image-model.project-id=...
langchain4j.open-ai.image-model.quality=...
langchain4j.open-ai.image-model.size=...
langchain4j.open-ai.image-model.timeout=...
langchain4j.open-ai.image-model.user=...

Token usage​

GPT image models report how many tokens a request used. OpenAiImageModel returns this as an OpenAiImageTokenUsage, which also splits the input and output tokens into image tokens and text tokens (OpenAI prices them differently):

Response<Image> response = model.generate("A watercolor painting of a lighthouse");

OpenAiImageTokenUsage tokenUsage = (OpenAiImageTokenUsage) response.tokenUsage();
tokenUsage.inputTokenCount();                    // all input tokens
tokenUsage.inputTokensDetails().textTokens();    // input tokens from the prompt
tokenUsage.inputTokensDetails().imageTokens();   // input tokens from input images (edits)
tokenUsage.outputTokenCount();                   // all output tokens

The details are null when OpenAI does not report them. DALL·E models do not report usage at all, so for them response.tokenUsage() is null.

Examples​