Skip to content

docs: update OpenAI page Vision section #3490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: 1.0.0
Choose a base branch
from

Conversation

NanKisu
Copy link

@NanKisu NanKisu commented Jun 9, 2025

📄 Documentation Bug

Problem:
#3486
In the 1.0.0 version of the documentation, the Vision example shown is actually from version 1.1.0-SNAPSHOT. Shouldn't the example below be included instead if it's truly the 1.0.0 documentation?

Suggested Fix:
Update OpenAI page Vision section

Version: Spring AI 1.0.0

@@ -228,17 +228,19 @@ The OpenAI link:https://platform.openai.com/docs/api-reference/chat/create#chat-
Spring AI’s link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-model/src/main/java/org/springframework/ai/chat/messages/Message.java[Message] interface facilitates multimodal AI models by introducing the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-model/src/main/java/org/springframework/ai/model/Media.java[Media] type.
This type encompasses data and details regarding media attachments in messages, utilizing Spring’s `org.springframework.util.MimeType` and a `org.springframework.core.io.Resource` for the raw media data.

Below is a code example excerpted from link:https://github.com/spring-projects/spring-ai/blob/c9a3e66f90187ce7eae7eb78c462ec622685de6c/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatModelIT.java#L293[OpenAiChatModelIT.java], illustrating the fusion of user text with an image using the `gpt-4o` model.
Below is a code example excerpted from link:https://github.com/spring-projects/spring-ai/blob/v1.0.0/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/OpenAiChatModelIT.java#L469[OpenAiChatModelIT.java], illustrating the fusion of user text with an image using the `gpt-4o` model.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it really matters which version of the test that we are linking here as the users will not really see what version we are linking to. More than that, since 1.0.0 is already out, the next version will be 1.0.1 and ideally we should point to the latest snapshot from 1.0.x. But, it doesn't really matter what we are linking to since we are showing some code examples mainly from a test.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review!

I think the example code currently shown in the documentation is not correct for version 1.0.0.
Specifically, this part:

var userMessage = new UserMessage("Explain what do you see on this picture?",
        new Media(MimeTypeUtils.IMAGE_PNG, this.imageResource));

ChatResponse response = chatModel.call(new Prompt(this.userMessage,
        OpenAiChatOptions.builder().model(OpenAiApi.ChatModel.GPT_4_O.getValue()).build()));

I’m planning to replace it with the following, which I believe reflects the correct usage:

var userMessage = UserMessage.builder()
    .text("Explain what do you see on this picture?")
    .media(List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData)))
    .build();
var response = this.chatModel
    .call(new Prompt(List.of(userMessage), OpenAiChatOptions.builder().model(modelName).build()));

For the same reason, I updated the test code link to point to the version where this correct example is actually used. The previous link was referencing code that I thought was incorrect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants