Bedrock Converse API

Amazon Bedrock Converse API 为对话式 AI 模型提供了一个统一的接口,具有增强的功能,包括函数/工具调用、多模态输入和流式响应。

Amazon Bedrock Converse API provides a unified interface for conversational AI models with enhanced capabilities including function/tool calling, multimodal inputs, and streaming responses.

Bedrock Converse API 具有以下高级功能:

The Bedrock Converse API has the following high-level features:

  • * 工具/函数调用:支持对话期间的函数定义和工具使用

  • Tool/Function Calling: Support for function definitions and tool use during conversations

  • * 多模态输入:能够在对话中处理文本和图像输入

  • Multimodal Input: Ability to process both text and image inputs in conversations

  • * 流式支持:模型响应的实时流式传输

  • Streaming Support: Real-time streaming of model responses

  • * 系统消息:支持系统级指令和上下文设置

  • System Messages: Support for system-level instructions and context setting

Bedrock Converse API 通过处理 AWS 特定的身份验证和基础设施问题,为多个模型提供商提供了一个统一的接口。目前,Converse API Supported Models 包括: Amazon TitanAmazon NovaAI21 LabsAnthropic ClaudeCohere CommandMeta LlamaMistral AI

The Bedrock Converse API provides a unified interface across multiple model providers while handling AWS-specific authentication and infrastructure concerns. Currently, the Converse API Supported Models include: Amazon Titan, Amazon Nova, AI21 Labs, Anthropic Claude, Cohere Command, Meta Llama, Mistral AI.

根据 Bedrock 的建议,Spring AI 正在转向使用 Amazon Bedrock 的 Converse API 来实现 Spring AI 中的所有聊天对话。虽然现有的 InvokeModel API 支持对话应用程序,但我们强烈建议所有聊天对话模型都采用 Converse API。

Following the Bedrock recommendations, Spring AI is transitioning to using Amazon Bedrock’s Converse API for all chat conversation implementations in Spring AI. While the existing InvokeModel API supports conversation applications, we strongly recommend adopting the Converse API for all Chat conversation models.

Converse API 不支持嵌入操作,因此这些操作将保留在当前 API 中,并且现有 InvokeModel API 中的嵌入模型功能将得到维护。

The Converse API does not support embedding operations, so these will remain in the current API and the embedding model functionality in the existing InvokeModel API will be maintained

Prerequisites

请参阅 Getting started with Amazon Bedrock 以设置 API 访问权限。

Refer to Getting started with Amazon Bedrock for setting up API access

  • 获取 AWS 凭据:如果您还没有 AWS 账户和 AWS CLI 配置,此视频指南可以帮助您配置: AWS CLI & SDK Setup in Less Than 4 Minutes! 。您应该能够获取您的访问密钥和安全密钥。

  • Obtain AWS credentials: If you don’t have an AWS account and AWS CLI configured yet, this video guide can help you configure it: AWS CLI & SDK Setup in Less Than 4 Minutes!. You should be able to obtain your access and security keys.

  • 启用要使用的模型:转到 Amazon Bedrock ,然后从左侧的 Model Access 菜单中,配置对您将要使用的模型的访问权限。

  • Enable the Models to use: Go to Amazon Bedrock and from the Model Access menu on the left, configure access to the models you are going to use.

Auto-configuration

Spring AI 自动配置、启动器模块的工件名称发生了重大变化。请参阅 upgrade notes 以获取更多信息。

There has been a significant change in the Spring AI auto-configuration, starter modules' artifact names. Please refer to the upgrade notes for more information.

spring-ai-starter-model-bedrock-converse 依赖项添加到您项目的 Maven pom.xml 或 Gradle build.gradle 构建文件中:

Add the spring-ai-starter-model-bedrock-converse dependency to your project’s Maven pom.xml or Gradle build.gradle build files:

  • Maven

  • Gradle

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-bedrock-converse</artifactId>
</dependency>
dependencies {
    implementation 'org.springframework.ai:spring-ai-starter-model-bedrock-converse'
}
  1. 参见 Dependency Management 部分,将 Spring AI BOM 添加到你的构建文件中。

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

Chat Properties

spring.ai.bedrock.aws 前缀是配置与 AWS Bedrock 的连接的属性前缀。

The prefix spring.ai.bedrock.aws is the property prefix to configure the connection to AWS Bedrock.

Property Description Default

spring.ai.bedrock.aws.region

AWS region to use.

us-east-1

spring.ai.bedrock.aws.timeout

AWS timeout to use.

5m

spring.ai.bedrock.aws.access-key

AWS access key.

-

spring.ai.bedrock.aws.secret-key

AWS secret key.

-

spring.ai.bedrock.aws.session-token

AWS session token for temporary credentials.

-

聊天自动配置的启用和禁用现在通过前缀为 spring.ai.model.chat 的顶级属性进行配置。

Enabling and disabling of the chat auto-configurations are now configured via top level properties with the prefix spring.ai.model.chat.

要启用,请设置 spring.ai.model.chat=bedrock-converse(默认已启用)

To enable, spring.ai.model.chat=bedrock-converse (It is enabled by default)

要禁用,请设置 spring.ai.model.chat=none(或任何不匹配 bedrock-converse 的值)

To disable, spring.ai.model.chat=none (or any value which doesn’t match bedrock-converse)

此更改旨在允许配置多个模型。

This change is done to allow configuration of multiple models.

前缀 spring.ai.bedrock.converse.chat 是配置 Converse API 聊天模型实现的属性前缀。

The prefix spring.ai.bedrock.converse.chat is the property prefix that configures the chat model implementation for the Converse API.

Property Description Default

spring.ai.bedrock.converse.chat.enabled (Removed and no longer valid)

Enable Bedrock Converse chat model.

true

spring.ai.model.chat

Enable Bedrock Converse chat model.

bedrock-converse

spring.ai.bedrock.converse.chat.options.model

The model ID to use. You can use the Supported models and model features

None. Select your modelId from the AWS Bedrock console.

spring.ai.bedrock.converse.chat.options.temperature

Controls the randomness of the output. Values can range over [0.0,1.0]

0.8

spring.ai.bedrock.converse.chat.options.top-p

The maximum cumulative probability of tokens to consider when sampling.

AWS Bedrock default

spring.ai.bedrock.converse.chat.options.top-k

Number of token choices for generating the next token.

AWS Bedrock default

spring.ai.bedrock.converse.chat.options.max-tokens

Maximum number of tokens in the generated response.

500

Runtime Options

使用便携式 ChatOptionsToolCallingChatOptions 便携式构建器创建模型配置,例如温度、最大令牌数、topP 等。

Use the portable ChatOptions or ToolCallingChatOptions portable builders to create model configurations, such as temperature, maxToken, topP, etc.

在启动时,可以使用 BedrockConverseProxyChatModel(api, options) 构造函数或 spring.ai.bedrock.converse.chat.options.* 属性配置默认选项。

On start-up, the default options can be configured with the BedrockConverseProxyChatModel(api, options) constructor or the spring.ai.bedrock.converse.chat.options.* properties.

在运行时,您可以通过向 Prompt 调用添加新的、特定于请求的选项来覆盖默认选项:

At run-time you can override the default options by adding new, request specific, options to the Prompt call:

var options = ToolCallingChatOptions.builder()
        .model("anthropic.claude-3-5-sonnet-20240620-v1:0")
        .temperature(0.6)
        .maxTokens(300)
        .toolCallbacks(List.of(FunctionToolCallback.builder("getCurrentWeather", new WeatherService())
            .description("Get the weather in location. Return temperature in 36°F or 36°C format. Use multi-turn if needed.")
            .inputType(WeatherService.Request.class)
            .build()))
        .build();

String response = ChatClient.create(this.chatModel)
    .prompt("What is current weather in Amsterdam?")
    .options(options)
    .call()
    .content();

Tool Calling

Bedrock Converse API 支持工具调用功能,允许模型在对话中使用工具。以下是定义和使用基于 @Tool 的工具的示例:

The Bedrock Converse API supports tool calling capabilities, allowing models to use tools during conversations. Here’s an example of how to define and use @Tool based tools:

public class WeatherService {

    @Tool(description = "Get the weather in location")
    public String weatherByLocation(@ToolParam(description= "City or state name") String location) {
        ...
    }
}

String response = ChatClient.create(this.chatModel)
        .prompt("What's the weather like in Boston?")
        .tools(new WeatherService())
        .call()
        .content();

您也可以使用 java.util.function bean 作为工具:

You can use the java.util.function beans as tools as well:

@Bean
@Description("Get the weather in location. Return temperature in 36°F or 36°C format.")
public Function<Request, Response> weatherFunction() {
    return new MockWeatherService();
}

String response = ChatClient.create(this.chatModel)
        .prompt("What's the weather like in Boston?")
        .tools("weatherFunction")
        .inputType(Request.class)
        .call()
        .content();

Tools 文档中查找更多信息。

Find more in Tools documentation.

Multimodal

多模态是指模型同时理解和处理来自各种来源(包括文本、图像、视频、pdf、doc、html、md 和更多数据格式)信息的能力。

Multimodality refers to a model’s ability to simultaneously understand and process information from various sources, including text, images, video, pdf, doc, html, md and more data formats.

Bedrock Converse API 支持多模态输入,包括文本和图像输入,并且可以根据组合输入生成文本响应。

The Bedrock Converse API supports multimodal inputs, including text and image inputs, and can generate a text response based on the combined input.

您需要一个支持多模态输入的模型,例如 Anthropic Claude 或 Amazon Nova 模型。

You need a model that supports multimodal inputs, such as the Anthropic Claude or Amazon Nova models.

Images

对于支持视觉多模态的 models ,例如 Amazon Nova、Anthropic Claude、Llama 3.2,Bedrock Converse API Amazon 允许您在有效负载中包含多个图像。这些模型可以分析传入的图像并回答问题、对图像进行分类以及根据提供的说明总结图像。

For models that support vision multimodality, such as Amazon Nova, Anthropic Claude, Llama 3.2, the Bedrock Converse API Amazon allows you to include multiple images in the payload. Those models can analyze the passed images and answer questions, classify an image, as well as summarize images based on provided instructions.

目前,Bedrock Converse 支持 base64 编码的 image/jpegimage/pngimage/gifimage/webp mime 类型的图像。

Currently, Bedrock Converse supports the base64 encoded images of image/jpeg, image/png, image/gif and image/webp mime types.

Spring AI 的 Message 接口通过引入 Media 类型支持多模态 AI 模型。它包含有关消息中媒体附件的数据和信息,使用 Spring 的 org.springframework.util.MimeType 和用于原始媒体数据的 java.lang.Object

Spring AI’s Message interface supports multimodal AI models by introducing the Media type. It contains data and information about media attachments in messages, using Spring’s org.springframework.util.MimeType and a java.lang.Object for the raw media data.

以下是一个简单的代码示例,演示了用户文本与图像的组合。

Below is a simple code example, demonstrating the combination of user text with an image.

String response = ChatClient.create(chatModel)
    .prompt()
    .user(u -> u.text("Explain what do you see on this picture?")
        .media(Media.Format.IMAGE_PNG, new ClassPathResource("/test.png")))
    .call()
    .content();

logger.info(response);

它将 test.png 图像作为输入:

It takes as an input the test.png image:

multimodal.test

并附带短信“解释一下你在这张图片上看到了什么?”,然后生成类似如下的回复:

along with the text message "Explain what do you see on this picture?", and generates a response something like:

The image shows a close-up view of a wire fruit basket containing several pieces of fruit.
...

Video

Amazon Nova models 允许你在有效载荷中包含单个视频,该视频可以以base64格式或通过Amazon S3 URI提供。

The Amazon Nova models allow you to include a single video in the payload, which can be provided either in base64 format or through an Amazon S3 URI.

目前,Bedrock Nova 支持 video/x-matrosvideo/quicktimevideo/mp4video/video/webmvideo/x-flvvideo/mpegvideo/x-ms-wmvimage/3gpp MIME 类型的图像。

Currently, Bedrock Nova supports the images of video/x-matros, video/quicktime, video/mp4, video/video/webm, video/x-flv, video/mpeg, video/x-ms-wmv and image/3gpp mime types.

Spring AI 的 Message 接口通过引入 Media` 类型来支持多模态 AI 模型。它包含有关消息中媒体附件的数据和信息,利用 Spring 的 org.springframework.util.MimeType 和用于原始媒体数据的 java.lang.Object

Spring AI’s Message interface supports multimodal AI models by introducing the Media` type. It contains data and information about media attachments in messages, using Spring’s org.springframework.util.MimeType and a java.lang.Object for the raw media data.

下面是一个简单的代码示例,演示了用户文本与视频的结合。

Below is a simple code example, demonstrating the combination of user text with a video.

String response = ChatClient.create(chatModel)
    .prompt()
    .user(u -> u.text("Explain what do you see in this video?")
        .media(Media.Format.VIDEO_MP4, new ClassPathResource("/test.video.mp4")))
    .call()
    .content();

logger.info(response);

它以 test.video.mp4 图像作为输入:

It takes as an input the test.video.mp4 image:

test.video

并附带短信“解释一下你在这个视频中看到了什么?”,然后生成类似如下的回复:

along with the text message "Explain what do you see in this video?", and generates a response something like:

The video shows a group of baby chickens, also known as chicks, huddled together on a surface
...

Documents

对于某些模型,Bedrock允许你通过Converse API文档支持在有效载荷中包含文档,这些文档可以以字节形式提供。文档支持有两种不同的变体,如下所述:

For some models, Bedrock allows you to include documents in the payload through Converse API document support, which can be provided in bytes. The document support has two different variants as explained below:

  • Text document types (txt, csv, html, md等),重点是文本理解。这些用例包括根据文档的文本元素进行回答。

  • Text document types (txt, csv, html, md, and so on), where the emphasis is on text understanding. These use case include answering based on textual elements of the document.

  • Media document types (pdf, docx, xlsx),重点是基于视觉的理解来回答问题。这些用例包括根据图表、图形等回答问题。

  • Media document types (pdf, docx, xlsx), where the emphasis is on vision-based understanding to answer questions. These use cases include answering questions based on charts, graphs, and so on.

目前Anthropic PDF support (beta) 和Amazon Bedrock Nova模型支持文档多模态。

Currently the Anthropic PDF support (beta) and Amazon Bedrock Nova models support document multimodality.

下面是一个简单的代码示例,演示了用户文本与媒体文档的结合。

Below is a simple code example, demonstrating the combination of user text with a media document.

String response = ChatClient.create(chatModel)
    .prompt()
    .user(u -> u.text(
            "You are a very professional document summarization specialist. Please summarize the given document.")
        .media(Media.Format.DOC_PDF, new ClassPathResource("/spring-ai-reference-overview.pdf")))
    .call()
    .content();

logger.info(response);

它以 spring-ai-reference-overview.pdf 文档作为输入:

It takes as an input the spring-ai-reference-overview.pdf document:

test.pdf

并附带短信“你是一位非常专业的文档总结专家。请总结一下给定的文档。”,然后生成类似如下的回复:

along with the text message "You are a very professional document summarization specialist. Please summarize the given document.", and generates a response something like:

**Introduction:**
- Spring AI is designed to simplify the development of applications with artificial intelligence (AI) capabilities, aiming to avoid unnecessary complexity.
...

Sample Controller

创建一个新的Spring Boot项目,并将 spring-ai-starter-model-bedrock-converse 添加到你的依赖项中。

Create a new Spring Boot project and add the spring-ai-starter-model-bedrock-converse to your dependencies.

src/main/resources 下添加一个 application.properties 文件:

Add an application.properties file under src/main/resources:

spring.ai.bedrock.aws.region=eu-central-1
spring.ai.bedrock.aws.timeout=10m
spring.ai.bedrock.aws.access-key=${AWS_ACCESS_KEY_ID}
spring.ai.bedrock.aws.secret-key=${AWS_SECRET_ACCESS_KEY}
# session token is only required for temporary credentials
spring.ai.bedrock.aws.session-token=${AWS_SESSION_TOKEN}

spring.ai.bedrock.converse.chat.options.temperature=0.8
spring.ai.bedrock.converse.chat.options.top-k=15

以下是一个使用聊天模型的控制器示例:

Here’s an example controller using the chat model:

@RestController
public class ChatController {

    private final ChatClient chatClient;

    @Autowired
    public ChatController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/ai/generate")
    public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return Map.of("generation", this.chatClient.prompt(message).call().content());
    }

    @GetMapping("/ai/generateStream")
    public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return this.chatClient.prompt(message).stream().content();
    }
}