Milvus

Milvus 是一个开源向量数据库，在数据科学和机器学习领域引起了极大关注。其突出特点之一是对向量索引和查询的强大支持。Milvus 采用最先进、前沿的算法来加速搜索进程，使其即使在处理海量数据集时也能高效地检索相似向量。

Milvus is an open-source vector database that has garnered significant attention in the fields of data science and machine learning. One of its standout features lies in its robust support for vector indexing and querying. Milvus employs state-of-the-art, cutting-edge algorithms to accelerate the search process, making it exceptionally efficient at retrieving similar vectors, even when handling extensive datasets.

Prerequisites

  • 如果需要， EmbeddingModel 的 API 密钥用于生成由 MilvusVectorStore 存储的嵌入。

  • If required, an API key for the EmbeddingModel to generate the embeddings stored by the MilvusVectorStore.

Dependencies

Spring AI 自动配置、启动器模块的工件名称发生了重大变化。请参阅 upgrade notes 以获取更多信息。

There has been a significant change in the Spring AI auto-configuration, starter modules' artifact names. Please refer to the upgrade notes for more information.

然后将 Milvus VectorStore 启动器依赖项添加到你的项目：

Then add the Milvus VectorStore boot starter dependency to your project:

<dependency>
	<groupId>org.springframework.ai</groupId>
	<artifactId>spring-ai-starter-vector-store-milvus</artifactId>
</dependency>

或添加到 Gradle build.gradle 构建文件中。

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-starter-vector-store-milvus'
}

请参阅 Dependency Management 部分，将 Spring AI BOM 添加到您的构建文件中。请参阅 Artifact Repositories 部分，将 Maven Central 和/或快照存储库添加到您的构建文件中。

Refer to the Dependency Management section to add the Spring AI BOM to your build file. Refer to the Artifact Repositories section to add Maven Central and/or Snapshot Repositories to your build file.

向量存储实现可以为你初始化所需模式，但你必须通过在适当的构造函数中指定 @s15 布尔值或在 @s17 文件中设置 @s16 来选择加入。

The vector store implementation can initialize the requisite schema for you, but you must opt-in by specifying the initializeSchema boolean in the appropriate constructor or by setting …​initialize-schema=true in the application.properties file.

这是一个重大更改！在早期版本的Spring AI中，此架构初始化是默认发生的。

this is a breaking change! In earlier versions of Spring AI, this schema initialization happened by default.

向量存储还需要一个 EmbeddingModel 实例来计算文档的嵌入。您可以选择 EmbeddingModel Implementations 中可用的一个。

The Vector Store, also requires an EmbeddingModel instance to calculate embeddings for the documents. You can pick one of the available EmbeddingModel Implementations.

要连接并配置 MilvusVectorStore，你需要提供你的实例的访问详细信息。可以使用 Spring Boot 的 `application.yml`提供简单配置

To connect to and configure the MilvusVectorStore, you need to provide access details for your instance. A simple configuration can either be provided via Spring Boot’s application.yml

spring:
	ai:
		vectorstore:
			milvus:
				client:
					host: "localhost"
					port: 19530
					username: "root"
					password: "milvus"
				databaseName: "default"
				collectionName: "vector_store"
				embeddingDimension: 1536
				indexType: IVF_FLAT
				metricType: COSINE

查看 configuration parameters 列表了解默认值和配置选项。

Check the list of milvus-properties to learn about the default values and configuration options.

现在，你可以在应用程序中自动装配 Milvus Vector Store 并使用它了

Now you can Auto-wire the Milvus Vector Store in your application and use it

@Autowired VectorStore vectorStore;

// ...

List <Document> documents = List.of(
    new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
    new Document("The World is Big and Salvation Lurks Around the Corner"),
    new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));

// Add the documents to Milvus Vector Store
vectorStore.add(documents);

// Retrieve documents similar to a query
List<Document> results = this.vectorStore.similaritySearch(SearchRequest.builder().query("Spring").topK(5).build());

Manual Configuration

除了使用 Spring Boot 自动配置外，还可以手动配置 MilvusVectorStore。将以下依赖项添加到你的项目：

Instead of using the Spring Boot auto-configuration, you can manually configure the MilvusVectorStore. To add the following dependencies to your project:

<dependency>
	<groupId>org.springframework.ai</groupId>
	<artifactId>spring-ai-milvus-store</artifactId>
</dependency>

  1. 参见 Dependency Management 部分，将 Spring AI BOM 添加到你的构建文件中。

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

可以在应用程序中使用以下设置配置 MilvusVectorStore：

To configure MilvusVectorStore in your application, you can use the following setup:

	@Bean
	public VectorStore vectorStore(MilvusServiceClient milvusClient, EmbeddingModel embeddingModel) {
		return MilvusVectorStore.builder(milvusClient, embeddingModel)
				.collectionName("test_vector_store")
				.databaseName("default")
				.indexType(IndexType.IVF_FLAT)
				.metricType(MetricType.COSINE)
				.batchingStrategy(new TokenCountBatchingStrategy())
				.initializeSchema(true)
				.build();
	}

	@Bean
	public MilvusServiceClient milvusClient() {
		return new MilvusServiceClient(ConnectParam.newBuilder()
			.withAuthorization("minioadmin", "minioadmin")
			.withUri(milvusContainer.getEndpoint())
			.build());
	}

Metadata filtering

您可以将通用的、可移植的 metadata filters 与 Milvus 存储结合使用。

You can leverage the generic, portable metadata filters with the Milvus store.

例如，你可以使用文本表达式语言：

For example, you can use either the text expression language:

vectorStore.similaritySearch(
    SearchRequest.builder()
    .query("The World")
    .topK(TOP_K)
    .similarityThreshold(SIMILARITY_THRESHOLD)
    .filterExpression("author in ['john', 'jill'] && article_type == 'blog'").build());

或使用 Filter.Expression DSL 以编程方式：

or programmatically using the Filter.Expression DSL:

FilterExpressionBuilder b = new FilterExpressionBuilder();

vectorStore.similaritySearch(SearchRequest.builder()
    .query("The World")
    .topK(TOP_K)
    .similarityThreshold(SIMILARITY_THRESHOLD)
    .filterExpression(b.and(
        b.in("author","john", "jill"),
        b.eq("article_type", "blog")).build()).build());

这些筛选表达式将转换为等效的 Milvus 筛选器。

These filter expressions are converted into the equivalent Milvus filters.

Using MilvusSearchRequest

MilvusSearchRequest 扩展了 SearchRequest，允许您使用 Milvus 特定的搜索参数，例如原生表达式和搜索参数 JSON。

MilvusSearchRequest extends SearchRequest, allowing you to use Milvus-specific search parameters such as native expressions and search parameter JSON.

MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
    .query("sample query")
    .topK(5)
    .similarityThreshold(0.7)
    .nativeExpression("metadata[\"age\"] > 30") // Overrides filterExpression if both are set
    .filterExpression("age <= 30") // Ignored if nativeExpression is set
    .searchParamsJson("{\"nprobe\":128}")
    .build();
List results = vectorStore.similaritySearch(request);

这在使用 Milvus 特定的搜索功能时提供了更大的灵活性。

This allows greater flexibility when using Milvus-specific search features.

Importance of nativeExpression and searchParamsJson in MilvusSearchRequest

这两个参数提高了 Milvus 搜索精度并确保最佳查询性能：

These two parameters enhance Milvus search precision and ensure optimal query performance:

nativeExpression ：使用 Milvus 的原生筛选表达式启用额外的筛选功能。 Milvus Filtering

nativeExpression: Enables additional filtering capabilities using Milvus' native filtering expressions. Milvus Filtering

示例：

Example:

MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
    .query("sample query")
    .topK(5)
    .nativeExpression("metadata['category'] == 'science'")
    .build();

searchParamsJson ：在使用 IVF_FLAT（Milvus 的默认索引）时，对于调整搜索行为至关重要。 Milvus Vector Index

searchParamsJson: Essential for tuning search behavior when using IVF_FLAT, Milvus' default index. Milvus Vector Index

默认情况下， IVF_FLAT 要求设置 nprobe 以获得准确结果。如果未指定， nprobe 默认为 1 ，这可能导致召回率不佳甚至零搜索结果。

By default, IVF_FLAT requires nprobe to be set for accurate results. If not specified, nprobe defaults to 1, which can lead to poor recall or even zero search results.

示例：

Example:

MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
    .query("sample query")
    .topK(5)
    .searchParamsJson("{\"nprobe\":128}")
    .build();

使用 nativeExpression 可确保高级筛选，而 searchParamsJson 可防止因默认 nprobe 值过低而导致的无效搜索。

Using nativeExpression ensures advanced filtering, while searchParamsJson prevents ineffective searches caused by a low default nprobe value.

Milvus VectorStore properties

您可以在 Spring Boot 配置中使用以下属性来定制 Milvus 向量存储。

You can use the following properties in your Spring Boot configuration to customize the Milvus vector store.

Property Description Default value

spring.ai.vectorstore.milvus.database-name

The name of the Milvus database to use.

default

spring.ai.vectorstore.milvus.collection-name

Milvus collection name to store the vectors

vector_store

spring.ai.vectorstore.milvus.initialize-schema

whether to initialize Milvus' backend

false

spring.ai.vectorstore.milvus.embedding-dimension

The dimension of the vectors to be stored in the Milvus collection.

1536

spring.ai.vectorstore.milvus.index-type

The type of the index to be created for the Milvus collection.

IVF_FLAT

spring.ai.vectorstore.milvus.metric-type

The metric type to be used for the Milvus collection.

COSINE

spring.ai.vectorstore.milvus.index-parameters

The index parameters to be used for the Milvus collection.

{"nlist":1024}

spring.ai.vectorstore.milvus.id-field-name

The ID field name for the collection

doc_id

spring.ai.vectorstore.milvus.is-auto-id

Boolean flag to indicate if the auto-id is used for the ID field

false

spring.ai.vectorstore.milvus.content-field-name

The content field name for the collection

content

spring.ai.vectorstore.milvus.metadata-field-name

The metadata field name for the collection

metadata

spring.ai.vectorstore.milvus.embedding-field-name

The embedding field name for the collection

embedding

spring.ai.vectorstore.milvus.client.host

The name or address of the host.

localhost

spring.ai.vectorstore.milvus.client.port

The connection port.

19530

spring.ai.vectorstore.milvus.client.uri

The uri of Milvus instance

-

spring.ai.vectorstore.milvus.client.token

Token serving as the key for identification and authentication purposes.

-

spring.ai.vectorstore.milvus.client.connect-timeout-ms

Connection timeout value of client channel. The timeout value must be greater than zero .

10000

spring.ai.vectorstore.milvus.client.keep-alive-time-ms

Keep-alive time value of client channel. The keep-alive value must be greater than zero.

55000

spring.ai.vectorstore.milvus.client.keep-alive-timeout-ms

The keep-alive timeout value of client channel. The timeout value must be greater than zero.

20000

spring.ai.vectorstore.milvus.client.rpc-deadline-ms

Deadline for how long you are willing to wait for a reply from the server. With a deadline setting, the client will wait when encounter fast RPC fail caused by network fluctuations. The deadline value must be larger than or equal to zero.

0

spring.ai.vectorstore.milvus.client.client-key-path

The client.key path for tls two-way authentication, only takes effect when "secure" is true

-

spring.ai.vectorstore.milvus.client.client-pem-path

The client.pem path for tls two-way authentication, only takes effect when "secure" is true

-

spring.ai.vectorstore.milvus.client.ca-pem-path

The ca.pem path for tls two-way authentication, only takes effect when "secure" is true

-

spring.ai.vectorstore.milvus.client.server-pem-path

server.pem path for tls one-way authentication, only takes effect when "secure" is true.

-

spring.ai.vectorstore.milvus.client.server-name

Sets the target name override for SSL host name checking, only takes effect when "secure" is True. Note: this value is passed to grpc.ssl_target_name_override

-

spring.ai.vectorstore.milvus.client.secure

Secure the authorization for this connection, set to True to enable TLS.

false

spring.ai.vectorstore.milvus.client.idle-timeout-ms

Idle timeout value of client channel. The timeout value must be larger than zero.

24h

spring.ai.vectorstore.milvus.client.username

The username and password for this connection.

root

spring.ai.vectorstore.milvus.client.password

The password for this connection.

milvus

Starting Milvus Store

在 src/test/resources/ 文件夹中运行：

From within the src/test/resources/ folder run:

docker-compose up

要清除环境：

To clean the environment:

docker-compose down; rm -Rf ./volumes

然后连接到 http://localhost:19530 上的向量存储，或者用于管理 http://localhost:9001（用户：minioadmin，密码：minioadmin）

Then connect to the vector store on http://localhost:19530 or for management http://localhost:9001 (user: minioadmin, pass: minioadmin)

Troubleshooting

如果 Docker 抱怨资源，则执行：

If Docker complains about resources, then execute:

docker system prune --all --force --volumes

Accessing the Native Client

Milvus 向量存储实现通过 getNativeClient() 方法提供对底层原生 Milvus 客户端 ( MilvusServiceClient ) 的访问：

The Milvus Vector Store implementation provides access to the underlying native Milvus client (MilvusServiceClient) through the getNativeClient() method:

MilvusVectorStore vectorStore = context.getBean(MilvusVectorStore.class);
Optional<MilvusServiceClient> nativeClient = vectorStore.getNativeClient();

if (nativeClient.isPresent()) {
    MilvusServiceClient client = nativeClient.get();
    // Use the native client for Milvus-specific operations
}

以下是使用 Gemini 翻译的中文版本：原生客户端让您可以访问 Milvus 特有的功能和操作，这些功能和操作可能不会通过 VectorStore 接口暴露。

The native client gives you access to Milvus-specific features and operations that might not be exposed through the VectorStore interface.