Prompts

提示是指导 AI 模型生成特定输出的输入。这些提示的设计和措辞显著地影响了模型的响应。

Prompts are the inputs that guide an AI model to generate specific outputs. The design and phrasing of these prompts significantly influence the model’s responses.

在 Spring AI 中与 AI 模型交互的最低级别上，在 Spring AI 中处理提示有点类似于在 Spring MVC 中管理“视图”。这涉及使用动态内容的占位符创建大量文本。这些占位符随后基于用户请求或应用程序中的其他代码进行替换。另一个类似之处是包含用于特定表达式的占位符的 SQL 语句。

At the lowest level of interaction with AI models in Spring AI, handling prompts in Spring AI is somewhat similar to managing the "View" in Spring MVC. This involves creating extensive text with placeholders for dynamic content. These placeholders are then replaced based on user requests or other code in the application. Another analogy is a SQL statement that contain placeholders for certain expressions.

随着 Spring AI 的发展，它将引入与 AI 模型交互的更高抽象级别。本节中描述的基础类在角色和功能方面可以比作 JDBC。例如， ChatModel 类类似于 JDK 中的核心 JDBC 库。 ChatClient 类可以比作 JdbcClient ，它构建在 ChatModel 之上，并通过 Advisor 提供更高级的构造，以考虑与模型的过去交互，用额外的上下文文档增强提示，并引入代理行为。

As Spring AI evolves, it will introduce higher levels of abstraction for interacting with AI models. The foundational classes described in this section can be likened to JDBC in terms of their role and functionality. The ChatModel class, for instance, is analogous to the core JDBC library in the JDK. The ChatClient class can be likened to the JdbcClient, built on top of ChatModel and providing more advanced constructs via Advisor to consider past interactions with the model, augment the prompt with additional contextual documents, and introduce agentic behavior.

提示的结构随着时间在 AI 领域内发生了演变。最初，提示是简单的字符串。随着时间的推移，它们逐渐包含用于特定输入的占位符，例如 AI 模型识别的“USER:”。OpenAI 通过在 AI 模型处理之前将多个消息字符串分类为不同的角色，为提示引入了更加结构化的内容。

The structure of prompts has evolved over time within the AI field. Initially, prompts were simple strings. Over time, they grew to include placeholders for specific inputs, like "USER:", which the AI model recognizes. OpenAI have introduced even more structure to prompts by categorizing multiple message strings into distinct roles before they are processed by the AI model.

API Overview

Prompt

通常使用 ChatModel 的 call() 方法，它接受一个 Prompt 实例并返回一个 ChatResponse 。

It is common to use the call() method of ChatModel that takes a Prompt instance and returns a ChatResponse.

Prompt 类充当有序的 Message 对象系列和请求 ChatOptions 的容器。每个 Message 在提示中都扮演着独特的角色，内容和意图各不相同。这些角色可以包括各种元素，从用户查询到 AI 生成的响应再到相关的背景信息。这种安排使得与 AI 模型进行复杂而详细的交互成为可能，因为提示由多个消息构成，每个消息都在对话中扮演特定的角色。

The Prompt class functions as a container for an organized series of Message objects and a request ChatOptions. Every Message embodies a unique role within the prompt, differing in its content and intent. These roles can encompass a variety of elements, from user inquiries to AI-generated responses to relevant background information. This arrangement enables intricate and detailed interactions with AI models, as the prompt is constructed from multiple messages, each assigned a specific role to play in the dialogue.

下面是 Prompt 类的截断版本，为简洁起见，省略了构造函数和实用方法：

Below is a truncated version of the Prompt class, with constructors and utility methods omitted for brevity:

public class Prompt implements ModelRequest<List<Message>> {

    private final List<Message> messages;

    private ChatOptions chatOptions;
}

Message

Message 接口封装了 Prompt 文本内容、一组元数据属性以及称为 MessageType 的分类。

The Message interface encapsulates a Prompt textual content, a collection of metadata attributes, and a categorization known as MessageType.

接口定义如下：

The interface is defined as follows:

public interface Content {

	String getContent();

	Map<String, Object> getMetadata();
}

public interface Message extends Content {

	MessageType getMessageType();
}

多模态消息类型也实现了 MediaContent 接口，提供 Media 内容对象的列表。

The multimodal message types implement also the MediaContent interface providing a list of Media content objects.

public interface MediaContent extends Content {

	Collection<Media> getMedia();

}

Message 接口的各种实现对应于 AI 模型可以处理的不同类别的消息。模型根据对话角色区分消息类别。

Various implementations of the Message interface correspond to different categories of messages that an AI model can process. The Models distinguish between message categories based on conversational roles.

spring ai message api

这些角色由 MessageType 有效映射，如下所述。

These roles are effectively mapped by the MessageType, as discussed below.

Roles

每条消息都被分配一个特定角色。这些角色对消息进行分类，阐明提示每个部分对 AI 模型的上下文和目的。这种结构化方法增强了与 AI 通信的细微差别和有效性，因为提示的每个部分在交互中都扮演着独特且明确的角色。

Each message is assigned a specific role. These roles categorize the messages, clarifying the context and purpose of each segment of the prompt for the AI model. This structured approach enhances the nuance and effectiveness of communication with the AI, as each part of the prompt plays a distinct and defined role in the interaction.

主要角色有：

The primary roles are:

  • 系统角色：指导 AI 的行为和响应风格，设置 AI 解释和回复输入方式的参数或规则。这类似于在开启对话前向 AI 提供指令。

  • System Role: Guides the AI’s behavior and response style, setting parameters or rules for how the AI interprets and replies to the input. It’s akin to providing instructions to the AI before initiating a conversation.

  • 用户角色：代表用户输入——他们对 AI 的问题、命令或陈述。此角色非常重要，因为它是 AI 响应的基础。

  • User Role: Represents the user’s input – their questions, commands, or statements to the AI. This role is fundamental as it forms the basis of the AI’s response.

  • 助理角色：AI 对用户输入的响应。它不仅仅是一个答案或反应，对于维持对话流程至关重要。通过跟踪 AI 之前的响应（其“助理角色”消息），系统确保连贯和上下文相关的交互。助理消息也可能包含函数工具调用请求信息。它就像 AI 中的一项特殊功能，在需要执行特定功能时使用，例如计算、获取数据或除了对话之外的其他任务。

  • Assistant Role: The AI’s response to the user’s input. More than just an answer or reaction, it’s crucial for maintaining the flow of the conversation. By tracking the AI’s previous responses (its 'Assistant Role' messages), the system ensures coherent and contextually relevant interactions. The Assistant message may contain Function Tool Call request information as well. It’s like a special feature in the AI, used when needed to perform specific functions such as calculations, fetching data, or other tasks beyond just talking.

  • 工具/函数角色：工具/函数角色侧重于响应工具调用助理消息返回额外信息。

  • Tool/Function Role: The Tool/Function Role focuses on returning additional information in response to Tool Call Assistant Messages.

角色在Spring AI中表示为一个枚举，如下所示

Roles are represented as an enumeration in Spring AI as shown below

public enum MessageType {

	USER("user"),

	ASSISTANT("assistant"),

	SYSTEM("system"),

	TOOL("tool");

    ...
}

PromptTemplate

Spring AI 中用于提示模板化的一个关键组件是 PromptTemplate 类，旨在促进创建结构化提示，然后将其发送到 AI 模型进行处理。

A key component for prompt templating in Spring AI is the PromptTemplate class, designed to facilitate the creation of structured prompts that are then sent to the AI model for processing

public class PromptTemplate implements PromptTemplateActions, PromptTemplateMessageActions {

    // Other methods to be discussed later
}

此类使用 TemplateRenderer API 渲染模板。默认情况下，Spring AI 使用 StTemplateRenderer 实现，它基于 Terence Parr 开发的开源 StringTemplate 引擎。模板变量由 {} 语法标识，但您也可以配置分隔符以使用其他语法。

This class uses the TemplateRenderer API to render templates. By default, Spring AI uses the StTemplateRenderer implementation, which is based on the open-source StringTemplate engine developed by Terence Parr. Template variables are identified by the {} syntax, but you can configure the delimiters to use other syntax as well.

public interface TemplateRenderer extends BiFunction<String, Map<String, Object>, String> {

	@Override
	String apply(String template, Map<String, Object> variables);

}

Spring AI 使用 TemplateRenderer 接口处理变量到模板字符串的实际替换。默认实现使用 [StringTemplate] 。如果您需要自定义逻辑，可以提供自己的 TemplateRenderer 实现。对于不需要模板渲染的场景（例如，模板字符串已完成），您可以使用提供的 NoOpTemplateRenderer 。

Spring AI uses the TemplateRenderer interface to handle the actual substitution of variables into the template string. The default implementation uses [StringTemplate]. You can provide your own implementation of TemplateRenderer if you need custom logic. For scenarios where no template rendering is required (e.g., the template string is already complete), you can use the provided NoOpTemplateRenderer.

Example using a custom StringTemplate renderer with '<' and '>' delimiters
PromptTemplate promptTemplate = PromptTemplate.builder()
    .renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
    .template("""
            Tell me the names of 5 movies whose soundtrack was composed by <composer>.
            """)
    .build();

String prompt = promptTemplate.render(Map.of("composer", "John Williams"));

该类实现的接口支持提示创建的各个方面：

The interfaces implemented by this class support different aspects of prompt creation:

`PromptTemplateStringActions`专注于创建和呈现提示字符串，代表最基本的提示生成形式。

PromptTemplateStringActions focuses on creating and rendering prompt strings, representing the most basic form of prompt generation.

PromptTemplateMessageActions 专为通过生成和操作 Message 对象来创建提示而定制。

PromptTemplateMessageActions is tailored for prompt creation through the generation and manipulation of Message objects.

PromptTemplateActions 旨在返回 Prompt 对象，该对象可以传递给 ChatModel 以生成响应。

PromptTemplateActions is designed to return the Prompt object, which can be passed to ChatModel for generating a response.

虽然在许多项目中可能不会广泛使用这些界面，但它们展示了提示创建的不同方法。

While these interfaces might not be used extensively in many projects, they show the different approaches to prompt creation.

实现的接口

The implemented interfaces are

public interface PromptTemplateStringActions {

	String render();

	String render(Map<String, Object> model);

}

方法`String render()`：将提示模板呈现为最终字符串格式，无需外部输入，适用于没有占位符或动态内容的模板。

The method String render(): Renders a prompt template into a final string format without external input, suitable for templates without placeholders or dynamic content.

方法 String render(Map<String, Object> model) ：增强渲染功能以包含动态内容。它使用 Map<String, Object> ，其中映射键是提示模板中的占位符名称，值是要插入的动态内容。

The method String render(Map<String, Object> model): Enhances rendering functionality to include dynamic content. It uses a Map<String, Object> where map keys are placeholder names in the prompt template, and values are the dynamic content to be inserted.

public interface PromptTemplateMessageActions {

	Message createMessage();

    Message createMessage(List<Media> mediaList);

	Message createMessage(Map<String, Object> model);

}

方法 Message createMessage() ：创建不带附加数据的 Message 对象，用于静态或预定义的消息内容。

The method Message createMessage(): Creates a Message object without additional data, used for static or predefined message content.

方法 Message createMessage(List<Media> mediaList) ：创建包含静态文本和媒体内容的 Message 对象。

The method Message createMessage(List<Media> mediaList): Creates a Message object with static textual and media content.

方法 Message createMessage(Map<String, Object> model) ：扩展消息创建以集成动态内容，接受 Map<String, Object> ，其中每个条目表示消息模板中的占位符及其对应的动态值。

The method Message createMessage(Map<String, Object> model): Extends message creation to integrate dynamic content, accepting a Map<String, Object> where each entry represents a placeholder in the message template and its corresponding dynamic value.

public interface PromptTemplateActions extends PromptTemplateStringActions {

	Prompt create();

	Prompt create(ChatOptions modelOptions);

	Prompt create(Map<String, Object> model);

	Prompt create(Map<String, Object> model, ChatOptions modelOptions);

}

方法 Prompt create() ：生成不带外部数据输入的 Prompt 对象，非常适合静态或预定义提示。

The method Prompt create(): Generates a Prompt object without external data inputs, ideal for static or predefined prompts.

方法 Prompt create(ChatOptions modelOptions) ：生成不带外部数据输入且带有聊天请求特定选项的 Prompt 对象。

The method Prompt create(ChatOptions modelOptions): Generates a Prompt object without external data inputs and with specific options for the chat request.

方法 Prompt create(Map<String, Object> model) ：扩展提示创建功能以包含动态内容，接受 Map<String, Object> ，其中每个映射条目是提示模板中的占位符及其关联的动态值。

The method Prompt create(Map<String, Object> model): Expands prompt creation capabilities to include dynamic content, taking a Map<String, Object> where each map entry is a placeholder in the prompt template and its associated dynamic value.

方法 Prompt create(Map<String, Object> model, ChatOptions modelOptions) ：扩展提示创建功能以包含动态内容，接受 Map<String, Object> ，其中每个映射条目是提示模板中的占位符及其关联的动态值，以及聊天请求的特定选项。

The method Prompt create(Map<String, Object> model, ChatOptions modelOptions): Expands prompt creation capabilities to include dynamic content, taking a Map<String, Object> where each map entry is a placeholder in the prompt template and its associated dynamic value, and specific options for the chat request.

Example Usage

以下所示是取自 AI Workshop on PromptTemplates 的一个简单示例。

A simple example taken from the AI Workshop on PromptTemplates is shown below.

PromptTemplate promptTemplate = new PromptTemplate("Tell me a {adjective} joke about {topic}");

Prompt prompt = promptTemplate.create(Map.of("adjective", adjective, "topic", topic));

return chatModel.call(prompt).getResult();

下面展示了从 ` AI Workshop on Roles` 摘取的另一个示例。

Another example taken from the AI Workshop on Roles is shown below.

String userText = """
    Tell me about three famous pirates from the Golden Age of Piracy and why they did.
    Write at least a sentence for each pirate.
    """;

Message userMessage = new UserMessage(userText);

String systemText = """
  You are a helpful AI assistant that helps people find information.
  Your name is {name}
  You should reply to the user's request with your name and also in the style of a {voice}.
  """;

SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemText);
Message systemMessage = systemPromptTemplate.createMessage(Map.of("name", name, "voice", voice));

Prompt prompt = new Prompt(List.of(userMessage, systemMessage));

List<Generation> response = chatModel.call(prompt).getResults();

这显示了您如何通过使用 SystemPromptTemplate 构建 Prompt 实例来创建具有系统角色的 Message ，并传入占位符值。然后将角色为 user 的消息与角色为 system 的消息组合以形成提示。然后将提示传递给 ChatModel 以获得生成响应。

This shows how you can build up the Prompt instance by using the SystemPromptTemplate to create a Message with the system role passing in placeholder values. The message with the role user is then combined with the message of the role system to form the prompt. The prompt is then passed to the ChatModel to get a generative response.

Using a custom template renderer

您可以通过实现 TemplateRenderer 接口并将其传递给 PromptTemplate 构造函数来使用自定义模板渲染器。您也可以继续使用默认的 StTemplateRenderer ，但带有自定义配置。

You can use a custom template renderer by implementing the TemplateRenderer interface and passing it to the PromptTemplate constructor. You can also keep using the default StTemplateRenderer, but with a custom configuration.

默认情况下，模板变量通过 {} 语法识别。如果您计划在提示中包含 JSON，您可能需要使用不同的语法来避免与 JSON 语法冲突。例如，您可以使用 < 和 > 分隔符。

By default, template variables are identified by the {} syntax. If you’re planning to include JSON in your prompt, you might want to use a different syntax to avoid conflicts with JSON syntax. For example, you can use the < and > delimiters.

PromptTemplate promptTemplate = PromptTemplate.builder()
    .renderer(StTemplateRenderer.builder().startDelimiterToken('<').endDelimiterToken('>').build())
    .template("""
            Tell me the names of 5 movies whose soundtrack was composed by <composer>.
            """)
    .build();

String prompt = promptTemplate.render(Map.of("composer", "John Williams"));

Using resources instead of raw Strings

Spring AI 支持 org.springframework.core.io.Resource 抽象，因此您可以将提示数据放在一个可以直接在 PromptTemplate 中使用的文件中。例如，您可以在 Spring 管理的组件中定义一个字段来检索 Resource 。

Spring AI supports the org.springframework.core.io.Resource abstraction, so you can put prompt data in a file that can directly be used in a PromptTemplate. For example, you can define a field in your Spring managed component to retrieve the Resource.

@Value("classpath:/prompts/system-message.st")
private Resource systemResource;

然后将该资源直接传递给 SystemPromptTemplate。

and then pass that resource to the SystemPromptTemplate directly.

SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(systemResource);

Prompt Engineering

在生成式 AI 中，提示的创建是开发人员的一项关键任务。这些提示的质量和结构会极大地影响 AI 的输出效果。投入时间和精力来设计经过深思熟虑的提示可以极大地改善 AI 的结果。

In generative AI, the creation of prompts is a crucial task for developers. The quality and structure of these prompts significantly influence the effectiveness of the AI’s output. Investing time and effort in designing thoughtful prompts can greatly improve the results from the AI.

在 AI 社区中，分享和讨论提示是一种常见做法。这种协作方式不仅创建一个共享的学习环境，而且还导致识别和使用高效的提示。

Sharing and discussing prompts is a common practice in the AI community. This collaborative approach not only creates a shared learning environment but also leads to the identification and use of highly effective prompts.

这个领域的研究通常涉及分析和比较不同的提示，以评估它们在不同情况下的有效性。例如，一项重大的研究表明，以“深呼吸，一步一步解决这个问题”开头可以显著提高解决问题的效率。这突出了精心选择用语对生成式 AI 系统性能的影响。

Research in this area often involves analyzing and comparing different prompts to assess their effectiveness in various situations. For example, a significant study demonstrated that starting a prompt with "Take a deep breath and work on this problem step by step" significantly enhanced problem-solving efficiency. This highlights the impact that well-chosen language can have on generative AI systems' performance.

掌握最有效的使用提示的方法，尤其是在 AI 技术快速发展的情况下，是一个持续的挑战。你应该认识到提示工程的重要性，并考虑利用社区和研究的见解来改进提示创建策略。

Grasping the most effective use of prompts, particularly with the rapid advancement of AI technologies, is a continuous challenge. You should recognize the importance of prompt engineering and consider using insights from the community and research to improve prompt creation strategies.

Creating effective prompts

在开发提示时，集成几个关键组件以确保清晰性和有效性非常重要：

When developing prompts, it’s important to integrate several key components to ensure clarity and effectiveness:

  • Instructions ：向 AI 提供清晰直接的指令，类似于您与人交流的方式。这种清晰度对于帮助 AI “理解”预期内容至关重要。

  • Instructions: Offer clear and direct instructions to the AI, similar to how you would communicate with a person. This clarity is essential for helping the AI "understand" what is expected.

  • External Context ：在必要时，为 AI 的响应包含相关的背景信息或具体指导。这种“外部上下文”构成了提示的框架，并帮助 AI 掌握整体场景。

  • External Context: Include relevant background information or specific guidance for the AI’s response when necessary. This "external context" frames the prompt and aids the AI in grasping the overall scenario.

  • User Input：这是简单明了的部分——用户的直接请求或问题构成了提示的核心。

  • User Input: This is the straightforward part - the user’s direct request or question forming the core of the prompt.

  • Output Indicator：这方面可能有些棘手。它涉及指定 AI 响应的期望格式，如 JSON。但请注意 AI 可能不会始终严格遵守此格式。例如，它可能在实际 JSON 数据前缀加一个诸如“这是你的 JSON”的短语，或者有时会生成不准确的类似 JSON 的结构。

  • Output Indicator: This aspect can be tricky. It involves specifying the desired format for the AI’s response, such as JSON. However, be aware that the AI might not always adhere strictly to this format. For instance, it might prepend a phrase like "here is your JSON" before the actual JSON data, or sometimes generate a JSON-like structure that is not accurate.

在编写提示时，向 AI 提供预期问题和答案格式的示例会非常有益。这种做法有助于 AI “理解”您查询的结构和意图，从而产生更精确和相关的响应。尽管本文档没有深入探讨这些技术，但它们为进一步探索 AI 提示工程提供了起点。

Providing the AI with examples of the anticipated question and answer format can be highly beneficial when crafting prompts. This practice helps the AI "understand" the structure and intent of your query, leading to more precise and relevant responses. While this documentation does not delve deeply into these techniques, they provide a starting point for further exploration in AI prompt engineering.

以下列出了一些供进一步调查的资源。

Following is a list of resources for further investigation.

Simple Techniques

  • Text Summarization：将大量的文本简化为简洁的摘要，捕捉重点和主要观点，同时忽略不太重要的细节。

  • Text Summarization: Reduces extensive text into concise summaries, capturing key points and main ideas while omitting less critical details.

  • Question Answering：侧重于从提供的文本中衍生出具体的答案，基于用户提出的问题。它关于根据查询明确指出和提取相关信息。

  • Question Answering: Focuses on deriving specific answers from provided text, based on user-posed questions. It’s about pinpointing and extracting relevant information in response to queries.

  • Text Classification：系统地将文本分类到预定义的类别或分组中，分析文本并根据其内容将其分配到最合适的类别。

  • Text Classification: Systematically categorizes text into predefined categories or groups, analyzing the text and assigning it to the most fitting category based on its content.

  • Conversation：创建交互式对话，其中人工智能可以与用户进行来回交流，模拟自然的对话流。

  • Conversation: Creates interactive dialogues where the AI can engage in back-and-forth communication with users, simulating a natural conversation flow.

  • Code Generation：根据特定的用户要求或描述生成功能性的代码片段，将自然语言指令翻译成可执行代码。

  • Code Generation: Generates functional code snippets based on specific user requirements or descriptions, translating natural language instructions into executable code.

Advanced Techniques

  • Zero-shot, Few-shot Learning：使模型能够对最小到没有特定问题类型先例的情况做出准确的预测或响应，利用学习到的概括理解和执行新任务。

  • Zero-shot, Few-shot Learning: Enables the model to make accurate predictions or responses with minimal to no prior examples of the specific problem type, understanding and acting on new tasks using learned generalizations.

  • Chain-of-Thought：链接多个人工智能响应以创建连贯且具有上下文意识的对话。它帮助人工智能维持讨论的主题，确保相关性和连续性。

  • Chain-of-Thought: Links multiple AI responses to create a coherent and contextually aware conversation. It helps the AI maintain the thread of the discussion, ensuring relevance and continuity.

  • ReAct (Reason + Act)：在这种方法中，人工智能首先分析（推理）输入，然后确定最适当的行为或响应。它将理解与决策相结合。

  • ReAct (Reason + Act): In this method, the AI first analyzes (reasons about) the input, then determines the most appropriate course of action or response. It combines understanding with decision-making.

Microsoft Guidance

  • Framework for Prompt Creation and Optimization：Microsoft 提供了一种开发和完善提示的结构化方法。此框架指导用户创建有效的提示，以从 AI 模型中引出所需的响应，以清晰和高效优化互动。

  • Framework for Prompt Creation and Optimization: Microsoft offers a structured approach to developing and refining prompts. This framework guides users in creating effective prompts that elicit the desired responses from AI models, optimizing the interaction for clarity and efficiency.

Tokens

令牌在 AI 模型处理文本中至关重要，充当将单词（正如我们理解的那样）转换成 AI 模型可以处理的格式的桥梁。此转换分两个阶段进行：输入时将单词转换为令牌，然后在输出中将这些令牌转换回单词。

Tokens are essential in how AI models process text, acting as a bridge that converts words (as we understand them) into a format that AI models can process. This conversion occurs in two stages: words are transformed into tokens upon input, and these tokens are then converted back into words in the output.

标记化（将文本分解为标记的过程）是 AI 模型理解和处理语言的基础。AI 模型使用这种标记化格式来理解和响应提示。

Tokenization, the process of breaking down text into tokens, is fundamental to how AI models comprehend and process language. The AI model works with this tokenized format to understand and respond to prompts.

为了更好地理解令牌，可以将它们视为单词的一部分。通常，一个令牌代表大约四分之三个单词。例如，莎士比亚的全部作品大约有 900,000 个单词，转换为大约 120 万个令牌。

To better understand tokens, think of them as portions of words. Typically, a token represents about three-quarters of a word. For instance, the complete works of Shakespeare, totaling roughly 900,000 words, would translate to around 1.2 million tokens.

使用 OpenAI Tokenizer UI 来试验如何将单词转换为标记。

Experiment with the OpenAI Tokenizer UI to see how words are converted into tokens.

除了在 AI 处理中的技术作用外，令牌还具有实际意义，尤其是在计费和模型功能方面：

Tokens have practical implications beyond their technical role in AI processing, especially regarding billing and model capabilities:

  • 计费：人工智能模型服务通常根据令牌使用情况进行计费。输入（提示）和输出（响应）都会计入令牌总数，从而使较短的提示更具成本效益。

  • Billing: AI model services often bill based on token usage. Both the input (prompt) and the output (response) contribute to the total token count, making shorter prompts more cost-effective.

  • 模型限制：不同的 AI 模型有不同的令牌限制，定义它们的“上下文窗口”——它们可以一次处理的最大信息量。例如，GPT-3 的限制是 4K 个令牌，而 Claude 2 和 Meta Llama 2 等其他模型的限制是 100K 个令牌，一些研究模型可以处理多达 100 万个令牌。

  • Model Limits: Different AI models have varying token limits, defining their "context window" – the maximum amount of information they can process at a time. For example, GPT-3’s limit is 4K tokens, while other models like Claude 2 and Meta Llama 2 have limits of 100K tokens, and some research models can handle up to 1 million tokens.

  • 上下文窗口：模型的令牌限制决定了它的上下文窗口。超过此限制的输入不会由模型处理。只发送最小有效的信息集合进行处理至关重要。例如，当询问“哈姆雷特”时，无需包含莎士比亚所有其他作品中的令牌。

  • Context Window: A model’s token limit determines its context window. Inputs exceeding this limit are not processed by the model. It’s crucial to send only the minimal effective set of information for processing. For example, when inquiring about "Hamlet," there’s no need to include tokens from all of Shakespeare’s other works.

  • 响应元数据：来自人工智能模型的响应的元数据包括所用令牌的数量，这是管理使用情况和成本的重要信息。

  • Response Metadata: The metadata of a response from an AI model includes the number of tokens used, a vital piece of information for managing usage and costs.