Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/en/latest/plugins/ai-proxy-multi.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy with

The `ai-proxy-multi` Plugin simplifies access to LLM and embedding models by transforming Plugin configurations into the designated request format for OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs. It extends the capabilities of [`ai-proxy`](./ai-proxy.md) with load balancing, retries, fallbacks, and health checks.

In addition, the Plugin also supports logging LLM request information in the access log, such as token usage, model, time to the first response, and more.
In addition, the Plugin also supports logging LLM request information in the access log, such as token usage, model, time to the first response, and more. These log entries are also consumed by logging plugins such as `http-logger` and `kafka-logger`, and do not affect error log.

## Request Format

Expand Down Expand Up @@ -73,9 +73,9 @@ In addition, the Plugin also supports logging LLM request information in the acc
| instances.auth.gcp.expire_early_secs| integer | False | 60 | minimum = 0 | Seconds to expire the access token before its actual expiration time to avoid edge cases. |
| instances.options | object | False | | | Model configurations. In addition to `model`, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, DeepSeek, or AIMLAPI, you can configure additional parameters such as `max_tokens`, `temperature`, `top_p`, and `stream`. See your LLM provider's API documentation for more available options. |
| instances.options.model | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more available models. |
| logging | object | False | | | Logging configurations. |
| logging.summaries | boolean | False | false | | If true, log request LLM model, duration, request, and response tokens. |
| logging.payloads | boolean | False | false | | If true, log request and response payload. |
| logging | object | False | | | Logging configurations. Does not affect `error.log`. |
| logging.summaries | boolean | False | false | | If true, logs request LLM model, duration, request, and response tokens. |
| logging.payloads | boolean | False | false | | If true, logs request and response payload. |
| logging.override | object | False | | | Override setting. |
| logging.override.endpoint | string | False | | | LLM provider endpoint to replace the default endpoint with. If not configured, the Plugin uses the default OpenAI endpoint `https://api.openai.com/v1/chat/completions`. |
| checks | object | False | | | Health check configurations. Note that at the moment, OpenAI, DeepSeek, and AIMLAPI do not provide an official health check endpoint. Other LLM services that you can configure under `openai-compatible` provider may have available health check endpoints. |
Expand Down
4 changes: 2 additions & 2 deletions docs/en/latest/plugins/ai-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ description: The ai-proxy Plugin simplifies access to LLM and embedding models p

The `ai-proxy` Plugin simplifies access to LLM and embedding models by transforming Plugin configurations into the designated request format. It supports the integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter, Gemini, Vertex AI, and other OpenAI-compatible APIs.

In addition, the Plugin also supports logging LLM request information in the access log, such as token usage, model, time to the first response, and more.
In addition, the Plugin also supports logging LLM request information in the access log, such as token usage, model, time to the first response, and more. These log entries are also consumed by logging plugins such as `http-logger` and `kafka-logger`. These options do not affect `error.log`.

## Request Format

Expand Down Expand Up @@ -66,7 +66,7 @@ In addition, the Plugin also supports logging LLM request information in the acc
| options.model | string | False | | | Name of the LLM model, such as `gpt-4` or `gpt-3.5`. Refer to the LLM provider's API documentation for available models. |
| override | object | False | | | Override setting. |
| override.endpoint | string | False | | | Custom LLM provider endpoint, required when `provider` is `openai-compatible`. |
| logging | object | False | | | Logging configurations. |
| logging | object | False | | | Logging configurations. Does not affect `error.log`. |
| logging.summaries | boolean | False | false | | If true, logs request LLM model, duration, request, and response tokens. |
| logging.payloads | boolean | False | false | | If true, logs request and response payload. |
| timeout | integer | False | 30000 | ≥ 1 | Request timeout in milliseconds when requesting the LLM service. |
Expand Down
4 changes: 2 additions & 2 deletions docs/zh/latest/plugins/ai-proxy-multi.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ description: ai-proxy-multi 插件通过负载均衡、重试、故障转移和

`ai-proxy-multi` 插件通过将插件配置转换为 OpenAI、DeepSeek、Azure、AIMLAPI、Anthropic、OpenRouter、Gemini、Vertex AI 和其他 OpenAI 兼容 API 的指定请求格式,简化了对 LLM 和嵌入模型的访问。它通过负载均衡、重试、故障转移和健康检查扩展了 [`ai-proxy`](./ai-proxy.md) 的功能。

此外,该插件还支持在访问日志中记录 LLM 请求信息,如令牌使用量、模型、首次响应时间等。
此外,该插件还支持在访问日志中记录 LLM 请求信息,如令牌使用量、模型、首次响应时间等。这些日志条目也会被 `http-logger`、`kafka-logger` 等日志插件消费,但不影响 `error.log`。

## 请求格式

Expand Down Expand Up @@ -73,7 +73,7 @@ description: ai-proxy-multi 插件通过负载均衡、重试、故障转移和
| instances.auth.gcp.expire_early_secs| integer | 否 | 60 | minimum = 0 | 在访问令牌实际过期时间之前使其过期的秒数,以避免边缘情况。 |
| instances.options | object | 否 | | | 模型配置。除了 `model` 之外,您还可以配置其他参数,它们将在请求体中转发到上游 LLM 服务。例如,如果您使用 OpenAI、DeepSeek 或 AIMLAPI,可以配置其他参数,如 `max_tokens`、`temperature`、`top_p` 和 `stream`。有关更多可用选项,请参阅您的 LLM 提供商的 API 文档。 |
| instances.options.model | string | 否 | | | LLM 模型的名称,如 `gpt-4` 或 `gpt-3.5`。有关更多可用模型,请参阅您的 LLM 提供商的 API 文档。 |
| logging | object | 否 | | | 日志配置。 |
| logging | object | 否 | | | 日志配置。不影响 `error.log`。 |
| logging.summaries | boolean | 否 | false | | 如果为 true,记录请求 LLM 模型、持续时间、请求和响应令牌。 |
| logging.payloads | boolean | 否 | false | | 如果为 true,记录请求和响应负载。 |
| logging.override | object | 否 | | | 覆盖设置。 |
Expand Down
4 changes: 2 additions & 2 deletions docs/zh/latest/plugins/ai-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ description: ai-proxy 插件通过将插件配置转换为所需的请求格式

`ai-proxy` 插件通过将插件配置转换为指定的请求格式,简化了对 LLM 和嵌入模型的访问。它支持与 OpenAI、DeepSeek、Azure、AIMLAPI、Anthropic、OpenRouter、Gemini、Vertex AI 和其他 OpenAI 兼容的 API 集成。

此外,该插件还支持在访问日志中记录 LLM 请求信息,如令牌使用量、模型、首次响应时间等。
此外,该插件还支持在访问日志中记录 LLM 请求信息,如令牌使用量、模型、首次响应时间等。这些日志条目也会被 `http-logger`、`kafka-logger` 等日志插件消费。这些选项不影响 `error.log`。

## 请求格式

Expand Down Expand Up @@ -66,7 +66,7 @@ description: ai-proxy 插件通过将插件配置转换为所需的请求格式
| options.model | string | 否 | | | LLM 模型的名称,如 `gpt-4` 或 `gpt-3.5`。请参阅 LLM 提供商的 API 文档以了解可用模型。 |
| override | object | 否 | | | 覆盖设置。 |
| override.endpoint | string | 否 | | | 自定义 LLM 提供商端点,当 `provider` 为 `openai-compatible` 时必需。 |
| logging | object | 否 | | | 日志配置。 |
| logging | object | 否 | | | 日志配置。不影响 `error.log`。 |
| logging.summaries | boolean | 否 | false | | 如果为 true,记录请求 LLM 模型、持续时间、请求和响应令牌。 |
| logging.payloads | boolean | 否 | false | | 如果为 true,记录请求和响应负载。 |
| timeout | integer | 否 | 30000 | ≥ 1 | 请求 LLM 服务时的请求超时时间(毫秒)。 |
Expand Down
Loading