Skip to content

[Bug]: Support max_completion_tokens for newer OpenAI models #153

Description

@ochnios

Bug Description

Ask O11y currently sends max_tokens in requests to the Grafana LLM plugin:

  • pkg/agent/types.go serializes MaxTokens as max_tokens
  • pkg/agent/loop.go sets this field for every agent LLM request

Grafana LLM app seems to reject this parameter for newer models and require max_completion_tokens instead. This causes agent runs to fail with this error:

Something went wrong
Grafana LLM app returned an internal error. Ask an admin to check grafana-llm-app provider configuration and backend logs.

and Grafana LLM app logging:

logger=context userId=20 orgId=1 uname=<redacted> t=<redacted> level=error msg="Request Completed" method=POST path=/api/plugins/grafana-llm-app/resources/openai/v1/chat/completions status=500 remote_addr=<redacted> time_ms=21 duration=21.5068ms size=81 referer= handler=/api/plugins/:pluginId/resources/* status_source=downstream
logger=plugin.grafana-llm-app t=<redacted> level=error msg="error establishing stream" err="this model is not supported MaxTokens, please use MaxCompletionTokens"

Steps to Reproduce

  1. Navigate to Administration > Plugins and data > Plugins > LLM > Configuration
  2. Choose "Use OpenAI-compatible API" with "Azure OpenAI"
  3. Configure API keys and model mappings
  4. Compare results for some older and newer model (e.g. gpt-4.1 vs gpt-5.6-sol) - it works for the first one but fails for the second one

Expected Behavior

Ask O11y should send max_completion_tokens instead of max_tokens and not fail with newer OpenAI models

Actual Behavior

Ask O11y send max_tokens and Grafana LLM plugin requests fail

Grafana Version

v13.0.1

Plugin Version

0.2.36

User Role

Admin

Browser

Edge

Operating System

Windows 11

Browser Console Errors

Grafana Server Logs

logger=context userId=20 orgId=1 uname=<redacted> t=<redacted> level=error msg="Request Completed" method=POST path=/api/plugins/grafana-llm-app/resources/openai/v1/chat/completions status=500 remote_addr=<redacted> time_ms=21 duration=21.5068ms size=81 referer= handler=/api/plugins/:pluginId/resources/* status_source=downstream
logger=plugin.grafana-llm-app t=<redacted> level=error msg="error establishing stream" err="this model is not supported MaxTokens, please use MaxCompletionTokens"

Configuration

No response

Screenshots

Image

Additional Context

No response

Pre-submission Checklist

  • I have searched existing issues to ensure this is not a duplicate
  • I have provided all required information above
  • I have included browser console errors (if applicable)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions