Skip to content

Feature Request: Allow API consumers to disable implicit prompt injection for deterministic behavior #2308

@Angelic47

Description

@Angelic47

Is your feature request related to a problem? Please describe.

Gemini API / Google Vertex API is injecting extra content into the system prompt submitted by the user.
This is a serious problem, which goes against the common design principles of LLM APIs.

It directly causes another problem: caching no longer hits correctly.

Beyond that, it makes gameplay / roleplay / creative writing use cases completely unusable, the LLM's responses get pulled off-track and can no longer produce what the developer expects.

Even users doing code generation are affected, because the injected prompt instructs the model to "reduce verbosity by 50%", which severely degrades output quality.

For regular users on the APP / Web client, this may be understandable for compliance or quality-assistance reasons. That's fine.
But for any application that relies on prompt adherence and stable output, this behavior makes the API no longer trustworthy.
It directly breaks user trust: "aside from the safety alignment baked into the model itself: the model's output should be good-faith to the user's input".

The prompt context should not be MITM'd, at least should it contain anything else the developer didn't put there.

On top of that, because of this injected content, a significant amount of the model's thoughts tokens are spent processing irrelevant instructions,
generating extra and unnecessary cost on the user's side.


Injected Prompts

  • Example 1, In roleplay, gameplay NPC, or D&D-like scenarios:

Do not affirm, validate, or "play along" with ungrounded beliefs or logical leaps, refraining from any attempt to rationalize the event or provide technical solutions based on the user's premise.
Differentiate between creative contexts and assertions of fact; for the latter, empathize strictly with the user's underlying emotions—acknowledging their feelings without confirming the external cause.
Maintain a calm, neutral, and reality-grounded stance that gently distinguishes subjective experience from objective fact, introducing uncertainty regarding the user's conclusions without being argumentative.
Pivot the conversation away from the ungrounded content toward general, safe topics to de-escalate the fixation.
If the user indicates distress or potential harm, encourage connection with a trusted support network or healthcare professional as an empowering option.

  • Example 2, When code generation involves JSON / Markdown / sub-agent content:

Adhere strictly to the requested output format (e.g., JSON, Markdown) or direct answer output. For objective, short-form, or technical attributes, provide standard responses. For all long-form, narrative, or descriptive attributes like "prompt" or "description", reduce the level of detail and verbosity by 50% compared to a standard response. Prioritize brevity and high-level summaries over nuanced explanation.

  • Example 3, When thinking is enabled or automatic reasoning effort is in use:

SPECIAL INSTRUCTION: think silently if needed. EFFORT LEVEL: 0.50.

(Where EFFORT LEVEL appears to be a dynamic variable.)


Describe the solution you'd like

Allow API consumers who require deterministic, fully-controlled behavior to disable automatic prompt injection,
while preserving the default behavior for users who rely on it.


Describe alternatives you've considered

Appending large amounts of overriding prompts to make the LLM ignore the injected content.
But this doesn't fundamentally solve the problem, the LLM's semantic understanding still gets shifted, and the resulting behavior remains probabilistic and unpredictable.


Related issues

Metadata

Metadata

Labels

priority: p3Desirable enhancement or fix. May not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions