-
Notifications
You must be signed in to change notification settings - Fork 155
Add Google engine safety settings and default thinking budget config #559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
0200258
fff6d0a
cdc5c2b
b6a1df0
2f3dd79
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -54,6 +54,27 @@ | |
| <label>{{ t('settings.engines.requestCooldown') }}</label> | ||
| <input type="number" name="requestCooldown" v-model.number="requestCooldown" min="0" step="100" :placeholder="t('settings.engines.requestCooldownHint')" @change="save"/> | ||
| </div> | ||
| <div class="form-field"> | ||
| <label>{{ t('settings.engines.google.safetySettings') }}</label> | ||
| <div class="form-subgroup"> | ||
| <select name="safetySettings" v-model="safetySettings" @change="save"> | ||
| <option value="">{{ t('settings.engines.google.safetyDefault') }}</option> | ||
| <option value="BLOCK_LOW_AND_ABOVE">{{ t('settings.engines.google.safetyBlockMost') }}</option> | ||
| <option value="BLOCK_MEDIUM_AND_ABOVE">{{ t('settings.engines.google.safetyBlockSome') }}</option> | ||
| <option value="BLOCK_ONLY_HIGH">{{ t('settings.engines.google.safetyBlockFew') }}</option> | ||
| <option value="BLOCK_NONE">{{ t('settings.engines.google.safetyBlockNone') }}</option> | ||
| <option value="OFF">{{ t('settings.engines.google.safetyOff') }}</option> | ||
| </select> | ||
| <span class="hint">{{ t('settings.engines.google.safetySettingsHint') }}</span> | ||
| </div> | ||
| </div> | ||
| <div class="form-field"> | ||
| <label>{{ t('settings.engines.google.defaultThinkingBudget') }}</label> | ||
| <div class="form-subgroup"> | ||
| <input type="number" name="defaultThinkingBudget" v-model.number="defaultThinkingBudget" min="-1" step="1" :placeholder="t('settings.engines.google.defaultThinkingBudgetHint')" @change="save"/> | ||
| <span class="hint">{{ t('settings.engines.google.defaultThinkingBudgetExplain') }}</span> | ||
| </div> | ||
| </div> | ||
| <div class="form-field horizontal"> | ||
| <input type="checkbox" id="google-disable-tools" name="disableTools" v-model="disableTools" @change="save" /> | ||
| <label for="google-disable-tools">{{ t('settings.engines.disableTools') }}</label> | ||
|
|
@@ -72,6 +93,7 @@ import RefreshButton from '@components/RefreshButton.vue' | |
| import ModelSelectPlus from '@components/ModelSelectPlus.vue' | ||
| import InputObfuscated from '@components/InputObfuscated.vue' | ||
| import { ChatModel, defaultCapabilities } from 'multi-llm-ts' | ||
| import { GoogleEngineConfig } from 'types/config' | ||
|
|
||
| const vertexai = ref(false) | ||
| const apiKey = ref(null) | ||
|
|
@@ -80,6 +102,8 @@ const project = ref('') | |
| const location = ref('') | ||
| const disableTools = ref(false) | ||
| const requestCooldown = ref<number>(null) | ||
| const safetySettings = ref<string>('') | ||
| const defaultThinkingBudget = ref<number>(null) | ||
|
Comment on lines
+105
to
+106
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🌐 Web query:
💡 Result: In Vue 3, v-model.number on an produces an empty string ("") when the input is cleared. Citations: 🏁 Script executed: # Find and read SettingsGoogle.vue to check lines 105-106 and 169-170
find . -name "SettingsGoogle.vue" -type fRepository: nbonamy/witsy Length of output: 99 🏁 Script executed: # Find google.ts service file
find . -name "google.ts" -path "*/renderer/services/llms/*" -type fRepository: nbonamy/witsy Length of output: 95 🏁 Script executed: # Read SettingsGoogle.vue to check lines 105-106 and 169-170
wc -l ./src/renderer/settings/SettingsGoogle.vueRepository: nbonamy/witsy Length of output: 103 🏁 Script executed: # Read the relevant sections of SettingsGoogle.vue
sed -n '100,180p' ./src/renderer/settings/SettingsGoogle.vueRepository: nbonamy/witsy Length of output: 2952 🏁 Script executed: # Read google.ts to understand how defaultThinkingBudget is handled
cat -n ./src/renderer/services/llms/google.ts | head -100Repository: nbonamy/witsy Length of output: 1874 🏁 Script executed: # Check the template section around lines 57-77 to see the input for defaultThinkingBudget
sed -n '1,100p' ./src/renderer/settings/SettingsGoogle.vueRepository: nbonamy/witsy Length of output: 5112 Normalize a cleared thinking budget before saving it. With Vue's Fix the type annotation to allow the empty string that Vue produces, and normalize it to Suggested fix-const defaultThinkingBudget = ref<number>(null)
+const defaultThinkingBudget = ref<number | '' | null>(null)- ;(store.config.engines.google as GoogleEngineConfig).defaultThinkingBudget = defaultThinkingBudget.value ?? undefined
+ ;(store.config.engines.google as GoogleEngineConfig).defaultThinkingBudget =
+ defaultThinkingBudget.value === '' || defaultThinkingBudget.value === null
+ ? undefined
+ : defaultThinkingBudget.value🤖 Prompt for AI Agents |
||
| const chat_model = ref<string>(null) | ||
| const vision_model = ref<string>(null) | ||
| const chat_models = ref<ChatModel[]>([]) | ||
|
|
@@ -102,6 +126,8 @@ const load = () => { | |
| vision_model.value = store.config.engines.google?.model?.vision || '' | ||
| disableTools.value = store.config.engines.google?.disableTools || false | ||
| requestCooldown.value = store.config.engines.google?.requestCooldown || null | ||
| safetySettings.value = (store.config.engines.google as GoogleEngineConfig)?.safetySettings || '' | ||
| defaultThinkingBudget.value = (store.config.engines.google as GoogleEngineConfig)?.defaultThinkingBudget ?? null | ||
| } | ||
|
|
||
| const getModels = async (): Promise<boolean> => { | ||
|
|
@@ -140,6 +166,8 @@ const save = () => { | |
| store.config.engines.google.model.vision = vision_model.value | ||
| store.config.engines.google.disableTools = disableTools.value | ||
| store.config.engines.google.requestCooldown = requestCooldown.value || undefined | ||
| ;(store.config.engines.google as GoogleEngineConfig).safetySettings = safetySettings.value || undefined | ||
| ;(store.config.engines.google as GoogleEngineConfig).defaultThinkingBudget = defaultThinkingBudget.value ?? undefined | ||
| store.saveSettings() | ||
| } | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| import { HarmCategory } from '@google/genai' | ||
| import GoogleEngine from '@services/llms/google' | ||
| import { ChatModel, Google, LlmCompletionOpts } from 'multi-llm-ts' | ||
| import { afterEach, describe, expect, test, vi } from 'vitest' | ||
|
|
||
| const model: ChatModel = { | ||
| id: 'gemini-2.5-pro', | ||
| name: 'Gemini 2.5 Pro', | ||
| capabilities: { | ||
| vision: false, | ||
| tools: true, | ||
| reasoning: true, | ||
| caching: false, | ||
| }, | ||
| } as ChatModel | ||
|
|
||
| const createEngine = (config: Record<string, unknown> = {}) => new GoogleEngine({ | ||
| apiKey: 'test-api-key', | ||
| ...config, | ||
| } as any) | ||
|
|
||
| const getGenerationConfig = (engine: GoogleEngine, opts?: LlmCompletionOpts) => { | ||
| return (engine as any).getGenerationConfig(model, opts) | ||
| } | ||
|
|
||
| describe('GoogleEngine', () => { | ||
| afterEach(() => { | ||
| vi.restoreAllMocks() | ||
| }) | ||
|
|
||
| test('uses configured default thinking budget and applies safety settings', async () => { | ||
| const superGetGenerationConfig = vi.spyOn(Google.prototype as any, 'getGenerationConfig').mockImplementation(async (_model, opts) => ({ | ||
| thinkingConfig: { | ||
| includeThoughts: true, | ||
| thinkingBudget: opts?.thinkingBudget, | ||
| }, | ||
| })) | ||
|
|
||
| const config = await getGenerationConfig(createEngine({ | ||
| defaultThinkingBudget: 1024, | ||
| safetySettings: 'BLOCK_NONE', | ||
| })) | ||
|
|
||
| expect(superGetGenerationConfig).toHaveBeenCalledWith(model, expect.objectContaining({ thinkingBudget: 1024 })) | ||
| expect(config).toMatchObject({ | ||
| thinkingConfig: { | ||
| includeThoughts: true, | ||
| thinkingBudget: 1024, | ||
| }, | ||
| safetySettings: [ | ||
| { category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: 'BLOCK_NONE' }, | ||
| { category: HarmCategory.HARM_CATEGORY_HATE_SPEECH, threshold: 'BLOCK_NONE' }, | ||
| { category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: 'BLOCK_NONE' }, | ||
| { category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: 'BLOCK_NONE' }, | ||
| ], | ||
| }) | ||
| }) | ||
|
|
||
| test('falls back to automatic thinking budget when no default is configured', async () => { | ||
| const superGetGenerationConfig = vi.spyOn(Google.prototype as any, 'getGenerationConfig').mockImplementation(async (_model, opts) => ({ | ||
| thinkingConfig: { | ||
| includeThoughts: true, | ||
| thinkingBudget: opts?.thinkingBudget, | ||
| }, | ||
| })) | ||
|
|
||
| const config = await getGenerationConfig(createEngine()) | ||
|
|
||
| expect(superGetGenerationConfig).toHaveBeenCalledWith(model, expect.objectContaining({ thinkingBudget: -1 })) | ||
| expect(config).toMatchObject({ | ||
| thinkingConfig: { | ||
| includeThoughts: true, | ||
| thinkingBudget: -1, | ||
| }, | ||
| }) | ||
| }) | ||
|
|
||
| test('preserves an explicit thinking budget override', async () => { | ||
| const superGetGenerationConfig = vi.spyOn(Google.prototype as any, 'getGenerationConfig').mockImplementation(async (_model, opts) => ({ | ||
| thinkingConfig: { | ||
| includeThoughts: true, | ||
| thinkingBudget: opts?.thinkingBudget, | ||
| }, | ||
| })) | ||
|
|
||
| const config = await getGenerationConfig(createEngine({ defaultThinkingBudget: 1024 }), { thinkingBudget: 256 }) | ||
|
|
||
| expect(superGetGenerationConfig).toHaveBeenCalledWith(model, expect.objectContaining({ thinkingBudget: 256 })) | ||
| expect(config).toMatchObject({ | ||
| thinkingConfig: { | ||
| includeThoughts: true, | ||
| thinkingBudget: 256, | ||
| }, | ||
| }) | ||
| }) | ||
|
|
||
| test('returns undefined when the base engine returns no config', async () => { | ||
| vi.spyOn(Google.prototype as any, 'getGenerationConfig').mockResolvedValue(undefined) | ||
|
|
||
| await expect(getGenerationConfig(createEngine({ | ||
| safetySettings: 'BLOCK_NONE', | ||
| }))).resolves.toBeUndefined() | ||
| }) | ||
| }) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🌐 Web query:
In@google/genai, what are the validHarmBlockThresholdvalues accepted byGenerateContentConfig.safetySettings?💡 Result:
The valid HarmBlockThreshold values accepted by GenerateContentConfig.safetySettings in
@google/genaiare: HARM_BLOCK_THRESHOLD_UNSPECIFIED, BLOCK_LOW_AND_ABOVE, BLOCK_MEDIUM_AND_ABOVE, BLOCK_ONLY_HIGH, BLOCK_NONE, and OFF. These are string values corresponding to the enum defined in the Gemini API protobufs, used across Google GenAI SDKs including JavaScript (@google/genai). They control the probability threshold at which content is blocked for each harm category in safety settings. - HARM_BLOCK_THRESHOLD_UNSPECIFIED: Uses default threshold (high confidence from multiple official sources). - BLOCK_LOW_AND_ABOVE: Blocks low, medium, high probability harm. - BLOCK_MEDIUM_AND_ABOVE: Blocks medium, high. - BLOCK_ONLY_HIGH: Blocks only high. - BLOCK_NONE: Blocks none (always allow). - OFF: Turns off safety filter. Example usage in@google/genai(inferred from API compatibility and JS examples): const safetySettings = [ { category: "HARM_CATEGORY_HATE_SPEECH", threshold: "BLOCK_LOW_AND_ABOVE" } ]; Official Gemini API docs confirm these values for safetySettings.threshold, and@google/genaiwraps the same REST API.Citations:
🏁 Script executed:
fd -t f "google.ts" src/renderer/services/llmsRepository: nbonamy/witsy
Length of output: 93
🏁 Script executed:
fd -t f "SettingsGoogle.vue" src/rendererRepository: nbonamy/witsy
Length of output: 97
🏁 Script executed:
Repository: nbonamy/witsy
Length of output: 1587
🏁 Script executed:
cat src/renderer/settings/SettingsGoogle.vue | head -100Repository: nbonamy/witsy
Length of output: 5112
🏁 Script executed:
grep -A 20 -B 5 "safetySettings" src/renderer/settings/SettingsGoogle.vueRepository: nbonamy/witsy
Length of output: 4567
🏁 Script executed:
grep -r "HARM_BLOCK_THRESHOLD_UNSPECIFIED" src/Repository: nbonamy/witsy
Length of output: 39
🏁 Script executed:
Repository: nbonamy/witsy
Length of output: 958
Validate
safetySettingsbefore using it with the Google API.Line 27 blindly casts
safetySettingsfrom persisted config toHarmBlockThresholdwithout validation. A hand-edited config file with an invalid threshold value will cause API errors. The UI restricts options to valid values, but the persisted setting can be modified outside the application. Add validation to whitelist known values and skip safety settings if an invalid value is encountered so the provider defaults are used instead.Valid
HarmBlockThresholdvalues are:HARM_BLOCK_THRESHOLD_UNSPECIFIED,BLOCK_LOW_AND_ABOVE,BLOCK_MEDIUM_AND_ABOVE,BLOCK_ONLY_HIGH,BLOCK_NONE, andOFF.Suggested guard
import { GenerateContentConfig, HarmBlockThreshold, HarmCategory } from '@google/genai' import { ChatModel, Google, LlmCompletionOpts } from 'multi-llm-ts' import { GoogleEngineConfig } from 'types/config' +const GOOGLE_SAFETY_THRESHOLDS = new Set([ + 'HARM_BLOCK_THRESHOLD_UNSPECIFIED', + 'BLOCK_LOW_AND_ABOVE', + 'BLOCK_MEDIUM_AND_ABOVE', + 'BLOCK_ONLY_HIGH', + 'BLOCK_NONE', + 'OFF', +]) + export default class GoogleEngine extends Google {📝 Committable suggestion
🤖 Prompt for AI Agents