fix(models): Allow context caches with empty content prefixes#6367
Open
hxaxd wants to merge 4 commits into
Open
fix(models): Allow context caches with empty content prefixes#6367hxaxd wants to merge 4 commits into
hxaxd wants to merge 4 commits into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Omit the contents field when the cacheable prefix is empty so the Google Gen AI SDK can create a cache from the system instruction and tools. Add a regression assertion for the zero-prefix request. Fixes google#6363
d51915b to
7b2808b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
Problem:
PR #6067 correctly changed fingerprint-only cache metadata to use the cacheable conversation prefix rather than all request contents. This keeps trailing user contents, including request-scoped dynamic instructions, out of the stable cache identity.
For an initial request containing only the latest user content, the cacheable conversation prefix is legitimately empty. ADK therefore stores fingerprint-only metadata with
contents_count == 0. It does not create a cache on this initial request because no token count from a previous response is available yet.On the next request, ADK uses the stored count to calculate the current fingerprint. When the system instruction and tools remain unchanged, that fingerprint matches the stored fingerprint. If the system instruction and tools are large enough to meet Gemini’s cache-size requirement, ADK makes its first real cache-creation attempt.
The stored count is deliberately reused because it defines the prefix covered by the matched fingerprint. Even if the later request now has a non-empty cacheable conversation prefix, ADK still slices it with the stored zero. This produces
llm_request.contents[:0] == [], which is passed asCreateCachedContentConfig(contents=[]).The Google Gen AI SDK distinguishes an omitted
contentsfield from an explicit empty list. It rejectscontents=[]withValueError: contents are required, whilecontents=Noneomits only the conversation contents and still allows the system instruction and tools to be included in the cache request.ADK catches the exception, logs a warning, and continues the model request without a cache, so the request itself does not crash. Because no
cache_namewas created, the caller preserves the fingerprint-only metadata and its originalcontents_count == 0. This behavior is intentional so transient cache-creation failures can retry the same cache identity.As long as the system instruction and tools remain unchanged, the next request again matches the same fingerprint using the preserved count. ADK then retries the same invalid
contents=[]configuration, logs another warning, and again receives nocache_name. This cycle repeats on every matching request, so the cache is never established.Solution:
Pass
Nonewhen the cacheable conversation prefix is empty so the SDK omits thecontentsfield. Preserve the existing list behavior for non-empty prefixes.The existing zero-prefix lifecycle test now verifies that the SDK-bound configuration uses
config.contents is None.Testing Plan
Unit Tests:
Focused test result:
tests/unittests/agents/test_gemini_context_cache_manager.py: 33 passed, 5 warnings.The five warnings are unrelated to this change: four are
BaseAgentConfigdeprecation warnings, and one reports that the experimentalAGENT_CONFIGfeature is enabled.The relevant context-cache tests passed under tox with Python 3.10 through 3.14.
Additional validation:
git diff --checkpassed.Manual End-to-End (E2E) Tests:
Not run because this environment cannot create and inspect a real Google cache resource.
The official SDK documentation and source confirm the relevant boundary behavior:
CreateCachedContentConfigallowscontentsto beNoneand definessystem_instructionandtoolsas separate fields.contentswhen it is notNone;system_instructionandtoolsare handled independently.t_contents()implementation explicitly rejects an empty list withValueError("contents are required.").Therefore,
Noneomits the optional field, while[]enters the transformer and is rejected. The unit test verifies that ADK produces the valid side of this boundary.Checklist
Additional context
The root cause of the missed regression in the existing tests is that
caches.create()is mocked and therefore does not reproduce the Google Gen AI SDK’s complete request transformation and validation behavior. The mock acceptedcontents=[]and returned success, so the existing zero-prefix lifecycle test passed even though the real SDK rejects that value before sending the request.No user-facing API or documentation changes are required.