perf(telemetry): skip non-recording LLM trace work#6371
Draft
dexhunter wants to merge 1 commit into
Draft
Conversation
Avoid telemetry configuration and payload serialization when the active span cannot record attributes. Refs google#4233
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
trace_call_llmbuilds telemetry configuration and serializes request/response content before writing span attributes, even when the supplied or current span is not recording. Those values cannot be retained by a non-recording span, so large LLM payloads incur avoidable CPU work.Solution:
Resolve the span first and return immediately when
span.is_recording()is false. Recording spans keep the existing telemetry and attribute path unchanged. The new unit test verifies that a non-recording span does not read telemetry configuration, serialize content, or write attributes.This addresses one narrow source of the repeated serialization reported in #4233; it does not claim to eliminate every serialization path in that issue.
Performance
An instrumented helper-level benchmark used content capture with a synthetic 108,167-byte response:
trace_call_llm, non-recording spanThis avoids about 99.91% of the helper overhead in that case (approximately 1,137x). It is not an end-to-end agent or model latency claim. Recording-span behavior is covered by correctness gates and remains unchanged.
The early-guard direction was identified through autoresearch with Weco. The final patch was rebuilt on current main and hardened with recording and non-recording counterexamples.
Testing Plan
Unit Tests:
Results:
pytest tests/unittests/telemetry/test_spans.py -q: 80 passedruff check,pyink --check, andgit diff --check: passedThe repository-wide tox run was also attempted without exclusions. Its only collection blocker was the checkout-external
open_source_workspace/.agents/skills/adk-verify-snippets/scripts/verify_md.py, which is not present in this checkout. Re-running all unit tests while excluding onlytest_verify_snippets.pypassed on Python 3.10 through 3.14.Manual End-to-End (E2E) Tests:
Not applicable for this pure tracing guard. The unit test and strict evaluator exercise both an explicitly supplied span and the normal recording/non-recording behavior without requiring an external model request.
Checklist
Additional context
The commit is cryptographically signed and verified by GitHub.