Skip to content

Add more tools (based on searchfox-py) to the LLM review agent, and make it patch stack aware to a certain extent#6051

Merged
marco-c merged 5 commits into
mozilla:masterfrom
padenot:searchfox-more-tools
May 28, 2026
Merged

Add more tools (based on searchfox-py) to the LLM review agent, and make it patch stack aware to a certain extent#6051
marco-c merged 5 commits into
mozilla:masterfrom
padenot:searchfox-more-tools

Conversation

@padenot
Copy link
Copy Markdown
Collaborator

@padenot padenot commented May 25, 2026

The two patches can be reviewed independently but are close in location so I'm putting one branch for both.

@padenot padenot force-pushed the searchfox-more-tools branch 2 times, most recently from 33dbdae to 798947f Compare May 25, 2026 16:12
@padenot padenot requested a review from suhaibmujahid May 25, 2026 16:14
@padenot
Copy link
Copy Markdown
Collaborator Author

padenot commented May 25, 2026

padenot/searchfox-cli@v0.10.12...v0.13.0 are the changes in searchfox-py for this (API tightening, adding async, etc.).

@padenot padenot force-pushed the searchfox-more-tools branch 6 times, most recently from 44f260f to 3917e53 Compare May 26, 2026 17:06
Comment thread bugbug/tools/code_review/agent.py Outdated
Comment thread bugbug/tools/code_review/langchain_tools.py Outdated
Comment thread bugbug/tools/code_review/langchain_tools.py
Comment thread bugbug/tools/code_review/langchain_tools.py Outdated
Comment thread bugbug/tools/code_review/searchfox_tools.py Outdated
Comment thread bugbug/tools/code_review/agent.py Outdated
Comment thread bugbug/tools/code_review/langchain_tools.py Outdated
@padenot padenot force-pushed the searchfox-more-tools branch from 3917e53 to a5eccfc Compare May 27, 2026 12:41
Comment thread tests/test_code_review.py Outdated
Comment thread bugbug/tools/code_review/langchain_tools.py
@padenot padenot force-pushed the searchfox-more-tools branch 4 times, most recently from d72e7e9 to 6eda3aa Compare May 28, 2026 12:18
@padenot
Copy link
Copy Markdown
Collaborator Author

padenot commented May 28, 2026

@marco-c last one is a fix for a new failure on main, ptal

marco-c
marco-c previously approved these changes May 28, 2026
suhaibmujahid
suhaibmujahid previously approved these changes May 28, 2026
Comment thread bugbug/tools/code_review/langchain_tools.py
Comment thread bugbug/tools/code_review/searchfox_tools.py Outdated
Comment thread bugbug/tools/code_review/searchfox_tools.py Outdated
Comment thread bugbug/tools/core/platforms/patch_apply.py
padenot added 5 commits May 28, 2026 15:02
expand_context now shows the file as it appears after the full patch stack
is applied. A new patch_apply module provides reusable primitives:

- strip_diff_prefix, find_patched_file, apply_patched_file: diff utilities
  with tenacity retries on both fetches for transient failures
- get_file_after_stack(patch, path, fetch): applies patch.patch_stack in
  order on top of whatever base the caller provides via fetch()

expand_context calls get_file_after_stack with patch.get_old_file as the
fetch function. The Patch base class gains patch_stack (default: [patch_set]),
get_base_revision(), and patch_stack_warning(); phabricator.py overrides all
three. Non-linear stacks fall back to the current patch only and set a warning
that expand_context surfaces prominently so the agent can flag it in the review.

expand_context also gains optional start_line / end_line — omitting either
returns from the beginning or to the end of the file.
Expose the searchfox Python API (searchfox-cli) as LangChain tools wired
into the code review agent. expand_context switches its fetch function
from patch.get_old_file to searchfox at the patch's base revision, using
get_file_after_stack from patch_apply.

- expand_context: searchfox at base revision + full patch stack applied
- search_text: free-text / regex search with file-type and path filters
- get_blame: per-line commit history
- get_field_layout: C++ class/struct memory layout
- find_definition: look up a function, method, class, or struct
- search_identifier: exact identifier search with file-type filters
- calls_from / calls_to / calls_between: call graph traversal with depth
- check_can_gc: SpiderMonkey GC hazard analysis for C++ functions

All tools support langs (list) and tests filters where applicable.

No additional tests for the tool wrappers: the tools are thin call-and-format
wrappers over searchfox-py, which is tested independently in its own repo.

Closes mozilla#5284

diff --git a/bugbug/tools/code_review/agent.py b/bugbug/tools/code_review/agent.py
- Define SEARCHFOX_TOOLS list in langchain_tools.py; import and use it
  in agent.py instead of listing tools inline
- Remove create_find_function_definition_tool and function_search
  parameter; the new find_definition tool is a cleaner replacement
- Move logger above _retry; use functools.cache for _get_client
- Add patch.get_old_file fallback in expand_context before falling
  back to searchfox tip
- Add get_function_at_line tool (searchfox-py 0.14.0); bump requirement
  and lockfile accordingly
- Rename langchain_tools.py to searchfox_tools.py
- Rename searchfox_tools.py back to langchain_tools.py
- Extract fetch closure to module-level _fetch_file to reduce nesting
- Use specific exception types instead of bare Exception where possible
Copy link
Copy Markdown
Member

@suhaibmujahid suhaibmujahid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@marco-c marco-c merged commit b91406b into mozilla:master May 28, 2026
6 checks passed
marco-c pushed a commit that referenced this pull request May 28, 2026
…ake it patch stack aware to a certain extent (#6051)

* Make expand_context patch-stack aware

expand_context now shows the file as it appears after the full patch stack
is applied. A new patch_apply module provides reusable primitives:

- strip_diff_prefix, find_patched_file, apply_patched_file: diff utilities
  with tenacity retries on both fetches for transient failures
- get_file_after_stack(patch, path, fetch): applies patch.patch_stack in
  order on top of whatever base the caller provides via fetch()

expand_context calls get_file_after_stack with patch.get_old_file as the
fetch function. The Patch base class gains patch_stack (default: [patch_set]),
get_base_revision(), and patch_stack_warning(); phabricator.py overrides all
three. Non-linear stacks fall back to the current patch only and set a warning
that expand_context surfaces prominently so the agent can flag it in the review.

expand_context also gains optional start_line / end_line — omitting either
returns from the beginning or to the end of the file.

* Add searchfox-based tools to the code review agent

Expose the searchfox Python API (searchfox-cli) as LangChain tools wired
into the code review agent. expand_context switches its fetch function
from patch.get_old_file to searchfox at the patch's base revision, using
get_file_after_stack from patch_apply.

- expand_context: searchfox at base revision + full patch stack applied
- search_text: free-text / regex search with file-type and path filters
- get_blame: per-line commit history
- get_field_layout: C++ class/struct memory layout
- find_definition: look up a function, method, class, or struct
- search_identifier: exact identifier search with file-type filters
- calls_from / calls_to / calls_between: call graph traversal with depth
- check_can_gc: SpiderMonkey GC hazard analysis for C++ functions

All tools support langs (list) and tests filters where applicable.

No additional tests for the tool wrappers: the tools are thin call-and-format
wrappers over searchfox-py, which is tested independently in its own repo.

Closes #5284

diff --git a/bugbug/tools/code_review/agent.py b/bugbug/tools/code_review/agent.py

* Address review comments, add get_function_at_line tool

- Define SEARCHFOX_TOOLS list in langchain_tools.py; import and use it
  in agent.py instead of listing tools inline
- Remove create_find_function_definition_tool and function_search
  parameter; the new find_definition tool is a cleaner replacement
- Move logger above _retry; use functools.cache for _get_client
- Add patch.get_old_file fallback in expand_context before falling
  back to searchfox tip
- Add get_function_at_line tool (searchfox-py 0.14.0); bump requirement
  and lockfile accordingly
- Rename langchain_tools.py to searchfox_tools.py

* Address review comments from suhaib

- Rename searchfox_tools.py back to langchain_tools.py
- Extract fetch closure to module-level _fetch_file to reduce nesting
- Use specific exception types instead of bare Exception where possible

* Try phabricator before searchfox when fetching base file content
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants