Add option to exclude ip/hostname from proxy

[corest]

Motivation for this change is very simple.
I have internal Elasticsearch, that I use as http output destination in presets.
When I use web proxy to access some of targets from different locations, I run into issue, that bbot can't write results into Elasticsearch because it tries to go via proxy.
This configuration option adds possibility to exclude listed endpoints from going via proxy.

Most of code is written by Claude. I'll gladly address any comments on how to improve this. Sorry in advance, if change doesn't follow some of contribution rules.

[TheTechromancer]

@corest thanks for the PR, this is a good feature to have.

This is pretty well implemented except for two things:

1. we should be using the radixtarget library for these lookups, which is written in rust and avoids unnecessary parsing. This is the same mechanism we use currently for targets / blacklists.
2. We need to have tests for it. Claude should be able to handle this pretty well based on the existing ones (I think we have proxy tests?).

Let me know if you want to make an update, otherwise we can do it but it may take a little longer.

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

1 out of 2 committers have signed the CLA.
✅ (corest)[https://github.com/corest]
❌ @liquidsec
_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[corest]

I have read the CLA Document and I hereby sign the CLA

[corest]

recheck

[corest]

@TheTechromancer thx for a quick feedback. I updated PR addressing both comments:

1. Using radixtarget library for lookups
2. Adding tests for proxy exclusions

[liquidsec]

before we get too deep, this is still targeting stable - with 3.0 release coming soon (hopefully) if we dont rebase to 3.0 now this will just create more work later.

I'm going to go ahead and change the base, you might get some conflicts... sorry

@TheTechromancer unless you think there's another 2.x.x release you want to stick this on?

[TheTechromancer]

@corest thanks for the updates; tests look good. We're currently replacing our web library so this will be merged as soon as that's done.

[liquidsec]

@corest If you can clear the current merge conflict, we can get this merged soon

[liquidsec]

Resolved the merge conflict on this branch.

The conflict was a modify/delete: this PR was adding proxy-exclusion logic to bbot/core/helpers/web/engine.py, but that file was deleted on dev as part of the blasthttp migration (the HTTPEngine subprocess no longer exists — HTTP requests now go through WebHelper directly, wrapping the Rust blasthttp client in-process).

I ported the equivalent logic to its new home in bbot/core/helpers/web/web.py:

• WebHelper.__init__ builds a RadixTarget of exclusion patterns + bypass_all flag, only when an http_proxy is set.
• _build_proxy_exclusion_target(patterns) normalizes NO_PROXY conventions (*, .domain, *.domain).
• _proxy_excluded(url) returns True if the URL's host matches.
• _build_blasthttp_kwargs sets proxy = None per-request when _proxy_excluded is True.

Everything else from the PR (defaults.yml, args.py --no-proxy, environ.py NO_PROXY export, scanner.py, the two new tests) auto-merged cleanly.

Verified with the two tests in this PR plus a programmatic check covering *, .domain, *.domain, CIDR membership, no-proxy-configured, and NO_PROXY env export — all green.

[codecov]

Codecov Report

❌ Patch coverage is 97.36842% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90%. Comparing base (b99f5c6) to head (770a1eb).

Files with missing lines	Patch %	Lines
bbot/scanner/preset/args.py	67%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##             dev   #3000   +/-   ##
=====================================
- Coverage     90%     90%   -0%     
=====================================
  Files        441     441           
  Lines      38965   39003   +38     
=====================================
+ Hits       34881   34909   +28     
- Misses      4084    4094   +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.