Skip to content

gh-132372: Speed up logging.config existing logger handling#150242

Open
esadomer wants to merge 1 commit into
python:mainfrom
esadomer:codex/find-stronger-cpython-speedup
Open

gh-132372: Speed up logging.config existing logger handling#150242
esadomer wants to merge 1 commit into
python:mainfrom
esadomer:codex/find-stronger-cpython-speedup

Conversation

@esadomer
Copy link
Copy Markdown

gh-132372

This speeds up logging.config.fileConfig() and logging.config.dictConfig() when they need to handle many existing loggers while disable_existing_loggers is active.

The old code kept the existing logger names in a sorted list, but still used repeated list membership, index(), tail scans, and remove() calls for every configured logger. This keeps the sorted list for child logger ordering, uses a set for membership/removal tracking, and uses bisect_left() to jump directly to the dotted child logger range.

Validation:

  • PCbuild\amd64\python_d.exe -m test test_logging -j0
  • PCbuild\amd64\python_d.exe Tools\patchcheck\patchcheck.py
  • git diff --check

Benchmark:

Loaded origin/main:Lib/logging/config.py and the patched Lib/logging/config.py into the same Python 3.12.8 release interpreter, then timed only the dictConfig() call while resetting identical existing logger state outside the timer.

dictConfig-existing-loggers-old-100: Mean +- std dev: 5.68 ms +- 0.18 ms
dictConfig-existing-loggers-new-100: Mean +- std dev: 3.92 ms +- 0.11 ms
speedup: 1.45x

dictConfig-existing-loggers-old-500: Mean +- std dev: 130 ms +- 3 ms
dictConfig-existing-loggers-new-500: Mean +- std dev: 82.6 ms +- 2.6 ms
speedup: 1.57x

@eendebakpt
Copy link
Copy Markdown
Contributor

The performance increase is moderate, but the PR reduces code duplication so I am +1.

@esadomer On your benchmark I could get a much larger speedud with this small commit 2b9ce22 (on top of your work). I have not fully tested this, but the speedup is > 5x. If you are interested you could test it further and pull into your branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants