feat(auth): Password hashing parity with Python Keystone by ymh1874 · Pull Request #835 · openstack-experimental/keystone

ymh1874 · 2026-06-23T17:37:22Z

Overview

This PR completes the password hashing implementation in the Rust Keystone
service so that hashes produced by the Rust service are verifiable by the
Python service and vice versa — enabling live database migration in either
direction without requiring a password reset.

Implementation

PasswordHasher trait

A PasswordHasher trait is introduced with async fn hash and
async fn verify. One zero-size struct per algorithm mirrors the class
layout in keystone/common/password_hashers/:

Rust struct	Python class
`BcryptHasher`	`bcrypt.py::Bcrypt`
`BcryptSha256Hasher`	`bcrypt.py::Bcrypt_sha256`
`ScryptHasher`	`scrypt.py::Scrypt`
`Pbkdf2Sha512Hasher`	`pbkdf2.py::Sha512`
`PlaintextHasher`	Rust-only extension

The public API (hash_password, verify_password,
get_or_init_dummy_hash, generate_dummy_hash) signatures are
unchanged — only the internal dispatch is restructured.

Scrypt

Uses scrypt::scrypt() directly with Keystone's hardcoded parameters
(ln=16, r=8, p=1, output=32 bytes) and STANDARD_NO_PAD base64,
matching Python's binascii.b2a_base64(...).rstrip(b"=\n"). The PHC
wrapper (Scrypt::hash_password) is not used as it applies different
defaults.

PBKDF2-SHA512

The pbkdf2 = "0.12" direct dependency is removed. PBKDF2-HMAC-SHA512
is implemented directly using the workspace hmac v0.13 + sha2 v0.11
following RFC 2898 §5.2. This eliminates a digest v0.10/v0.11 version
conflict that would otherwise arise from pbkdf2 v0.12's internal sha2.

Dummy-hash cache reset on config reload

reset_dummy_hash_cache() clears the (algorithm, rounds)-keyed dummy
hash cache. It is wired to ConfigManager::notify_tx in
crates/keystone/src/bin/keystone.rs via a reset_dummy_hash_on_reload
task that subscribes to the broadcast channel and clears the cache on
every config reload (lagged receivers are also treated as a reload since
clearing is always safe).

Tooling

tools/generate_kats.py — updated with the correct pre-dispatch
truncation policy (BCRYPT_MAX_LENGTH = 72, truncate() helper)
mirroring verify_length_and_trunc_password.
tools/cross_verify.py — new script for bidirectional verification
(see below).

Verification suite

The PR ships a two-directional verification suite that guarantees
compatibility for anyone migrating between the Python and Rust services.

How it works

Direction 1 — Python produces, Rust verifies (KAT vectors)

Seven known-answer test vectors are generated from the real, unmodified
Python Keystone hashers (bcrypt 5.0, cryptography 49.0) using
tools/generate_kats.py. These vectors are hardcoded into
#[tokio::test] cases and verified by verify_password on every CI
run — no Python runtime needed at test time.

Coverage: all 4 algorithms, including the 72-byte bcrypt truncation
boundary, 73-byte inputs to BcryptSha256/Scrypt/PBKDF2 (must NOT
truncate), and multi-byte UTF-8 passwords.

Direction 2 — Rust produces, Python verifies (cross-verify)

tools/cross_verify.py takes an algorithm name, a plaintext password,
and a hash string, imports the real keystone.common.password_hashers
classes, and exits 0 (verified), 1 (rejected), or 2 (error). It is
the definitive oracle for whether a Rust-produced hash is acceptable
to the Python service.

The four test_cross_verify_* tests in the Rust suite call this script
via tokio::process::Command. They are gated on the
KEYSTONE_PYTHON_CHECKOUT environment variable and skipped in standard
CI — set it to a real Keystone checkout to run them locally:

KEYSTONE_PYTHON_CHECKOUT=~/Projects/openstack/keystone \
  cargo test -p openstack-keystone-core -- cross_verify

Both directions were confirmed end-to-end before this PR was opened.
Correct password → exit 0 and wrong password → exit 1 for all four
algorithms in both directions.

Running the verification suite

# Standard CI (KAT vectors only — no Python required)
cargo test -p openstack-keystone-core

# Full bidirectional (requires a real Keystone Python checkout)
KEYSTONE_PYTHON_CHECKOUT=~/Projects/openstack/keystone \
  cargo test -p openstack-keystone-core -- cross_verify

# Regenerate KAT vectors from Python (run from the Keystone checkout)
cd ~/Projects/openstack/keystone
python tools/generate_kats.py

# Spot-check a single Rust-produced hash against Python
python tools/cross_verify.py bcrypt "mypassword" "<hash>"

Open items for discussion

Cache reset option — implemented as option (a): pub-sub via
ConfigManager::notify_tx. Option (b) (provider hook via
ProviderHooks) is an alternative if preferred.
Missing config fields — scrypt_block_size, scrypt_parallelism,
salt_bytesize, strict_password_check are not yet in
IdentityProvider; Keystone's hardcoded defaults are used. Suggested
as follow-up issues.
ADR — the PasswordHasher trait and cache lifecycle design may
warrant an ADR per the spec process.
$2$ / $2x$ bcrypt idents — recognised by detect_algo but
not tested; low-priority follow-up.

Supersedes #793.

Note: This PR was prepared with the help of AI

gtema

This looks great, thanks.

This is not a very thorough review, but gives a further recommendations to make the review easier (one huge file is hard to review)

gtema

let's fix this one issue with the unnecessary test submodule and land the change. Afterwards we continue polishing it

Add BcryptSha256, Scrypt, Pbkdf2Sha512 enum variants. Wire get_or_init_dummy_hash with double-checked locking. Replace Passlib-specific base64 with standard STANDARD/STANDARD_NO_PAD engines. PBKDF2 now emits bare-integer format and reads configured rounds. Replace magic numbers with named bcrypt constants. Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

One struct per algorithm (BcryptHasher, BcryptSha256Hasher, ScryptHasher, Pbkdf2Sha512Hasher, PlaintextHasher), each mirroring the corresponding keystone.common.password_hashers Python class. ScryptHasher now uses ln=16,r=8,p=1 and STANDARD_NO_PAD to match Python Keystone's wire format exactly. PBKDF2 is implemented directly with workspace hmac/sha2 to avoid a digest v0.10/v0.11 type conflict with pbkdf2 v0.12. Cache reset hook added for config-reload wiring. KAT generator gains correct pre-dispatch truncation policy. Adds tools/cross_verify.py for bidirectional Python compatibility testing. Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

Subscribe to ConfigManager::notify_tx in the keystone binary and clear the dummy-password-hash cache on every configuration reload. The cache is keyed by (algorithm, rounds); without this, changing password_hashing_algorithm or password_hash_rounds at runtime would keep serving stale dummy hashes and reopen the timing side-channel the dummy hash exists to close. A lagged receiver is treated as a reload and still clears the cache, which is always safe. Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

Add env-gated tests that hash a password with the Rust hashers and verify the result through tools/cross_verify.py against the real Keystone Python hashers, closing the loop the one-way KAT vectors miss. Gated on KEYSTONE_PYTHON_CHECKOUT so they skip cleanly without a Python install; tokio's process feature is added as a dev-only dependency. Also strengthens the BcryptSha256 verify comment explaining why the 2-digit cost pad is required, and fixes two needless-borrow lints. Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

Replace the cross-compatibility hash vectors, previously produced by a libxcrypt-based reimplementation, with output from the real unmodified Keystone Python hashers (bcrypt 5.0, cryptography 49.0). Each vector was self-verified by the producing Python hasher. Regenerating the 73-byte bcrypt_sha256 case surfaced that the prior vector had been truncated to 72 bytes; the corrected vector confirms BcryptSha256 does not truncate, matching Keystone. Both directions now verified end to end: Rust accepts real Python hashes (these tests) and Python accepts Rust hashes (the env-gated cross_verify tests). Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

- Sort PasswordHashingAlgo variants alphabetically - Move each hasher into its own file under password_hashing/ with colocated unit tests - Rewrite stale BcryptSha256 salt comment; explain the canonical Radix64 encoding invariant clearly - Centralize salt generation into generate_salt() in mod.rs - Replace all em-dash characters with ASCII hyphens - Restore Cargo.lock to upstream/password_hash baseline (eliminates openraft alpha.23 version skew) Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

Drop the separate pub(crate) mod test_support submodule. TEST_PASSWORD and mock_config are now #[cfg(test)] items directly in mod.rs; child modules access them via the private ancestor path (super::super::mock_config) without needing pub(crate). Also set up Python Keystone in CI so the cross-verify tests run for real instead of being silently skipped: clone openstack/keystone, install bcrypt + cryptography, and set KEYSTONE_PYTHON_CHECKOUT for the unit test step. Note: This commit was done with the help of AI Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

Note: This commit was done with the help of AI. Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

cross_verify.py was invoked via absolute path so Python's sys.path[0] was the script dir, not the keystone checkout. Add os.getcwd() to sys.path so the checkout is always importable regardless of how the script is called. Update CI to install the full keystone requirements.txt (not just bcrypt + cryptography) since the import chain pulls in oslo.messaging, oslo.cache and more. Note: This commit was done with the help of AI. Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

gtema · 2026-06-25T14:33:13Z

+/// Password reused across the Keystone cross-compatibility test vectors.
+#[cfg(test)]
+const TEST_PASSWORD: &str = "openstack123";
+
+/// Build a `Config` with the password-hashing fields set.
+///
+/// Accessible from every child module's test block via `super::super::mock_config`
+/// without needing `pub(crate)` — child modules can reach private items in their
+/// ancestor modules.
+#[cfg(test)]
+fn mock_config(algo: openstack_keystone_config::PasswordHashingAlgo, max_len: usize) -> Config {
+    let mut conf = Config::default();
+    conf.identity.password_hashing_algorithm = algo;
+    conf.identity.password_hash_rounds = Some(12);
+    conf.identity.max_password_length = max_len;
+    conf
+}
+
+#[cfg(test)]
+mod tests {


please merge all of this into the single tests submodule

gtema · 2026-06-25T19:15:51Z

        uses: swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0

+      - name: Set up Python for cross-verification tests
+        uses: actions/setup-python@v5


please use the tagged action for setup-python

gtema · 2026-06-25T19:17:48Z

+        with:
+          python-version: '3.x'
+
+      - name: Clone Python Keystone and install dependencies


it's actually not necessary to clone the repo. What you need to ensure is that keystone is installed for the python that is invoked by the test. It should be enough just to do "pip install keystone" and that's it, not even to set the checkout env

ymh1874 force-pushed the feature/password-hashing branch from ab782e6 to ce46b1f Compare June 23, 2026 17:50

ymh1874 marked this pull request as ready for review June 23, 2026 17:54

gtema requested changes Jun 23, 2026

View reviewed changes

gtema requested changes Jun 25, 2026

View reviewed changes

Comment thread crates/core/src/common/password_hashing/mod.rs

Comment thread crates/core/src/common/password_hashing/mod.rs

ymh1874 added 7 commits June 25, 2026 13:59

ymh1874 force-pushed the feature/password-hashing branch from 5703ff6 to b4fc7c5 Compare June 25, 2026 10:59

chore: Ignore CLAUDE.md from version control

424108c

Note: This commit was done with the help of AI. Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

ymh1874 force-pushed the feature/password-hashing branch from e7b8a94 to 424108c Compare June 25, 2026 11:01

ymh1874 added 2 commits June 25, 2026 14:03

chore: Revert CLAUDE.md gitignore entry

6f22191

Note: This commit was done with the help of AI. Signed-off-by: Yousef Hussein <ymh1874@gmail.com>

ymh1874 requested a review from gtema June 25, 2026 11:19

gtema reviewed Jun 25, 2026

View reviewed changes

gtema requested changes Jun 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(auth): Password hashing parity with Python Keystone#835

feat(auth): Password hashing parity with Python Keystone#835
ymh1874 wants to merge 10 commits into
openstack-experimental:password_hashfrom
ymh1874:feature/password-hashing

ymh1874 commented Jun 23, 2026 •

edited

Loading

Uh oh!

gtema left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gtema left a comment

Uh oh!

Uh oh!

Uh oh!

gtema Jun 25, 2026

Uh oh!

gtema Jun 25, 2026

Uh oh!

gtema Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

ymh1874 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Implementation

PasswordHasher trait

Scrypt

PBKDF2-SHA512

Dummy-hash cache reset on config reload

Tooling

Verification suite

How it works

Running the verification suite

Open items for discussion

Uh oh!

gtema left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gtema left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gtema Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

gtema Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

gtema Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ymh1874 commented Jun 23, 2026 •

edited

Loading