backport: Merge bitcoin#27005, 28644, 26940, 27511, 28403, 28361, 28409, 28352#7245
backport: Merge bitcoin#27005, 28644, 26940, 27511, 28403, 28361, 28409, 28352#7245vijaydasmp wants to merge 8 commits intodashpay:developfrom
Conversation
✅ No Merge Conflicts DetectedThis PR currently has no conflicts with other open PRs. |
b35f9dc to
50ee95f
Compare
23729db to
a9026d7
Compare
fad7af7 Use steady clock for logging timer (MarcoFalke) Pull request description: The logging timer has many issues: * The underlying clock is mockable, meaning that benchmarks are useless when mocktime was set at the beginning or end of the benchmark. * The underlying clock is not monotonic, meaning that benchmarks are useless when the system time was changed during the benchmark. Fix all issues in this patch. ACKs for top commit: stickies-v: Approach ACK fad7af7 john-moffett: ACK fad7af7 Tree-SHA512: bec8da0f338ed4611e1807937575e1b2afda25139d88015b1c29fa7d13946fbfbc4ee589b576c0508d505df5e5fafafcbc07d63ce4bab4b01475260d9d5d2107
faa190b test: Fuzz merge with -use_value_profile=0 for now (MarcoFalke) Pull request description: Seems odd that this has to be done, but for now there are (unknown) size limits on the qa-assets repo. Also, a larger size means that cloning and iterating over the files takes a longer time. Not sure how to measure the net impact of this, but with some backups reverting this commit, it can be limited on the downside? ACKs for top commit: dergoegge: ACK faa190b Tree-SHA512: 9f8b3f4526f60e4ff6fca97859a725d145a8339c216bd15c92fad7e53f84308745fee47727527de459c0245ef9d474a9dc836fee599ab2b556b519bd900b9a33
…helper, dedupe add_coin 4275195 De-duplicate add_coin methods to a test util helper (Jon Atack) 9d92c3d Create InsecureRandMoneyAmount() test util helper (Jon Atack) 81f5ade Move random test util code from setup_common to random (Jon Atack) Pull request description: - Move random test utilities from `setup_common` to a new `random` file, as many tests don't use this code. - Create a helper to generate semi-random CAmounts up to `MONEY_RANGE` rather than only uint32, and use the helper in the unit tests. - De-duplicate a shared `add_coin` method by extracting it to a `coins` test utility. ACKs for top commit: pinheadmz: ACK 4275195 achow101: ACK 4275195 john-moffett: ACK 4275195 Tree-SHA512: 3ed974251149c7417f935ef2f8865aa0dcc33b281b47522b0f96f1979dff94bb8527957f098fe4d210f40d715c00f29512f2ffe189097102229023b7284a3a27 Merge bitcoin#26940: test: create random and coins utils, add amount helper, dedupe add_coin 4275195 De-duplicate add_coin methods to a test util helper (Jon Atack) 9d92c3d Create InsecureRandMoneyAmount() test util helper (Jon Atack) 81f5ade Move random test util code from setup_common to random (Jon Atack) Pull request description: - Move random test utilities from `setup_common` to a new `random` file, as many tests don't use this code. - Create a helper to generate semi-random CAmounts up to `MONEY_RANGE` rather than only uint32, and use the helper in the unit tests. - De-duplicate a shared `add_coin` method by extracting it to a `coins` test utility. ACKs for top commit: pinheadmz: ACK 4275195 achow101: ACK 4275195 john-moffett: ACK 4275195 Tree-SHA512: 3ed974251149c7417f935ef2f8865aa0dcc33b281b47522b0f96f1979dff94bb8527957f098fe4d210f40d715c00f29512f2ffe189097102229023b7284a3a27 Merge bitcoin#26940: test: create random and coins utils, add amount helper, dedupe add_coin 4275195 De-duplicate add_coin methods to a test util helper (Jon Atack) 9d92c3d Create InsecureRandMoneyAmount() test util helper (Jon Atack) 81f5ade Move random test util code from setup_common to random (Jon Atack) Pull request description: - Move random test utilities from `setup_common` to a new `random` file, as many tests don't use this code. - Create a helper to generate semi-random CAmounts up to `MONEY_RANGE` rather than only uint32, and use the helper in the unit tests. - De-duplicate a shared `add_coin` method by extracting it to a `coins` test utility. ACKs for top commit: pinheadmz: ACK 4275195 achow101: ACK 4275195 john-moffett: ACK 4275195 Tree-SHA512: 3ed974251149c7417f935ef2f8865aa0dcc33b281b47522b0f96f1979dff94bb8527957f098fe4d210f40d715c00f29512f2ffe189097102229023b7284a3a27
…ied table address count 28bac81 test: add functional test for getaddrmaninfo (stratospher) c8eb8da rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table (stratospher) Pull request description: implements bitcoin#26907. split off from bitcoin#26988 to keep RPC, CLI discussions separate. This PR introduces a new RPC `getaddrmaninfo`which returns the count of addresses in the new/tried table of a node's addrman broken down by network type. This would be useful for users who want to see the distribution of addresses from different networks across new/tried table in the addrman. ```jsx $ getaddrmaninfo Result: { (json object) json object with network type as keys "network" : { (json object) The network (ipv4, ipv6, onion, i2p, cjdns) "new" : n, (numeric) number of addresses in new table "tried" : n, (numeric) number of addresses in tried table "total" : n (numeric) total number of addresses in both new/tried tables from a network }, ... } ``` ### additional context from [original PR](bitcoin#26988) 1. network coverage tests were skipped because there’s a small chance that addresses from different networks could hash to the same bucket and cause count of different network addresses in the tests to fail. see bitcoin#26988 (comment). 2. bitcoin#26988 uses this RPC in -addrinfo CLI. Slight preference for keeping the RPC hidden since this info will mostly be useful to only super users. see bitcoin#26988 (comment). ACKs for top commit: 0xB10C: ACK 28bac81 willcl-ark: reACK 28bac81 achow101: ACK 28bac81 brunoerg: reACK 28bac81 theStack: Code-review ACK 28bac81 Tree-SHA512: 346390167e1ebed7ca5c79328ea452633736aff8b7feefea77460e04d4489059334ae78a3f757f32f5fb7827b309d7186bebab3c3760b3dfb016d564a647371a
…termittent issues fa28f5a test: Bump walletpassphrase timeouts to avoid intermittent issues (MarcoFalke) Pull request description: This bumps all timeouts for all `walletpassphrase` to avoid intermittent issues in `valgrind` (or other sanitizers). As an idea for a follow-up, `walletpassphrase` could be changed to treat `0` as "no timeout" instead of "instant timeout". Example failure: ``` node0 2023-09-03T22:44:38.374955Z [httpworker.3] [rpc/server.cpp:594] [RPCRunLater] [rpc] queue run of timer lockwallet(w6) in 60 seconds (using HTTP) test 2023-09-03T22:44:40.173000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'getnewaddress', '', 'legacy'] node0 2023-09-03T22:44:59.810893Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:48928 node0 2023-09-03T22:44:59.813132Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getnewaddress user=__cookie__ node0 2023-09-03T22:44:59.837183Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?) node0 2023-09-03T22:44:59.929735Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?) node0 2023-09-03T22:44:59.934484Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?) node0 2023-09-03T22:44:59.935467Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?) test 2023-09-03T22:45:02.328000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'signmessage', 'mqatqH4VQmrZ81nxUfrnfcLnxgbzhZb4PC', 'test'] node0 2023-09-03T22:45:20.269375Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:44618 node0 2023-09-03T22:45:20.270670Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=signmessage user=__cookie__ test 2023-09-03T22:45:23.490000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'keypoolrefill', '1'] node0 2023-09-03T22:45:40.244603Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:32854 node0 2023-09-03T22:45:40.293021Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=keypoolrefill user=__cookie__ test 2023-09-03T22:45:41.852000Z TestFramework (ERROR): JSONRPC error Traceback (most recent call last): File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 131, in main self.run_test() File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_createwallet.py", line 156, in run_test w6.keypoolrefill(1) File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 732, in __call__ return self.cli.send_cli(self.command, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 795, in send_cli raise JSONRPCException(dict(code=int(code), message=message)) test_framework.authproxy.JSONRPCException: Error: Please enter the wallet passphrase with walletpassphrase first. (-13) ACKs for top commit: achow101: ACK fa28f5a Tree-SHA512: 58caa569cec39acc121d4cc038a4190937af34e85d2696272ed4f2792fd386469b0cfefd2cb564438fedded97b21b23d8bf46ba27b5633671a277ed4679f0d5d
1580e3b fuzz: add ConstructPubKeyBytes function (josibake) Pull request description: In bitcoin#28246 and bitcoin#28122 , we add a `PubKeyDestination` and a `V0SilentPaymentsDestination`. Both of these PRs update `fuzz/util.cpp` and need a way to create well-formed pubkeys. Currently in `fuzz/util.cpp`, we have some logic for creating pubkeys in the multisig data provider. This logic is duplicated in bitcoin#28246 and duplicated again in bitcoin#28122. Seems much better to have a `ConstructPubKeyBytes` function that both PRs (and any future work) can reuse. This PR introduces a function to do this and has the existing code use it. While the purpose is to introduce a utility function, the previous multisig code used `ConsumeIntegralInRange(4, 7)` which would have created some uncompressed pubkeys with the prefix 0x05, which is incorrect (see https://bitcoin.stackexchange.com/questions/57855/c-secp256k1-what-do-prefixes-0x06-and-0x07-in-an-uncompressed-public-key-signif) tldr; using `PickValueFromArray` is more correct as it limits to the set of defined prefixes for compressed and uncompressed pubkeys. ACKs for top commit: Sjors: ACK 1580e3b Tree-SHA512: c87c8bcd1f6b3a97ef772be93102efb912811c59f32211cfd531a116f1da8a57c8c6ff106b34f2a2b88d8b34fb5bc30d9f9ed6d2720113ffcaaa2f8d5dc9eb27
…ping fae0b21 test: Combine sync_send_with_ping and sync_with_ping (MarcoFalke) Pull request description: This reduces bloat, complexity, and makes tests less fragile to intermittent failures, see bitcoin#27675 (comment). This should not cause any noticeable slowdown, or may even be faster, because active polling will be done at most once. ACKs for top commit: glozow: Concept ACK fae0b21 theStack: ACK fae0b21 🏓 Tree-SHA512: 6c543241a7b85458dc7ff6a6203316b80a6227d83d38427e74f53f4c666a882647a8a221e5259071ee7bb5dfd63476fb03c9b558a1ea546734b14728c3c619ba
…es.py faf7e69 test: Support powerpc64le in get_previous_releases.py (MarcoFalke) Pull request description: To test: `test/get_previous_releases.py -b -t /tmp/prev_releases v22.0` On master: `Not sure which binary to download for powerpc64le-unknown-linux-gnu` Here: (pass) ACKs for top commit: fanquake: ACK faf7e69 Tree-SHA512: 33d9348f99e0d3924a6a5cba8833ec9e413e80167012b557922f3628069dabd555b02f98a6bfd0eb80e2bbbcdb50865b7bca216e1d080b1546ee4812abda4bc2
Review GateCommit:
|
WalkthroughThis pull request primarily reorganizes test infrastructure and introduces a new RPC command. It moves random utility functions from Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
test/functional/p2p_ibd_stalling.py (1)
161-164: Consider renaming the helper for clarity.The method now calls
sync_with_ping(), soall_sync_send_with_pingis slightly misleading. A rename would reduce cognitive overhead in this test.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/functional/p2p_ibd_stalling.py` around lines 161 - 164, Rename the helper function all_sync_send_with_ping to a clearer name that reflects it calls each peer's sync_with_ping method (e.g., sync_all_with_ping or peers_sync_with_ping); update the function definition (all_sync_send_with_ping) and every call site to the new name, and ensure any tests or imports referencing the old name are adjusted so behavior remains identical and no references remain to all_sync_send_with_ping.src/test/fuzz/util.cpp (1)
319-329: Guard against shortbyte_dataspans inConstructPubKeyBytes.This helper assumes at least 33/65 bytes and does unchecked iterator arithmetic. Adding a precondition assert makes the function safer for future reuse.
♻️ Proposed hardening
std::vector<uint8_t> ConstructPubKeyBytes(FuzzedDataProvider& fuzzed_data_provider, Span<const uint8_t> byte_data, const bool compressed) noexcept { + const size_t needed = compressed ? CPubKey::COMPRESSED_SIZE : CPubKey::SIZE; + assert(byte_data.size() >= needed); uint8_t pk_type; if (compressed) { pk_type = fuzzed_data_provider.PickValueInArray({0x02, 0x03}); } else { pk_type = fuzzed_data_provider.PickValueInArray({0x04, 0x06, 0x07}); } - std::vector<uint8_t> pk_data{byte_data.begin(), byte_data.begin() + (compressed ? CPubKey::COMPRESSED_SIZE : CPubKey::SIZE)}; + std::vector<uint8_t> pk_data{byte_data.begin(), byte_data.begin() + needed}; pk_data[0] = pk_type; return pk_data; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/test/fuzz/util.cpp` around lines 319 - 329, ConstructPubKeyBytes assumes byte_data has at least CPubKey::COMPRESSED_SIZE or CPubKey::SIZE bytes; add a guard at the start of ConstructPubKeyBytes that checks byte_data.size() >= (compressed ? CPubKey::COMPRESSED_SIZE : CPubKey::SIZE) and assert or return an empty/std::vector<uint8_t>() if the precondition fails, then proceed to PickValueInArray and slice using the validated size (referencing ConstructPubKeyBytes, fuzzed_data_provider, CPubKey::COMPRESSED_SIZE, CPubKey::SIZE).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/get_previous_releases.py`:
- Around line 267-270: The platforms mapping added an entry for
'powerpc64le-*-linux-*' -> 'powerpc64le-linux-gnu' but there are no
corresponding SHA256_SUMS entries and download_binary() rejects tarballs without
checksum entries; either revert/remove the 'powerpc64le-*-linux-*' entry from
the platforms dict (so host detection keeps it unmapped) or add the official
Dash release assets and matching checksum lines for 'powerpc64le-linux-gnu' into
the SHA256_SUMS data used by download_binary() and ensure the checksum keys
match exactly the platform string; update the SHA256_SUMS source and test
download_binary() to accept those checksums if you choose to add support.
---
Nitpick comments:
In `@src/test/fuzz/util.cpp`:
- Around line 319-329: ConstructPubKeyBytes assumes byte_data has at least
CPubKey::COMPRESSED_SIZE or CPubKey::SIZE bytes; add a guard at the start of
ConstructPubKeyBytes that checks byte_data.size() >= (compressed ?
CPubKey::COMPRESSED_SIZE : CPubKey::SIZE) and assert or return an
empty/std::vector<uint8_t>() if the precondition fails, then proceed to
PickValueInArray and slice using the validated size (referencing
ConstructPubKeyBytes, fuzzed_data_provider, CPubKey::COMPRESSED_SIZE,
CPubKey::SIZE).
In `@test/functional/p2p_ibd_stalling.py`:
- Around line 161-164: Rename the helper function all_sync_send_with_ping to a
clearer name that reflects it calls each peer's sync_with_ping method (e.g.,
sync_all_with_ping or peers_sync_with_ping); update the function definition
(all_sync_send_with_ping) and every call site to the new name, and ensure any
tests or imports referencing the old name are adjusted so behavior remains
identical and no references remain to all_sync_send_with_ping.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 600d104e-11b6-4612-9e78-a9f8b7ab2e16
📒 Files selected for processing (62)
src/Makefile.test_util.includesrc/logging/timer.hsrc/rpc/net.cppsrc/test/base58_tests.cppsrc/test/bip324_tests.cppsrc/test/blockencodings_tests.cppsrc/test/bloom_tests.cppsrc/test/checkqueue_tests.cppsrc/test/coins_tests.cppsrc/test/crypto_tests.cppsrc/test/cuckoocache_tests.cppsrc/test/dbwrapper_tests.cppsrc/test/fuzz/rpc.cppsrc/test/fuzz/util.cppsrc/test/hash_tests.cppsrc/test/key_tests.cppsrc/test/logging_tests.cppsrc/test/merkle_tests.cppsrc/test/miner_tests.cppsrc/test/minisketch_tests.cppsrc/test/net_tests.cppsrc/test/orphanage_tests.cppsrc/test/pmt_tests.cppsrc/test/pool_tests.cppsrc/test/pow_tests.cppsrc/test/prevector_tests.cppsrc/test/script_tests.cppsrc/test/serfloat_tests.cppsrc/test/sighash_tests.cppsrc/test/skiplist_tests.cppsrc/test/streams_tests.cppsrc/test/transaction_tests.cppsrc/test/txpackage_tests.cppsrc/test/util/blockfilter.cppsrc/test/util/coins.cppsrc/test/util/coins.hsrc/test/util/llmq_tests.hsrc/test/util/random.hsrc/test/util/setup_common.hsrc/test/util_tests.cppsrc/test/validation_block_tests.cppsrc/test/validation_chainstate_tests.cppsrc/test/validation_chainstatemanager_tests.cppsrc/test/validation_flush_tests.cppsrc/test/versionbits_tests.cppsrc/wallet/test/wallet_crypto_tests.cpptest/functional/p2p_addrfetch.pytest/functional/p2p_blocksonly.pytest/functional/p2p_compactblocks_blocksonly.pytest/functional/p2p_filter.pytest/functional/p2p_ibd_stalling.pytest/functional/p2p_ibd_txrelay.pytest/functional/rpc_net.pytest/functional/test_framework/p2p.pytest/functional/wallet_createwallet.pytest/functional/wallet_descriptor.pytest/functional/wallet_dump.pytest/functional/wallet_encryption.pytest/functional/wallet_fundrawtransaction.pytest/functional/wallet_keypool.pytest/fuzz/test_runner.pytest/get_previous_releases.py
💤 Files with no reviewable changes (2)
- src/test/util/blockfilter.cpp
- src/test/util/setup_common.h
| platforms = { | ||
| 'aarch64-*-linux*': 'aarch64-linux-gnu', | ||
| 'powerpc64le-*-linux-*': 'powerpc64le-linux-gnu', | ||
| 'x86_64-*-linux*': 'x86_64-linux-gnu', |
There was a problem hiding this comment.
powerpc64le now passes host detection but still can't download any Dash release.
Line 269 enables the platform, but this file still has no powerpc64le-linux-gnu entries in SHA256_SUMS, and download_binary() rejects tarballs without a matching checksum entry. On a powerpc64le host this now gets past detection only to fail later for every default tag. Please either add the matching release assets/checksums or keep the host unmapped until Dash publishes supported binaries.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/get_previous_releases.py` around lines 267 - 270, The platforms mapping
added an entry for 'powerpc64le-*-linux-*' -> 'powerpc64le-linux-gnu' but there
are no corresponding SHA256_SUMS entries and download_binary() rejects tarballs
without checksum entries; either revert/remove the 'powerpc64le-*-linux-*' entry
from the platforms dict (so host detection keeps it unmapped) or add the
official Dash release assets and matching checksum lines for
'powerpc64le-linux-gnu' into the SHA256_SUMS data used by download_binary() and
ensure the checksum keys match exactly the platform string; update the
SHA256_SUMS source and test download_binary() to accept those checksums if you
choose to add support.
bitcoin backports