sha256: smoke-test caller-supplied compression function - #1905
sha256: smoke-test caller-supplied compression function#1905real-or-random wants to merge 1 commit into
Conversation
The existing selftest hashes one 63 byte string against a known digest. Which catches a compression function that is wrong everywhere, but not one that is wrong on multi-block calls, unaligned input, or a state that is not the IV. This introduces secp256k1_sha256_smoke_test, which hashes messages of various lengths and starting offsets with the supplied function, accumulates the resulting digests with supplied function into a single one, compares it to the precomputed correct value. The check runs once during startup, so a faulty compression function is caught early rather than silently producing incorrect output later. Measured locally, this takes well below 1ms, which should be negligible for any application. See the introduced test for a clear view of the bugs this catches. Co-authored-by: Tim Ruffing <me@real-or-random.org>
76952c1 to
c1c6365
Compare
|
Differences:
|
There was a problem hiding this comment.
Pull request overview
This PR strengthens validation of caller-supplied SHA256 compression callbacks by adding a multi-case “smoke test” that cross-checks behavior across message lengths and input alignments, and integrating it into the existing SHA256 selftest pathway. This helps catch subtle but dangerous misimplementations (e.g., multi-block handling, state handling, or alignment assumptions) early when installing a custom compression function.
Changes:
- Add
secp256k1_sha256_smoke_test()to validate supplied SHA256 compression callbacks against a precomputed accumulated digest. - Extend
secp256k1_selftest_sha256()to run the smoke test for non-default compression functions after the existing known-digest check. - Add unit tests covering several intentionally broken compression callbacks plus a known-good callback wrapper.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/tests.c | Adds targeted unit tests exercising the new smoke test against good/bad compression callbacks. |
| src/selftest.h | Extends SHA256 selftest to run the new smoke test for non-default compression functions. |
| src/hash_impl.h | Introduces the new SHA256 compression smoke test implementation. |
| include/secp256k1.h | Documents the new behavior when installing a custom SHA256 compression function. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /* sizeof(msg) = length of longest message | ||
| * + 2 * MAX_ALIGNMENT - 1 for the offset in the initial value of m | ||
| * + 2 * MAX_ALIGNMENT - 1 for the offset in the final value of m */ |
| return (secp256k1_memcmp_var(accum_expected, out, 32) == 0); | ||
| #undef LONGEST | ||
| #undef MAX_ALIGNMENT | ||
| } |
| * Note: The provided function is tested against a set of known SHA256 | ||
| * digests, aborting on any mismatch, in order to catch basic misbehavior | ||
| * early. Takes well under 1 ms on a desktop machine. | ||
| * This is NOT a substitute for having proper test coverage of the | ||
| * supplied function outside this library. |
theStack
left a comment
There was a problem hiding this comment.
ACK c1c6365
Being thorough with alignment checks seems the right approach, even though it adds quite a bit of complexity to the code (it LGTM, but it's very easy to get something wrong there or overlook something as reviewer).
The following Copilot suggestions seem to make sense to tackle:
| * typically hashes four or eight at a time, then any left over one by one. | ||
| * These lengths cover every number from 1 to 9, which includes counts that | ||
| * divide evenly and counts leaving one, two or three over. */ | ||
| #define LONGEST 576 /* local macro, undef'd below */ |
There was a problem hiding this comment.
nit: an alternative here and for MAX_ALIGNMENT below would be
| #define LONGEST 576 /* local macro, undef'd below */ | |
| enum { LONGEST = 576 }; |
for not having to #undef below, leading to a smaller patch; but not sure if that's strictly better or if we even have a preference in the project (we seem to use both)
There was a problem hiding this comment.
I tend to think that enum is better in this case. I always forget that it's a compile-time constant that can be used in array sizes.
| secp256k1_sha256_initialize(&sha_accum); | ||
|
|
||
| /* Make msg a pointer into msg_buf aligned to 2 * MAX_ALIGNMENT boundary. */ | ||
| offset = ((2 * MAX_ALIGNMENT) - ((uintptr_t)msg_buf % (2 * MAX_ALIGNMENT))) % (2 * MAX_ALIGNMENT); |
There was a problem hiding this comment.
nit-like (?): uintptr_t does not seem to be part of C89; it was introduced in C99 and is optional even there. Not sure how strict we want to be about this level of portability, though, given that we already rely on stdint.h being available.
There was a problem hiding this comment.
Hm, yes. Almost all architectures have an uintptr_t, but some don't.
The reason not to provide one is that there is no integer type large enough to hold a pointer value so that it can be converted back without loss. We don't need to conversion back, but we still can't just cast to any unsigned integer type because C99 says: If the result cannot be represented in the integer type, the behavior is undefined.
We could ignore the problem and simply require uintptr_t. We could wrap the offset calculation into a preprocessor check that checks for the presence of the type (#ifdef UINTPTR_MAX) and fall back if it's not present (e.g., simply set offset = 0, effectively not doing alignment checks) for these rare architectures.
[1] CHERI archs can't provide it it according to https://www.ralfj.de/blog/2022/04/11/provenance-exposed.html. edit: I was wrong here. CHERI-C has it, and it's 64 bits long. According to https://www.open-std.org/JTC1/SC22/WG14/www/docs/n2873.htm, there are at least two other architectures, but they're not mentioned explicitly in the document.
furszy
left a comment
There was a problem hiding this comment.
Simple coverage for the introduced changes:
diff --git a/src/tests.c b/src/tests.c
--- a/src/tests.c
+++ b/src/tests.c
@@ -517,12 +517,28 @@
s[0] ^= 1;
}
+/* Wrong when input is 64-byte aligned, like a broken SIMD fast path. */
+static void sha256_transform_aligned_fail(uint32_t *s, const unsigned char *chunk, size_t blocks) {
+ int aligned = ((uintptr_t)chunk % 64) == 0;
+ secp256k1_sha256_transform(s, chunk, blocks);
+ if (aligned) s[0] ^= 1;
+}
+
+/* Wrong on any unaligned input. */
+static void sha256_transform_unaligned_fail(uint32_t *s, const unsigned char *chunk, size_t blocks) {
+ int aligned = ((uintptr_t)chunk % 64) == 0;
+ secp256k1_sha256_transform(s, chunk, blocks);
+ if (!aligned) s[0] ^= 1;
+}
+
static void run_sha256_compression_smoke_test_tests(void) {
CHECK(secp256k1_sha256_smoke_test(sha256_transform_noadvance) == 0);
CHECK(secp256k1_sha256_smoke_test(sha256_transform_short) == 0);
CHECK(secp256k1_sha256_smoke_test(sha256_transform_ivreset) == 0);
CHECK(secp256k1_sha256_smoke_test(sha256_transform_batch4) == 0);
CHECK(secp256k1_sha256_smoke_test(sha256_transform_corrupt) == 0);
+ CHECK(secp256k1_sha256_smoke_test(sha256_transform_aligned_fail) == 0);
+ CHECK(secp256k1_sha256_smoke_test(sha256_transform_unaligned_fail) == 0);
CHECK(secp256k1_sha256_smoke_test(good_sha256_compression) == 1);
}
originally posted in bitcoin-core#1905 (review)
Extended version of #1904. I'd say if we get enough reviews/confidence before the release, let's merge this here. Otherwise, let's stick with the simpler #1904 and leave this here for after the release.