feat(cpu): Blake3/Alephium CPU mining — first CPU algorithm#172
Conversation
b38c058 to
dbca62a
Compare
luminousmining
left a comment
luminousmining
left a comment
There was a problem hiding this comment.
I have only looked at the first files.
I will do a more thorough analysis later.
dbca62a to
f3a2304
Compare
Add a CPU mining path for BLAKE3 (Alephium): a no-SDK DeviceCpu that mirrors the mocker device and runs on the global BLAKE3 pool, a ResolverCpuBlake3 nonce scanner, and a self-owned persistent pinned std::jthread pool (CpuThreadPool) exposing --cpu_threads (pool size) and --cpu_affinity (hex core bitmask). The pool replaces a serial scan with real multicore on every platform (the previous OpenMP pragma was a no-op under clang-cl, leaving Windows single-threaded). Also enumerate the CPU device, map BLAKE3 to it, and show CPU hashrate in the dashboard.
Make the CPU resolver an independent build axis from the GPU backend: GPU=amd|nvidia|both|none combined with CPU=ON|OFF across docker/Dockerfile.linux, docker/Dockerfile.windows-cross, and scripts/docker-build.ps1, with CI building CPU-only images (GPU=none CPU=ON). CMake gains BUILD_CPU + the CPU_ENABLE define and links the host blake3_ref/blake3_pow under BUILD_CPU. The CPU resolver parallelizes via an in-process std::thread pool, so no OpenMP/libomp runtime is installed or staged.
Update ARCHITECTURE.md and CLAUDE.md (CPU resolver = std::thread pool), document --cpu_threads/--cpu_affinity in PARAMETERS.md, add CPU_TUNING.md (thread-count/affinity guidance incl. reserving cores for the GPU), and refresh the docker build docs for the GPU/CPU axes.
- Self-guard resolver/cpu/blake3.hpp with CPU_ENABLE so device.cpp and device_manager.cpp include the CPU headers unconditionally; deactivation is the header's own responsibility, not every include site. - DeviceCpu::getMinimumKernelExecuted: use common::max_limit (project helper) in place of std::min. - Drop the explanatory comments flagged in review.
f3a2304 to
943f6ba
Compare
|
Rebased onto The conflict came from the vcpkg/deps CMake refactor that landed on
The docker/scripts CPU-axis changes merged cleanly onto the refactored docker layout, and the Verified by cross-compiling the new tree ( Ready for the deeper review when you have time. |
|
Thank you for this first CPU resolver implementation! My understanding: Remarks: The Proposal 1: Proposal 2: Problem: I will let you tell me what you think — I am open to any proposal. |
|
|
||
| uint32_t device::Device::getMinimumKernelExecuted() const | ||
| { | ||
| return common::Config::instance().occupancy.kernelMinimunExecuteNeeded.value_or(100u); |
There was a problem hiding this comment.
common::Config& config{ common::Config::instance() };
uint32_t const executeCount{ .occupancy.kernelMinimunExecuteNeeded };
return executeCount;Does not need .value_or(100u) the default value is 100
| } | ||
| #endif | ||
|
|
||
| #if defined(CPU_ENABLE) |
| #include <utility> | ||
|
|
||
|
|
||
| namespace resolver::cpu_detail |
There was a problem hiding this comment.
wrong define namespace look resolver nvidia or amd
namespace resolver { namespace cpu
| bool resolver::pinThisThreadToCore([[maybe_unused]] uint32_t const coreIndex) | ||
| { | ||
| #if defined(_WIN32) | ||
| DWORD_PTR const mask{ static_cast<DWORD_PTR>(1ull << coreIndex) }; |
There was a problem hiding this comment.
Can you create a custom cast in cast.hpp ?
| { | ||
| // Pin the CALLING thread to a single logical core. Returns true on success. | ||
| // macOS (and any platform without a hard-affinity API): best-effort no-op, returns false. | ||
| bool pinThisThreadToCore(uint32_t coreIndex); |
| resolver::CpuThreadPool::~CpuThreadPool() | ||
| { | ||
| { | ||
| std::lock_guard<std::mutex> const lock{ mutex }; |
There was a problem hiding this comment.
use custom UNIQUE_LOCK from custom.hpp
| // core at startup. parallelFor() splits [0, count) into contiguous per-worker chunks and | ||
| // blocks until all workers finish. Mining hits are astronomically rare, so callers guard | ||
| // any shared write with their own mutex inside the chunk function. | ||
| class CpuThreadPool |
There was a problem hiding this comment.
I understand the use of jthread but in this project I prefer to stay close to the boost framework, and to keep consistency with device_manager can you study the possibility of switching to boost::thread or boost::scoped_thread or other alternatives — I will leave the research to you.
Taking into account my remark that 1 thread could be a devicecpu, CpuThreadPool would therefore no longer be needed.
|
Thanks for the careful read, and for wanting to settle the On the "wait" — you're right, and I'd fix it one layer in rather than with device-per-threadThe batch barrier in But I'd remove it by making the pool workers persistent and work-stealing — each pulls its next nonce range from a lock-free counter and submits shares immediately — rather than by exploding into one device per thread. Mining has no fixed completion point (you search a job's nonce space until the next job arrives, ~1/s), so this turns the sync rate from per-batch into per-job, and that one sync is the atomic job flag we already use. The per-range atomic amortizes to well under 1% (tunable via range size; invisible for slow algos). It also stays correct where we can't pin — macOS gives no hard affinity on Apple Silicon, only QoS hints — because stealing self-balances wherever the OS places a thread. Worth noting the hand-managed pool already beats OpenMP substantially on Windows (1.5 → 11 MH/s). So OpenMP is settled; the open question now is just the work-distribution model on top of the pool, where the steady-state delta between approaches is small and the real differentiator is dataset ownership. Why I'd keep parallelism inside the deviceThe manager's per-device model assumes devices are independent and share no state — true for GPUs, where each card is its own world. CPU cores running a dataset algorithm are not independent: they share a multi-GB dataset. Device-per-thread then forces either N copies of that dataset (RandomX fast-mode is 2 GB × N threads → OOM) or a new shared-dataset-plus-barrier subsystem in the manager — which is exactly the Problem you flagged. The pool already is that subsystem: it's the one piece the device model deliberately lacks, parallel workers that share state. To be fair about it: for dataset algos the pool does carry a barrier — all workers wait for the dataset to be built. But that barrier fires once per epoch/seed change (~3 days for RandomX, ~5 for ethash-family), the build is parallelizable across the same pool workers (seconds, not minutes), and it amortizes to ~0.002%. It also exists in every shared-dataset design — device-per-thread has the same barrier, just harder (cross-device coordination instead of one owner). And if those seconds ever matter, the pool can double-buffer the dataset exactly like the GPU path double-buffers streams: mine the old dataset while building the new, then hot-swap. Why this isn't a BLAKE3-only call — the algo landscapeCPU algorithms split cleanly by one question: self-contained hash, or shared dataset?
Bucket A doesn't push back on either design (the whole yespower family is one parametrized resolver — nice cheap breadth later). Bucket B is where the architecture is decided, and it holds the algos actually worth CPU-mining. The next target after BLAKE3 is RandomX — I've already prototyped a resolver that shares a light-mode cache across per-worker VMs, and that sharing is trivial with one owner and painful across N devices. ProgPoW-on-CPU (your suggestion) isn't competitive to mine (4 GB DAG, bandwidth-bound), but it's a great free validation of the Bucket B path since the GPU side already has the DAG infra. So I'd let Bucket B drive it, which points to: one Proposal 2 (x64 vs arm) — agree on the goal, one layer downNone of the NetKeep the device as the owner of CPU parallelism, swap the batch barrier for a work-stealing pool, set granularity at the cache/NUMA domain (one device for now), and push arch/ISA into the resolver via runtime dispatch. The steady-state perf difference between this and device-per-thread is ~nil; the real difference is dataset ownership, where the pool wins. Happy to sketch the |
- cpu_params.hpp: inline specifier on its own line (nthSetBit, parseHexMask); use castU32/castU64 from cast.hpp instead of static_cast - cpu_affinity.hpp: const-qualify pinThisThreadToCore parameter - tests/cpu_params.cpp: move using-namespace into each TEST body - device_manager.cpp: two blank lines between CPU/CUDA guard blocks
|
Pushed
Still open, intentionally:
|
|
Your solution works perfectly for me! I would change only one thing to keep the same logic throughout the code. I would like Also, |
|
Regarding
namespace toto
{
namespace tata
{
namespace titi
{
// code here
}
}
} |
- thread pool: switch std::jthread/std::mutex/std::condition_variable to
boost::thread/boost::mutex/boost::condition_variable + the UNIQUE_LOCK
macro for consistency with device_manager; join workers explicitly in
the destructor.
- thread pool API: replace parallelFor(count, fn) with setCallback(fn) +
run(count); ResolverCpuBlake3 calls threadPool.run().
- rename pool member `pool` -> `threadPool` (avoid confusion with a
mining pool).
- rename namespace resolver::cpu_detail -> resolver::cpu (folder layer).
- config: occupancy.kernelMinimunExecuteNeeded becomes a plain
uint32_t{100} default instead of optional + value_or(100u) at the read.
- resolver/cpu/cpu.cpp: hoist the threads/blocks ternaries into locals.
- blake3.cpp: drop the anonymous namespace; move the occupancy constants
into the constructor body.
- cast.hpp: add a _WIN32-guarded castDWORDPTR macro; use it in
cpu_affinity.cpp.
CPU + Blake3 unit suites green (CpuParams, CpuThreadPool, ResolverBlake3Cpu).
|
Thanks for the review. The round-2 items are in (the callback One thing to settle before I wire The change. Move the CPU pool from blocking per-batch ( This is intentionally not the GPU model. The GPU already balances in hardware and hides batch cost with double-buffered streams, and its kernels are bounded launches — a persistent "run until the job changes" kernel would risk driver-watchdog kills — so the batch loop is the right fit there. The CPU has neither property: static equal chunks leave fast cores idle on heterogeneous cores, and CPU threads can run free. So the CPU path diverges, behind one virtual method, with the GPU path unchanged. Pros
Cons
Lighter alternative. Dynamic stealing inside the existing Mild preference for the continuous version given RandomX is next, but the lighter one is a reasonable stopping point if you'd rather keep |
|
For the CPU Some explanations: The For GPUs, the use of 2 streams allows launching an Parallel with the CPU: The The This can cause a "blocking" in the unique case where But even in that scenario we can ignore this "latency". |
|
Agreed — the double-buffer is the cleaner route, and it reuses the stream pattern that's already there. It also keeps To confirm I have it right: One orthogonal addition worth considering. The double-buffer removes the wait between batches (the thing you flagged). What it doesn't touch is imbalance inside a batch: with static equal chunks, on heterogeneous cores (P/E, big.LITTLE, SMT, thermal) a fast core finishes its chunk early and idles at the batch tail until the slow one catches up. A cheap fix that composes with the double-buffer: inside a batch, replace the equal split with a small atomic cursor each worker pulls ranges from, so a fast core just takes more ranges. No change to the buffer scheme or the loop — purely how It's optional — on a uniform desktop it's a no-op, and it mainly pays off on laptops/hybrid CPUs and on variable-cost algos (GhostRider, RandomX). Happy to fold it in or leave it; either way I'll start with the double-buffer. |
|
You understood correctly! For For the execution imbalance, I think this is an acceptable tradeoff to keep the code readable and maintainable! We could bench both approaches in a separate executable outside of this scope to compare if we really have any waiting. The benchmark should be run on a machine with no other processes running to be ISO prod on a real mining session. |
|
Thanks — that settles the double-buffer. I've wired On the in-batch imbalance — I hear you on keeping it simple, and I don't want a second dispatch path to maintain. So instead of a static-vs-stealing fork, what about one path with a And I'd settle it with data the way you suggested: an A/B (default grain vs small grain) in a separate benchmark exe, idle machine, ISO-prod — keep the knob only if it actually shows waiting on a hybrid CPU; otherwise it stays a no-op default and we drop it. If you'd rather not touch the equal-split at all for this PR, I'm happy to ship the double-buffer alone and revisit the knob alongside RandomX (where the shared dataset makes it matter more). Your call. |
|
Regarding |
|
Sure — here's the whole idea in pseudocode. The pool keeps one dispatch path; Behaviour is entirely controlled by Default — Each worker does ~one Opt-in — smaller A fast core that finishes its slice loops back and grabs the next free one; a slow core grabs fewer. Coverage is still exactly [0,count), once each — the only difference is a fast core no longer idles at the tail waiting on the slowest. So there's no second mode and no flag branch in the loop: the prod default reproduces the current behaviour (same slices, one per worker), and "work-stealing" is purely "use a smaller grain" — which I'd only enable if the separate benchmark shows real tail-waiting on a hybrid CPU. |
Thread Pool Design — Atomic Cursor & Work-StealingVariables
CpuThreadPoolEach Each worker runs the same loop: lo = cursor.fetch_add(grain, std::memory_order_acquire);
if (lo >= count) break;
callback(lo, lo + grain);The With Each worker does exactly 1 The 3
|
grain |
Behaviour | Problem |
|---|---|---|
>= count |
1 worker does everything, others do nothing | Near-sequential |
= count / workers |
1 slice per worker, current default | None |
| Too small | Work-stealing, fast cores cover slow ones | Contention on the atomic |
⚠️ count too large → log warning
If count is very large, each worker ends up with a huge slice to process. All workers take a long time to finish — significant wait before the next batch. The grid is not optimised and the user must be warned:
[WARNING] Grid size is too large (threads=X blocks=Y count=Z).
Consider reducing --threads or --blocks to improve batch responsiveness.
|
If my previous message is correct and this suits you, we can move on to the implementation. |
|
Your spec matches what I had in mind — agreed, let's implement. Final design as I'll build it:
One small note on the cursor ordering: since I'll also switch |
Mirror the GPU two-stream double-buffer on the CPU path so the device thread reads/submits the previous batch and syncs with stratum while the worker pool computes the next one: - CpuThreadPool gains runAsync()/wait() alongside the blocking run(); a batch is now dispatched through a single atomic cursor that workers pull grain-sized slices from. The default grain (ceil(count / workers)) reproduces the old static equal split exactly; a smaller grain turns the same loop into work-stealing (no-op default, left for a separate benchmark to enable). - ResolverCpuBlake3::executeAsync now waits on the in-flight batch, harvests it, swaps buffers and launches the next one before returning, matching ResolverNvidiaBlake3::executeAsync. Header/target/base are copied per buffer since the worker closure outlives the dispatching call. executeSync stays blocking for tests/debug. - Warn when blocks*threads is large enough to hurt batch responsiveness. Tests: cursor coverage under small grain, runAsync()+wait() coverage, and an executeAsync double-buffer find/submit. CPU suites green (CpuThreadPool, ResolverBlake3CpuTest, CpuParams, ResolverProgpowCpuTest).
|
Pushed Double-buffer (
Grain cursor (
Grid warning —
Tests (CPU suite green on an arm64 macOS host, CPU-only build): cursor coverage under small grain, The in-batch imbalance knob (smaller grain) is wired but off by default per your call — I'll A/B it in the separate benchmark exe on an idle hybrid CPU and only keep it if it shows real tail-waiting. |
…freed threadPool is declared before batch[], so it is destroyed last: when the device exits with an async batch still in flight, a worker finishing that batch during threadPool's join would read already-freed batch[] memory (use-after-free). Wait on the in-flight batch in ~ResolverCpuBlake3 before the buffers are torn down.
|
Pushed
Fix: drain the in-flight batch in resolver::ResolverCpuBlake3::~ResolverCpuBlake3()
{
if (true == inFlight)
{
threadPool.wait();
inFlight = false;
}
}It's a narrow window (shutdown / algorithm switch with a batch outstanding) and on Blake3 it's a benign read of POD buffers rather than a crash, but it's a real UAF so worth closing. Blake3+thread-pool tests stay green. |
|
|
||
| uint32_t device::Device::getMinimumKernelExecuted() const | ||
| { | ||
| return common::Config::instance().occupancy.kernelMinimunExecuteNeeded; |
| #include <device/device_manager.hpp> | ||
| #include <device/mocker.hpp> | ||
| #include <device/nvidia.hpp> | ||
| #include <device/cpu.hpp> |
| uint64_t const mask, | ||
| uint32_t const hardwareConcurrency) | ||
| { | ||
| if (threads.has_value() && 0u < *threads) |
There was a problem hiding this comment.
prefere compare std::nullopt
| { | ||
| // 0-based bit position of the k-th set bit of mask, or 64 if fewer than k+1 bits set. | ||
| inline | ||
| uint32_t nthSetBit(uint64_t const mask, uint32_t k) |
There was a problem hiding this comment.
This is not a function specific to the CPU but rather bit manipulation.
Place it in bitwise.hpp.
| #include <stratum/job_info.hpp> | ||
|
|
||
|
|
||
| namespace |
There was a problem hiding this comment.
Namespace empty is forbidden
| { | ||
| using namespace resolver::cpu; | ||
|
|
||
| for (uint64_t const total : { 0ull, 1ull, 7ull, 100ull, 262144ull }) |
There was a problem hiding this comment.
Create tmp var contains { 0ull, 1ull, 7ull, 100ull, 262144ull }
|
|
||
| for (uint64_t const total : { 0ull, 1ull, 7ull, 100ull, 262144ull }) | ||
| { | ||
| for (uint32_t const n : { 1u, 3u, 8u }) |
There was a problem hiding this comment.
Create var to contain { 1u, 3u, 8u }.
|
|
||
| for (uint64_t const total : { 0ull, 1ull, 7ull, 100ull, 262144ull }) | ||
| { | ||
| for (uint32_t const n : { 1u, 3u, 8u }) |
|
|
||
| // Parse the affinity mask exactly once: the worker-count resolution needs it (popcount | ||
| // when --cpu_threads is unset) and the pool needs it for pinning. | ||
| uint64_t const mask{ config.cpu.affinity.has_value() ? resolver::cpu::parseHexMask(*config.cpu.affinity) : 0ull }; |
|
I have done a preliminary review. |
- move nthSetBit and parseHexMask into algo/bitwise.hpp; rename the latter hexToDecimal and template it over 8/16/32/64-bit widths - nest CpuThreadPool and the affinity helper under resolver::cpu - compare std::nullopt instead of std::optional::has_value() - hoist temporaries (getMinimumKernelExecuted, affinity success flags, test literal lists) and drop the static one-shot warning guard - alphabetize the device/cpu.hpp include; drop a stale cast.hpp comment - replace the anonymous namespace in the CPU blake3 test with file-scope constexpr; relocate the moved helpers' tests to algo/tests/bitwise.cpp
| class CpuThreadPool | ||
| { | ||
| public: | ||
| using ChunkFn = std::function<void(uint64_t lo, uint64_t hi, uint32_t workerIndex)>; |
There was a problem hiding this comment.
look stratum.hpp alias for function must start with callback
Exemples :
using callbackUpdateJob = std::function<void(uint32_t const stratumUUID, StratumJobInfo const&)>;
using callbackShareStatus =
std::function<void(bool const isValid, uint32_t const requestID, uint32_t const stratumUUID)>;There was a problem hiding this comment.
Done in 3a7b8eb — renamed the alias ChunkFn → callbackJob to match the callback* convention in stratum.hpp.
| CpuThreadPool& operator=(CpuThreadPool const&) = delete; | ||
|
|
||
| void setCallback(ChunkFn const& fn); | ||
| // grain == 0 selects the default ceil(count / workers) (one slice per worker). |
There was a problem hiding this comment.
Done in 3a7b8eb — removed.
| private: | ||
| void workerLoop(uint32_t index); | ||
|
|
||
| uint32_t poolSize; |
There was a problem hiding this comment.
Global message, variable should be all time initialize by default.
There was a problem hiding this comment.
Done in 3a7b8eb — poolSize{ 1u } and mask{ 0ull } are now default-initialized.
| // boost::thread does not auto-join on destruction: join each worker explicitly. | ||
| for (boost::thread& worker : workers) | ||
| { | ||
| if (true == worker.joinable()) |
There was a problem hiding this comment.
I kept the explicit stopRequested flag rather than .interrupt(). boost::thread::interrupt() works by throwing boost::thread_interrupted at interruption points, which adds an exception path through the worker hot loop; the pool is also always quiesced before teardown (~ResolverCpuBlake3 drains any in-flight batch first), so the flag + notify_all is enough and keeps shutdown exception-free. Glad to switch to .interrupt() if you prefer it for consistency.
There was a problem hiding this comment.
Done in 6389b21 — switched to worker.interrupt() in the destructor; the worker loop now runs while (false == boost::this_thread::interruption_requested()) and catches the boost::thread_interrupted thrown out of the interruptible cvWork.wait(). The stop flag is gone.
| UNIQUE_LOCK(mutex); | ||
| cvDone.wait( | ||
| lock, | ||
| [&] |
There was a problem hiding this comment.
create lambda auto isRemaining
There was a problem hiding this comment.
Done in 3a7b8eb — hoisted to auto const isRemaining{ [&]() { return 0u == remaining; } };.
|
|
||
| uint32_t poolSize; | ||
| uint64_t mask; | ||
| ChunkFn job{}; |
There was a problem hiding this comment.
Done in 3a7b8eb — member job → cbJob.
| ChunkFn job{}; | ||
| uint64_t total{ 0ull }; | ||
| uint64_t sliceGrain{ 1ull }; | ||
| std::atomic<uint64_t> cursor{ 0ull }; |
There was a problem hiding this comment.
You should use AtomicCounter sources/common/atomic_counter.hpp
You can add new function setMemoryOrder to define if the boost::memory_order.
current.fetch_add(value, boost::memory_order::seq_cst); -> current.fetch_add(value, memoryOrder);
There was a problem hiding this comment.
AtomicCounter currently exposes only void add(), but the cursor needs the previous value returned by the RMW (lo = cursor.fetch_add(grain)), which AtomicCounter cannot provide today. To adopt it I would add a value-returning fetchAdd() plus a setMemoryOrder() to common/atomic_counter.hpp (the only other user is device.hpp). OK to extend the shared type that way?
There was a problem hiding this comment.
Done in 6389b21 — AtomicCounter::add (and sub) now return the RMW result and there is a setMemoryOrder() (plus a store() for resetting). get/sub/store all honor the configurable memoryOrder, which defaults to seq_cst so the device.cpp synchronizer counters are unchanged. The cursor is now common::AtomicCounter<uint64_t> set to relaxed ordering.
| { | ||
| break; | ||
| } | ||
| uint64_t const hi{ std::min<uint64_t>(lo + jobGrain, jobTotal) }; |
There was a problem hiding this comment.
I could not find sources/common/math.hpp or a MIN() anywhere on this branch (or on main). Did you mean a helper landing separately, or common/custom.hpp's max_limit/min_limit (those clamp, not min)? Kept std::min for now — point me at the intended one and I will switch.
There was a problem hiding this comment.
Done in 6389b21 — now uses algo::min() from sources/algo/math.hpp (I had been looking under common/). Thanks for the pointer.
Thread pool (sources/resolver/cpu/thread_pool.*): - rename callback alias ChunkFn -> callbackJob (callback-prefixed, matching the stratum.hpp convention) and member job -> cbJob - default-initialize poolSize and mask - replace both for(;;) worker loops with explicit conditions: the outer loop is driven by an atomic stopRequested, the inner cursor drain by a for-loop condition - hoist the wait predicates into named hasWork / isRemaining lambdas and the pinned-core index into a local - drop a redundant comment Blake3 resolver (sources/resolver/cpu/blake3.*): - install the chunk callback once in the constructor instead of reassigning it on every execute call; it scans into whichever buffer currentIndexStream selects - reuse the base-class double-buffer index (currentIndexStream/swapIndexStream) instead of a private currentIndex - mark the unused worker-index parameter [[maybe_unused]] - name the nonce-width magic numbers (SEARCH/FULL_NONCE_HEX_LENGTH) Verified: CPU-only unit_test (GPU=none CPU=ON BUILD_TESTS=ON) builds and the CPU/thread-pool/blake3 suites pass (incl. the executeAsync double-buffer test).
|
Pushed Verified: CPU-only Three I left as-is pending your steer (details inline):
|
|
You can make the void add(T const value)
{
current.fetch_add(value, boost::memory_order::seq_cst);
}I prefer |
…, interrupt) Following the maintainer's answers to the three open questions: - thread_pool: use algo::min() (sources/algo/math.hpp) instead of std::min. - AtomicCounter (sources/common/atomic_counter.hpp): add/sub now return the RMW result, add setMemoryOrder() and store(); get/sub/store honor the configurable memoryOrder (default seq_cst, so device.cpp's synchronizer counters are unchanged). The CpuThreadPool cursor is now a common::AtomicCounter<uint64_t> set to relaxed ordering. - thread_pool shutdown: replace the stop flag with boost::thread::interrupt(); the worker loop runs while !interruption_requested() and catches boost::thread_interrupted thrown out of the interruptible cvWork.wait(). Verified: CPU-only unit_test (GPU=none CPU=ON BUILD_TESTS=ON) builds and the full suite passes (92/92), incl. the executeAsync double-buffer test and the thread-pool / device synchronizer coverage.
|
Pushed
Verified: CPU-only |
| // (returns 0), while the widest value that fits is parsed exactly. | ||
| TEST(Bitwise, hexToDecimalWidths) | ||
| { | ||
| EXPECT_EQ(static_cast<uint8_t>(0xABu), algo::hexToDecimal<uint8_t>("AB")); |
There was a problem hiding this comment.
Done in 8122a55 — castU8/castU16 from cast.hpp instead of static_cast.
|
|
||
|
|
||
| #if defined(_WIN32) | ||
| //////////////////////////////////////////////////////////////////////////////// |
There was a problem hiding this comment.
Done in 8122a55 — removed the stray separator line.
|
|
||
| uint32_t device::Device::getMinimumKernelExecuted() const | ||
| { | ||
| uint32_t const executeCount{ common::Config::instance().occupancy.kernelMinimunExecuteNeeded }; |
There was a problem hiding this comment.
common::Config const& config{ common::Config::instance() };
return config..occupancy.kernelMinimunExecuteNeeded;There was a problem hiding this comment.
Done in 8122a55 — getMinimumKernelExecuted now takes a config const-ref and returns config.occupancy.kernelMinimunExecuteNeeded directly.
| // batch sizes including count < workers and count not divisible by workers. | ||
| TEST(CpuThreadPool, runCoversAllIndices) | ||
| { | ||
| for (uint32_t const workers : { 1u, 2u, 4u, 8u }) |
There was a problem hiding this comment.
create variable to assign { 1u, 2u, 4u, 8u }
There was a problem hiding this comment.
Done in 8122a55 — hoisted to constexpr std::array<uint32_t, 4> workerCounts{ 1u, 2u, 4u, 8u };.
| { | ||
| for (uint32_t const workers : { 1u, 2u, 4u, 8u }) | ||
| { | ||
| for (uint64_t const count : { 0ull, 1ull, 5ull, 1000ull }) |
There was a problem hiding this comment.
create variable to assign { 0ull, 1ull, 5ull, 1000ull }
There was a problem hiding this comment.
Done in 8122a55 — hoisted to constexpr std::array<uint64_t, 4> counts{ 0ull, 1ull, 5ull, 1000ull };.
| { | ||
| constexpr uint64_t count{ 5000ull }; | ||
| resolver::cpu::CpuThreadPool pool{ 4u, 0ull }; | ||
| std::vector<std::atomic<int>> seen(static_cast<size_t>(count)); |
There was a problem hiding this comment.
Done in 8122a55 — using castSize() from cast.hpp.
| { | ||
| for (uint64_t i{ lo }; i < hi; ++i) | ||
| { | ||
| seen[static_cast<size_t>(i)].fetch_add(1); |
There was a problem hiding this comment.
Done in 8122a55 — using castSize() from cast.hpp.
| { | ||
| constexpr uint64_t count{ 1000ull }; | ||
| resolver::cpu::CpuThreadPool pool{ workers, 0ull }; | ||
| std::vector<std::atomic<int>> seen(static_cast<size_t>(count)); |
There was a problem hiding this comment.
Done in 8122a55 — using castSize() from cast.hpp.
| { | ||
| for (uint64_t i{ lo }; i < hi; ++i) | ||
| { | ||
| seen[static_cast<size_t>(i)].fetch_add(1); |
There was a problem hiding this comment.
Done in 8122a55 — using castSize() from cast.hpp.
| for (uint64_t const count : { 0ull, 1ull, 5ull, 1000ull }) | ||
| { | ||
| resolver::cpu::CpuThreadPool pool{ workers, 0ull }; | ||
| std::vector<std::atomic<int>> seen(static_cast<size_t>(count)); |
There was a problem hiding this comment.
Done in 8122a55 — using castSize() from cast.hpp.
|
Last remarks and it's fine ! |
…test lists) - thread_pool test: hoist the worker-count / count / grain literal lists into constexpr std::array variables (matching the cpu_params test convention) and use castSize() from cast.hpp instead of static_cast<size_t>. - bitwise test: use castU8/castU16 from cast.hpp instead of static_cast. - cast.hpp: drop the stray separator line inside the _WIN32 castDWORDPTR block. - device::getMinimumKernelExecuted: take a config const-ref and return the field directly instead of through a temporary. Verified: CPU-only unit_test (GPU=none CPU=ON BUILD_TESTS=ON) builds and the full suite passes (92/92).
|
Pushed
Verified: CPU-only |
2 participants
Summary
Adds the first CPU mining algorithm by wiring the previously-unused CPU layer to
Blake3/Alephium support. A CPU is enumerated as a mining device alongside GPUs and runs
on the same global Blake3 pool/stratum path. The per-batch nonce scan is parallelized by a
self-owned, persistent, pinned
std::threadpool — not OpenMP — exposing two newcontrols,
--cpu_threadsand--cpu_affinity.Why a custom thread pool instead of OpenMP?
The scan originally used
#pragma omp parallel for. That was replaced because:the pragma silently degraded to a serial, single-core scan — measured at ~1.5 MH/s
on Windows vs ~12 MH/s on Linux (libomp linked) on the same host, an ~8× gap that was
purely serial-vs-multicore.
worker count or pin workers to cores. The pool adds
--cpu_threads/--cpu_affinity.Linux
libomp-devapt-install +libomp.so.5runtime staging, a macOS Homebrew-libompCMake probe, and a "best-effort serial" caveat on Windows.
The
std::jthreadpool fixes all three: real multicore on every platform, tunable, andzero shipped runtime dependency. All OpenMP/libomp plumbing is deleted.
What's included
DEVICE_TYPE::CPU,DeviceCpu(a no-SDK device modeled onDeviceMocker), enumerated byDeviceManagerand dispatched throughsetAlgorithm().ResolverCpuBlake3— reuses the hostblake3::hashRef()(double-BLAKE3Alephium PoW) for the nonce scan, fanned across the pool, with the same winner predicate /
share-submit path as the GPU resolvers.
CpuThreadPool: persistent worker threads pinned at startup;parallelFor()splits abatch into contiguous per-worker chunks and blocks until done. Pure helpers
(
cpu_params.hpp) and the pin wrapper (cpu_affinity) are unit-tested.--cpu_threads(pool size, default = all logical cores) and--cpu_affinity(hex core bitmask, default none). See
documentation/CPU_TUNING.md.GPU=amd|nvidia|both|none×CPU=ON|OFF) across the Linux + Windows-cross Dockerfiles,docker-build.ps1, and CI;all
find_package(OpenMP)/ libomp install / staging removed.CPU(notUNKNOWN); CPU-appropriate occupancy sothe hashrate displays out of the box.
--threads/--blocksstill override.Benchmarks (BLAKE3, live on HeroMiners, one config at a time)
Host: 16-core CPU + AMD RX 9070 XT.
Headline: Windows CPU now matches Linux (~11 vs ~10.7 MH/s) — the OpenMP Windows gap is
closed.
GPU + CPU caveat: running all-core CPU mining next to the GPU starves the GPU's host
feeder thread, collapsing it to ~0.4 GH/s. Reserve cores for the GPU with affinity —
--cpu_threads=8 --cpu_affinity=0xFFrestores the full 1.70 GH/s GPU + ~7 MH/s CPU.Details in
CPU_TUNING.md.Testing
unit-testjob green; newCpuParams(4),CpuThreadPool(2), and existingResolverBlake3Cpu(3) suites pass.(Docker) — jobs received, hashed, shares accepted.
Notes
parallelism is now the in-process
std::threadpool, which ships zero runtime deps.BUILD_CPU=OFF) simply don't compile the CPU resolver, so they'reotherwise unchanged.