Release Process for v23

[PastaPastaPasta]

Release Process

• Update translations, see translation_process.md.
• Update manpages (after rebuilding the binaries), see gen-manpages.py.

Before every minor and major release:

• Review "Needs backport" labels.
• Update DIPs with any changes introduced by this release (see this pull request for an example)
• Update version in configure.ac (don't forget to set CLIENT_VERSION_IS_RELEASE to true)
• Write release notes (see below). To clear the release notes: cp doc/release-notes-empty-template.md doc/release-notes.md
• Update src/chainparams.cpp nMinimumChainWork with information from the getblockchaininfo rpc.
• Update src/chainparams.cpp defaultAssumeValid with information from the getblockhash rpc.
  • The selected value must not be orphaned so it may be useful to set the value two blocks back from the tip.
  • Testnet should be set some tens of thousands back from the tip due to reorgs there.
  • This update should be reviewed with a reindex-chainstate with assumevalid=0 to catch any defect
    that causes rejection of blocks in the past history.
• Ensure all TODOs are evaluated and resolved if needed
• Verify Insight works
• Verify p2pool works (unmaintained; no responsible party)
• Tag version and push (see below)
• Validate that CI passes

Before every major release:

• Update hardcoded seeds, see this pull request for an example.
• Update src/chainparams.cpp m_assumed_blockchain_size and m_assumed_chain_state_size with the current size plus some overhead (see this for information on how to calculate them).
• Update src/chainparams.cpp chainTxData with statistics about the transaction count and rate. Use the output of the getchaintxstats RPC, see
  this pull request for an example. Reviewers can verify the results by running getchaintxstats <window_block_count> <window_last_block_hash> with the window_block_count and window_last_block_hash from your output.

First time / New builders

Install Guix using one of the installation methods detailed in
contrib/guix/INSTALL.md.

Check out the source code in the following directory hierarchy.

cd /path/to/your/toplevel/build
git clone https://github.com/dashpay/guix.sigs.git
git clone https://github.com/dashpay/dash-detached-sigs.git
git clone https://github.com/dashpay/dash.git

Dash Core maintainers/release engineers, suggestion for writing release notes

Write release notes. git shortlog helps a lot, for example:

git shortlog --no-merges v(current version, e.g. 19.3.0)..v(new version, e.g. 20.0.0)

Generate list of authors:

git log --format='- %aN' v(current version, e.g. 19.3.0)..v(new version, e.g. 20.0.0) | sort -fiu

Tag version (or release candidate) in git

git tag -s v(new version, e.g. 20.0.0)

Setup and perform Guix builds

Checkout the Dash Core version you'd like to build:

pushd ./dash
export SIGNER='(your builder key, ie udjinm6, pasta, etc)'
export VERSION='(new version, e.g. 20.0.0)'
git fetch origin "v${VERSION}"
git checkout "v${VERSION}"
popd

Ensure your guix.sigs are up-to-date if you wish to guix-verify your builds
against other guix-attest signatures.

git -C ./guix.sigs pull

Create the macOS SDK tarball (first time, or when SDK version changes)

Note: this step can be skipped if our CI still uses bitcoin's SDK package (see SDK_URL)

Create the macOS SDK tarball, see the macOS build
instructions for
details.

Build and attest to build outputs

Follow the relevant Guix README.md sections:

• Building
• Attesting to build outputs

Note: we ship releases for only some supported HOSTs so consider providing limited HOSTS variable or run ./contrib/containers/guix/scripts/guix-start instead of ./contrib/guix/guix-build when building binaries for quicker builds that exclude the supported but not shipped HOSTs

Verify other builders' signatures to your own (optional)

• Add other builders keys to your gpg keyring, and/or refresh keys
• Verifying build output attestations

Commit your non codesigned signature to guix.sigs

pushd guix.sigs
git add "${VERSION}/${SIGNER}/noncodesigned.SHA256SUMS{,.asc}"
git commit -a
git push  # Assuming you can push to the guix.sigs tree
popd

Codesigning

macOS codesigner only: Create detached macOS signatures (assuming signapple is installed and up to date with master branch)

• Transfer dashcore-osx-unsigned.tar.gz to macOS for signing

• Extract and sign:

  tar xf dashcore-osx-unsigned.tar.gz
  ./detached-sig-create.sh /path/to/codesign.p12 -o runtime

• Enter the keychain password and authorize the signature

• signature-osx.tar.gz will be created

Windows codesigner only: Create detached Windows signatures

• Extract and sign:

  tar xf dashcore-win-unsigned.tar.gz
  ./detached-sig-create.sh -key /path/to/codesign.key

• Enter the passphrase for the key when prompted

• signature-win.tar.gz will be created

Windows and macOS codesigners only: test code signatures

It is advised to test that the code signature attaches properly prior to tagging by performing the guix-codesign step.
However if this is done, once the release has been tagged in the dash-detached-sigs repo, the guix-codesign step must be performed again in order for the guix attestation to be valid when compared against the attestations of non-codesigner builds.

Windows and macOS codesigners only: Commit the detached codesign payloads

pushd ~/dashcore-detached-sigs
# checkout the appropriate branch for this release series
git checkout "v${VERSION}"
rm -rf *
tar xf signature-osx.tar.gz
tar xf signature-win.tar.gz
git add -A
git commit -m "add detached sigs for win/osx for ${VERSION}"
git push
popd

Non-codesigners: wait for Windows and macOS detached signatures

• Once the Windows and macOS builds each have 3 matching signatures, they will be signed with their respective release keys.
• Detached signatures will then be committed to the dash-detached-sigs repository, which can be combined with the unsigned apps to create signed binaries.

Create the codesigned build outputs

• Codesigning build outputs

Verify other builders' signatures to your own (optional)

• Add other builders keys to your gpg keyring, and/or refresh keys
• Verifying build output attestations

Commit your codesigned signature to guix.sigs (for the signed macOS/Windows binaries)

pushd ./guix.sigs
git add "${VERSION}/${SIGNER}"/all.SHA256SUMS{,.asc}
git commit -m "Add attestations by ${SIGNER} for ${VERSION} codesigned"
git push  # Assuming you can push to the guix.sigs tree
popd

After 3 or more people have guix-built and their results match

• Combine the all.SHA256SUMS.asc file from all signers into SHA256SUMS.asc:

  cat "$VERSION"/*/all.SHA256SUMS.asc > SHA256SUMS.asc

• GPG sign each download / binary
• Upload zips and installers, as well as SHA256SUMS.asc from last step, to GitHub as GitHub draft release.

  1. The contents of each ./dash/guix-build-${VERSION}/output/${HOST}/ directory, except for
     *-debug* files.

     Guix will output all of the results into host subdirectories, but the SHA256SUMS
     file does not include these subdirectories. In order for downloads via torrent
     to verify without directory structure modification, all of the uploaded files
     need to be in the same directory as the SHA256SUMS file.

     The *-debug* files generated by the guix build contain debug symbols
     for troubleshooting by developers. It is assumed that anyone that is
     interested in debugging can run guix to generate the files for
     themselves. To avoid end-user confusion about which file to pick, as well
     as save storage space do not upload these to the dash.org server.

     find guix-build-${VERSION}/output/ -maxdepth 2 -type f -not -name "SHA256SUMS.part" -and -not -name "*debug*" -exec scp {} user@dash.org:/var/www/bin/dash-core-${VERSION} \;

  2. The SHA256SUMS file

  3. The SHA256SUMS.asc combined signature file you just created

• Validate SHA256SUMS.asc and all binaries attached to GitHub draft release are correct
• Notarize macOS binaries
• Publish release on GitHub
• Fast-forward master branch on GitHub
• Update the dash.org download links
• Ensure that docker hub images are up to date

Announce the release:

• Release on Dash forum: https://www.dash.org/forum/topic/official-announcements.54/ (necessary so we have a permalink to use on twitter, reddit, etc.)
• Prepare product brief (major versions only)
• Prepare a release announcement tweet
• Follow-up tweets with any important block heights for consensus changes
• Post on Reddit
• Celebrate

After the release:

• Submit patches to BTCPay to ensure they use latest / compatible version see Release Schedule for v18.0.0 #4211 (comment)
• Update Core and User docs (docs.dash.org)
• Test Docker build runs without error in Dashmate
• Add new Release Process items to repo Release Process document
• Merge master branch back into develop so that master could be fast-forwarded on next release again

MacOS Notarization

Prerequisites

Make sure you have the latest Xcode installed on your macOS device. You can download it from the Apple Developer website.
You should have a valid Apple Developer ID under the team you are using which is necessary for the notarization process.
To avoid including your password as cleartext in a notarization script, you can provide a reference to a keychain item. You can add a new keychain item named AC_PASSWORD from the command line using the notarytool utility:

xcrun notarytool store-credentials "AC_PASSWORD" --apple-id "AC_USERNAME" --team-id <WWDRTeamID> --password <secret_2FA_password>

Notarization

Open Terminal, and navigate to the location of the .dmg file.

Then, run the following command to notarize the .dmg file:

xcrun notarytool submit dashcore-{version}-{x86_64, arm64}-apple-darwin.dmg --keychain-profile "AC_PASSWORD" --wait

Replace {version} with the version you are notarizing. This command uploads the .dmg file to Apple's notary service.

The --wait option makes the command wait to return until the notarization process is complete.

If the notarization process is successful, the notary service generates a log file URL. Please save this URL, as it contains valuable information regarding the notarization process.

Notarization Validation

After successfully notarizing the .dmg file, extract Dash-Qt.app from the .dmg.
To verify that the notarization process was successful, run the following command:

spctl -a -vv -t install Dash-Qt.app

Replace Dash-Qt.app with the path to your .app file. This command checks whether your .app file passes Gatekeeper’s
checks. If the app is successfully notarized, the command line will include a line stating source=<Notarized Developer ID>.

Additional information

How to calculate m_assumed_blockchain_size and m_assumed_chain_state_size

Both variables are used as a guideline for how much space the user needs on their drive in total, not just strictly for the blockchain.
Note that all values should be taken from a fully synced node and have an overhead of 5-10% added on top of its base value.

To calculate m_assumed_blockchain_size:

• For mainnet -> Take the size of the data directory, excluding /regtest and /testnet3 directories.
• For testnet -> Take the size of the /testnet3 directory.

To calculate m_assumed_chain_state_size:

• For mainnet -> Take the size of the /chainstate directory.
• For testnet -> Take the size of the /testnet3/chainstate directory.

Notes:

• When taking the size for m_assumed_blockchain_size, there's no need to exclude the /chainstate directory since it's a guideline value and an overhead will be added anyway.
• The expected overhead for growth may change over time, so it may not be the same value as last release; pay attention to that when changing the variables.