Overview of Mainnet Stress Test and Dash Core v0.14.0.3 Release

Last week there was an abnormally high network load on the Dash network consisting of around a million 1 input 1 output transactions with a fee just higher than 1 duff per byte. After contacting all likely community members who could have executed this and finding that none of them were involved, it appears that the artificial load on the Dash network was either a stress test by someone outside of the Dash developer community or was an attempted attack.

Over the course of 2 days, there were two large spikes in transactions:

• Wed, 2019–08–07 0000–0830 (8.5 hrs) — ~650k txs
• Wed/Thu, 2019–08–07/08 2300–0230 (3.5 hrs) — ~300k txs

The Dash mempool spiked to 46 MB around 11:30AM on the 7th of August before being cleared in about 3 hours. Before that, at around 3AM on the 7th of August, the mempool spiked to around 17 MB and was sustained for around 4 hours. After analyzing the Dash blockchain as well as MN logs and their PoSe score, there have been multiple interesting discoveries.

1. Some pools are capped at 1MB blocks

This is likely due to custom implementations. Mining pools will change this naturally once usage approaches 1MB in an effort to capture more fees and be more profitable.

2. Some pools seems not to be emptying the mempool and are producing near empty blocks

We are in active communication with these pools in order to figure out what we can do such that the pools include transactions while not including what appears to be spam (1 input, 1 output, very low fee).

3. Some Masternodes crashed due to not having enough storage space

This was a result of having to store the IS Lock for every transaction. This resulted in nodes running on minimal storage (around 15–20 GB) not having enough space to store all of the IS Locks. Previously, locks were removed after around 7 days. After 0.14.0.3 (PR#3048), IS Locks will be removed as soon as they are confirmed via a ChainLock. This should reduce the size of the `llmq` directory significantly over time.

4. Some Masternodes banned other Masternodes under high load when on the brink of a new quorum becoming active

In order to fix this, we need to detect this situation and then re-verify failed sigs with the previous active set. This is being implemented by PR#3052 and will also be released in 0.14.0.3. This issue resulted in some blocks not receiving a ChainLock and some transactions not receiving IS Locks.

5. Some user transactions did not get included in a block rapidly since they used the default fee

The fee selection algorithm and UI in the Dash Core wallet is being updated in 0.14.1 thanks to the backporting of Bitcoin 0.15. The fee selection algorithms in the mobile wallets are being reviewed as well.

We are exploring ways that we can try to minimize the load that spam and an attacker can have on the network. There are many potential ideas, however, each of these ideas have their benefits and problems and we are still investigating what we can do in order to ensure that users can send funds at a sub cent fee while also preventing spam and attacks on the Dash network.

In response to the discoveries outlined above, 0.14.0.3 is being released. The release includes various bug fixes and improvements. The upgrade is strongly recommended for all Masternodes and is also recommended for all users, exchanges, partners and full node operators.

Notable Changes in Dash Core v0.14.0.3

• Database space usage improvements
• As described above in point 3
• DKG and LLMQ signing failures fixed
• As described above in point 4
• Signed binaries for Windows
• MacOS: disable AppNap during sync and mixing
• New RPC command: quorum memberof <proTxHash>
• More information about number of InstantSend locks

Please see the full changelog for more details.

Author: Pasta
Original link: https://blog.dash.org/overview-of-mainnet-stress-test-and-dash-core-v0-14-0-3-release-db6b978da42c