Solana outage caused by error in processing durable transaction nonces

On Wednesday, June 1, the Solana network failed to produce new blocks for more than four hours due to a failure in processing ‘durable transaction nonces’, which led to a disruption of consensus. The developers disabled this type of operation and asked validators to restart the blockchain.

Validator operators successfully completed a cluster restart of Mainnet Beta at 9:00 PM UTC, following a roughly 4 and a half hour outage after the network failed to reach consensus. Network operators an dapps will continue to restore client services over the next several hours.

— Solana Status (@SolanaStatus) June 1, 2022

Durable transaction nonces (durable transaction nonces) are popular mainly among cryptocurrency exchanges. The feature is designed for token holders who use complex automatic signing setups, which means they cannot always prepare transactions for registration on the blockchain promptly.

For example, a custodian that signs a transaction from two isolated devices may not finish the work within a single block. A normal operation in the Solana network in such a case would not pass, however the aforementioned feature allows bypassing this limitation and signing offline in advance.

One of Solana’s validators — Stakewiz — noted that the issue with this type of transactions has long been known. In his words, on May 28 a similar error had already been observed, but it did not affect the cluster.

A durable nonce is a way in which a transaction can be signed offline ahead of time, without requiring a recent block hash (which expires after two minutes). Usage has recently increased, particularly by exchanges, possibly due to their cold storage setups.

— Laine | stakewiz.com (@laine_sa_) June 1, 2022

According to Solana, the block time was 4 hours 10 minutes. At the time of writing the blockchain is operating stably. Meanwhile, observers such as Solscan and Solana Beach had problems displaying block issuance times and transaction times — values are shown with a six-hour delay.

This was caused by a desynchronisation of the blockchain with real time, which occurred last week. However, at the time it was stated that the network’s internal clocks lagged by only 30 minutes.

Solana founder Anatoly Yakovenko emphasised that the development team already has a solution — it will be implemented after testing.

if average slot time is < 720ms, then clock is going to speed up. there is a fix that should get activated after some testing that will speed up the clock as long as < 1s

— T◎lyxNFT, 🇺🇸 (@aeyakovenko) June 1, 2022

In the night of 1 May 2022, Solana suspended operations due to a high volume of incoming transactions and heavy traffic.

Follow ForkLog Bitcoin news on our Telegram — cryptocurrency news, prices and analytics.