add logs

expected Bohr hard fork date:
- Testnet: 2024-08-20 01:23:16 AM UTC
- Mainnet: it is not determined yet, target Later Sep 2024

.nancy-ignore

@@ -1 +1,2 @@
 CVE-2024-34478 # "CWE-754: Improper Check for Unusual or Exceptional Conditions." This vulnerability is BTC only, BSC does not have the issue.
+CVE-2024-6104 # "CWE-532: Information Exposure Through Log Files" This is caused by the vulnerabilities go-retryablehttp@v0.7.4, it is only used in cmd devp2p, impact is limited. will upgrade to v0.7.7 later

CHANGELOG.md

@@ -1,4 +1,105 @@
 # Changelog
+## v1.4.13
+
+### BUGFIX
+* [\#2602](https://github.com/bnb-chain/bsc/pull/2602) fix: prune-state when specify --triesInMemory 32
+* [\#2579](https://github.com/bnb-chain/bsc/pull/00025790) fix: only take non-mempool tx to calculate bid price
+
+### FEATURE
+* [\#2634](https://github.com/bnb-chain/bsc/pull/2634) config: setup Testnet Bohr hardfork date
+* [\#2482](https://github.com/bnb-chain/bsc/pull/2482) BEP-341: Validators can produce consecutive blocks
+* [\#2502](https://github.com/bnb-chain/bsc/pull/2502) BEP-402: Complete Missing Fields in Block Header to Generate Signature
+* [\#2558](https://github.com/bnb-chain/bsc/pull/2558) BEP-404: Clear Miner History when Switching Validators Set
+* [\#2605](https://github.com/bnb-chain/bsc/pull/2605) feat: add bohr upgrade contracts bytecode
+* [\#2614](https://github.com/bnb-chain/bsc/pull/2614) fix: update stakehub bytecode after zero address agent issue fixed
+* [\#2608](https://github.com/bnb-chain/bsc/pull/2608) consensus/parlia: modify mining time for last block in one turn
+* [\#2618](https://github.com/bnb-chain/bsc/pull/2618) consensus/parlia: exclude inturn validator when calculate backoffTime
+* [\#2621](https://github.com/bnb-chain/bsc/pull/2621) core: not record zero hash beacon block root with Parlia engine
+
+### IMPROVEMENT
+* [\#2589](https://github.com/bnb-chain/bsc/pull/2589) core/vote: vote before committing state and writing block
+* [\#2596](https://github.com/bnb-chain/bsc/pull/2596) core: improve the network stability when double sign happens
+* [\#2600](https://github.com/bnb-chain/bsc/pull/2600) core: cache block after wroten into db
+* [\#2629](https://github.com/bnb-chain/bsc/pull/2629) utils: add GetTopAddr to analyse large traffic
+* [\#2591](https://github.com/bnb-chain/bsc/pull/2591) consensus/parlia: add GetJustifiedNumber and GetFinalizedNumber
+* [\#2611](https://github.com/bnb-chain/bsc/pull/2611) cmd/utils: add new flag OverridePassedForkTime
+* [\#2603](https://github.com/bnb-chain/bsc/pull/2603) faucet: rate limit initial implementation
+* [\#2622](https://github.com/bnb-chain/bsc/pull/2622) tests: fix evm-test CI
+* [\#2628](https://github.com/bnb-chain/bsc/pull/2628) Makefile: use docker compose v2 instead of v1
+
+## v1.4.12
+
+### BUGFIX
+* [\#2557](https://github.com/bnb-chain/bsc/pull/2557) fix: fix state inspect error after pruned state
+* [\#2562](https://github.com/bnb-chain/bsc/pull/2562) fix: delete unexpected block
+* [\#2566](https://github.com/bnb-chain/bsc/pull/2566) core: avoid to cache block before wroten into db
+* [\#2567](https://github.com/bnb-chain/bsc/pull/2567) fix: fix statedb copy
+* [\#2574](https://github.com/bnb-chain/bsc/pull/2574) core: adapt highestVerifiedHeader to FastFinality
+* [\#2542](https://github.com/bnb-chain/bsc/pull/2542) fix: pruneancient freeze from the previous position when the first time
+* [\#2564](https://github.com/bnb-chain/bsc/pull/2564) fix: the bug of blobsidecars and downloader with multi-database
+* [\#2582](https://github.com/bnb-chain/bsc/pull/2582) fix: remove delete and dangling side chains in prunefreezer
+
+### FEATURE
+* [\#2513](https://github.com/bnb-chain/bsc/pull/2513) cmd/jsutils: add a tool to get performance between a range of blocks
+* [\#2569](https://github.com/bnb-chain/bsc/pull/2569) cmd/jsutils: add a tool to get slash count
+* [\#2583](https://github.com/bnb-chain/bsc/pull/2583) cmd/jsutill: add log about validator name
+
+### IMPROVEMENT
+* [\#2546](https://github.com/bnb-chain/bsc/pull/2546) go.mod: update missing dependency
+* [\#2559](https://github.com/bnb-chain/bsc/pull/2559) nancy: ignore go-retryablehttp@v0.7.4 in .nancy-ignore
+* [\#2556](https://github.com/bnb-chain/bsc/pull/2556) chore: update greenfield cometbft version
+* [\#2561](https://github.com/bnb-chain/bsc/pull/2561) tests: fix unstable test
+* [\#2572](https://github.com/bnb-chain/bsc/pull/2572) core: clearup testflag for Cancun and Haber
+* [\#2573](https://github.com/bnb-chain/bsc/pull/2573) cmd/utils: support use NetworkId to distinguish chapel when do syncing
+* [\#2538](https://github.com/bnb-chain/bsc/pull/2538) feat: enhance bid comparison and reply bidding results && detail logs
+* [\#2568](https://github.com/bnb-chain/bsc/pull/2568) core/vote: not vote if too late for next in turn validator
+* [\#2576](https://github.com/bnb-chain/bsc/pull/2576) miner/worker: broadcast block immediately once sealed
+* [\#2580](https://github.com/bnb-chain/bsc/pull/2580) freezer: Opt freezer env checking
+
+## v1.4.11
+
+### BUGFIX
+* [\#2534](https://github.com/bnb-chain/bsc/pull/2534) fix: nil pointer when clear simulating bid
+* [\#2535](https://github.com/bnb-chain/bsc/pull/2535) upgrade: add HaberFix hardfork
+
+## v1.4.10
+### FEATURE
+NA
+
+### IMPROVEMENT
+* [\#2512](https://github.com/bnb-chain/bsc/pull/2512) feat: add mev helper params and func
+* [\#2508](https://github.com/bnb-chain/bsc/pull/2508) perf: speedup pbss trienode read
+* [\#2509](https://github.com/bnb-chain/bsc/pull/2509) perf: optimize chain commit performance for multi-database
+* [\#2451](https://github.com/bnb-chain/bsc/pull/2451) core/forkchoice: improve stability when inturn block not generate
+
+### BUGFIX
+* [\#2518](https://github.com/bnb-chain/bsc/pull/2518) fix: remove zero gasprice check for BSC
+* [\#2519](https://github.com/bnb-chain/bsc/pull/2519) UT: random failure of TestSnapSyncWithBlobs
+* [\#2515](https://github.com/bnb-chain/bsc/pull/2515) fix getBlobSidecars by ethclient
+* [\#2525](https://github.com/bnb-chain/bsc/pull/2525) fix: ensure empty withdrawals after cancun before broadcast
+
+## v1.4.9
+### FEATURE
+* [\#2463](https://github.com/bnb-chain/bsc/pull/2463)  utils: add check_blobtx.js
+* [\#2470](https://github.com/bnb-chain/bsc/pull/2470)  jsutils: faucet successful requests within blocks
+* [\#2467](https://github.com/bnb-chain/bsc/pull/2467)  internal/ethapi: add optional parameter for blobSidecars
+
+### IMPROVEMENT
+* [\#2462](https://github.com/bnb-chain/bsc/pull/2462)  cmd/utils: add a flag to change breathe block interval for testing
+* [\#2497](https://github.com/bnb-chain/bsc/pull/2497)  params/config: add Bohr hardfork
+* [\#2479](https://github.com/bnb-chain/bsc/pull/2479)  dev: ensure consistency in BPS bundle result
+
+### BUGFIX
+* [\#2461](https://github.com/bnb-chain/bsc/pull/2461)  eth/handler: check lists in body before broadcast blocks
+* [\#2455](https://github.com/bnb-chain/bsc/pull/2455)  cmd: fix memory leak when big dataset
+* [\#2466](https://github.com/bnb-chain/bsc/pull/2466)  sync: fix some sync issues caused by prune-block.
+* [\#2475](https://github.com/bnb-chain/bsc/pull/2475)  fix: move mev op to MinerAPI & add command to console
+* [\#2473](https://github.com/bnb-chain/bsc/pull/2473)  fix: limit the gas price of the mev bid
+* [\#2484](https://github.com/bnb-chain/bsc/pull/2484)  fix: fix inspect database error
+* [\#2481](https://github.com/bnb-chain/bsc/pull/2481)  fix: keep 9W blocks in ancient db when prune block
+* [\#2495](https://github.com/bnb-chain/bsc/pull/2495)  fix: add an empty freeze db
+* [\#2507](https://github.com/bnb-chain/bsc/pull/2507)  fix: waiting for the last simulation before pick best bid
+
 ## v1.4.8
 ### FEATURE
 * [\#2483](https://github.com/bnb-chain/bsc/pull/2483) core/vm: add secp256r1 into PrecompiledContractsHaber

Makefile

@@ -29,11 +29,11 @@ truffle-test:
 	docker build . -f ./docker/Dockerfile --target bsc-genesis -t bsc-genesis
 	docker build . -f ./docker/Dockerfile --target bsc -t bsc
 	docker build . -f ./docker/Dockerfile.truffle -t truffle-test
-	docker-compose -f ./tests/truffle/docker-compose.yml up genesis
-	docker-compose -f ./tests/truffle/docker-compose.yml up -d bsc-rpc bsc-validator1
+	docker compose -f ./tests/truffle/docker-compose.yml up genesis
+	docker compose -f ./tests/truffle/docker-compose.yml up -d bsc-rpc bsc-validator1
 	sleep 30
-	docker-compose -f ./tests/truffle/docker-compose.yml up --exit-code-from truffle-test truffle-test
-	docker-compose -f ./tests/truffle/docker-compose.yml down
+	docker compose -f ./tests/truffle/docker-compose.yml up --exit-code-from truffle-test truffle-test
+	docker compose -f ./tests/truffle/docker-compose.yml down
 
 #? lint: Run certain pre-selected linters
 lint: ## Run linters.

README.md

@@ -149,8 +149,6 @@ unzip testnet.zip
 #### 3. Download snapshot
 Download latest chaindata snapshot from [here](https://github.com/bnb-chain/bsc-snapshots). Follow the guide to structure your files.
 
-Note: If you encounter difficulties downloading the chaindata snapshot and prefer to synchronize from the genesis block on the Chapel testnet, remember to include the additional flag `--chapel` when initially launching Geth.
-
 #### 4. Start a full node
 ```shell
 ./geth --config ./config.toml --datadir ./node  --cache 8000 --rpc.allow-unprotected-txs --history.transactions 0

cmd/evm/internal/t8ntool/execution.go

@@ -227,7 +227,7 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
 			return nil, nil, nil, err
 		}
 		vmConfig.Tracer = tracer
-		statedb.SetTxContext(tx.Hash(), txIndex, 0)
+		statedb.SetTxContext(tx.Hash(), txIndex)
 
 		var (
 			txContext = core.NewEVMTxContext(msg)

cmd/extradump/extradump_test.go

@@ -43,7 +43,42 @@ func TestExtraParse(t *testing.T) {
 		}
 	}
 
-	// case 3, |---Extra Vanity---|---Empty---|---Vote Attestation---|---Extra Seal---|
+	// case 3, |---Extra Vanity---|---Validators Number and Validators Bytes---|---Turn Length---|---Empty---|---Extra Seal---|
+	{
+		extraData := "0xd983010209846765746889676f312e31392e3131856c696e75780000a6bf97c1152465176c461afb316ebc773c61faee85a6515daa8a923564c6ffd37fb2fe9f118ef88092e8762c7addb526ab7eb1e772baef85181f892c731be0c1891a50e6b06262c816295e26495cef6f69dfa69911d9d8e4f3bbadb89b977cf58294f7239d515e15b24cfeb82494056cf691eaf729b165f32c9757c429dba5051155903067e56ebe3698678e912d4c407bbe49438ed859fe965b140dcf1aab71a993c1f7f6929d1fe2a17b4e14614ef9fc5bdc713d6631d675403fbeefac55611bf612700b1b65f4744861b80b0f7d6ab03f349bbafec1551819b8be1efea2fc46ca749aa184248a459464eec1a21e7fc7b71a053d9644e9bb8da4853b8f872cd7c1d6b324bf1922829830646ceadfb658d3de009a61dd481a114a2e761c554b641742c973867899d300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000069c77a677c40c7fbea129d4b171a39b7a8ddabfab2317f59d86abfaf690850223d90e9e7593d91a29331dfc2f84d5adecc75fc39ecab4632c1b4400a3dd1e1298835bcca70f657164e5b75689b64b7fd1fa275f334f28e1896a26afa1295da81418593bd12814463d9f6e45c36a0e47eb4cd3e5b6af29c41e2a3a5636430155a466e216585af3ba772b61c6014342d914470ec7ac2975be345796c2b81db0422a5fd08e40db1fc2368d2245e4b18b1d0b85c921aaaafd2e341760e29fc613edd39f71254614e2055c3287a517ae2f5b9e386cd1b50a4550696d957cb4900f03ab84f83ff2df44193496793b847f64e9d6db1b3953682bb95edd096eb1e69bbd357c200992ca78050d0cbe180cfaa018e8b6c8fd93d6f4cea42bbb345dbc6f0dfdb5bec73a8a257074e82b881cfa06ef3eb4efeca060c2531359abd0eab8af1e3edfa2025fca464ac9c3fd123f6c24a0d78869485a6f79b60359f141df90a0c745125b131caaffd12000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000b218c5d6af1f979ac42bc68d98a5a0d796c6ab01000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000b4dd66d7c2c7e57f628210187192fb89d4b99dd4000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000be807dddb074639cd9fa61b47676c064fc50d62cb1f2c71577def3144fabeb75a8a1c8cb5b51d1d1b4a05eec67988b8685008baa17459ec425dbaebc852f496dc92196cdcc8e6d00c17eb431350c6c50d8b8f05176b90b11b3a3d4feb825ae9702711566df5dbf38e82add4dd1b573b95d2466fa6501ccb81e9d26a352b96150ccbf7b697fd0a419d1d6bf74282782b0b3eb1413c901d6ecf02e8e28000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e2d3a739effcd3a99387d015e260eefac72ebea1956c470ddff48cb49300200b5f83497f3a3ccb3aeb83c5edd9818569038e61d197184f4aa6939ea5e9911e3e98ac6d21e9ae3261a475a27bb1028f140bc2a7c843318afd000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000ea0a6e3c511bbd10f4519ece37dc24887e11b55db2d4c6283c44a1c7bd503aaba7666e9f0c830e0ff016c1c750a5e48757a713d0836b1cabfd5c281b1de3b77d1c192183ee226379db83cffc681495730c11fdde79ba4c0c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000ef0274e31810c9df02f98fafde0f841f4e66a1cd00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004e99f701bb14cb7dfb68b90bd3e6d1ca656964630de71beffc7f33f7f08ec99d336ec51ad9fad0ac84ae77ca2e8ad9512acc56e0d7c93f3c2ce7de1b69149a5a400"
+		extra, err := parseExtra(extraData)
+		assert.NoError(t, err)
+		{
+			var have = extra.ValidatorSize
+			var want = uint8(21)
+			if have != want {
+				t.Fatalf("extra.ValidatorSize mismatch, have %d, want %d", have, want)
+			}
+		}
+		{
+			var have = common.Bytes2Hex(extra.Validators[14].Address[:])
+			var want = "cc8e6d00c17eb431350c6c50d8b8f05176b90b11"
+			if have != want {
+				t.Fatalf("extra.Validators[14].Address mismatch, have %s, want %s", have, want)
+			}
+		}
+		{
+			var have = common.Bytes2Hex(extra.Validators[18].BLSPublicKey[:])
+			var want = "b2d4c6283c44a1c7bd503aaba7666e9f0c830e0ff016c1c750a5e48757a713d0836b1cabfd5c281b1de3b77d1c192183"
+			if have != want {
+				t.Fatalf("extra.Validators[18].BLSPublicKey mismatch, have %s, want %s", have, want)
+			}
+		}
+		{
+			var have = extra.TurnLength
+			var want = uint8(4)
+			if *have != want {
+				t.Fatalf("extra.TurnLength mismatch, have %d, want %d", *have, want)
+			}
+		}
+	}
+
+	// case 4, |---Extra Vanity---|---Empty---|---Vote Attestation---|---Extra Seal---|
 	{
 		extraData := "0xd883010205846765746888676f312e32302e35856c696e75780000002995c52af8b5830563efb86089cf168dcf4c5d3cb057926628ad1bf0f03ea67eef1458485578a4f8489afa8a853ecc7af45e2d145c21b70641c4b29f0febd2dd2c61fa1ba174be3fd47f1f5fa2ab9b5c318563d8b70ca58d0d51e79ee32b2fb721649e2cb9d36538361fba11f84c8401d14bb7a0fa67ddb3ba654d6006bf788710032247aa4d1be0707273e696b422b3ff72e9798401d14bbaa01225f505f5a0e1aefadcd2913b7aac9009fe4fb3d1bf57399e0b9dce5947f94280fe6d3647276c4127f437af59eb7c7985b2ae1ebe432619860695cb6106b80cc66c735bc1709afd11f233a2c97409d38ebaf7178aa53e895aea2fe0a229f71ec601"
 		extra, err := parseExtra(extraData)
@@ -64,9 +99,9 @@ func TestExtraParse(t *testing.T) {
 		}
 	}
 
-	// case 4, |---Extra Vanity---|---Validators Number and Validators Bytes---|---Vote Attestation---|---Extra Seal---|
+	// case 5, |---Extra Vanity---|---Validators Number and Validators Bytes---|---Vote Attestation---|---Extra Seal---|
 	{
-		extraData := "0xd883010209846765746888676f312e31392e38856c696e7578000000dc55905c071284214b9b9c85549ab3d2b972df0deef66ac2c98e82934ca974fdcd97f3309de967d3c9c43fa711a8d673af5d75465844bf8969c8d1948d903748ac7b8b1720fa64e50c35552c16704d214347f29fa77f77da6d75d7c752b742ad4855bae330426b823e742da31f816cc83bc16d69a9134be0cfb4a1d17ec34f1b5b32d5c20440b8536b1e88f0f247788386d0ed6c748e03a53160b4b30ed3748cc5000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000980a75ecd1309ea12fa2ed87a8744fbfc9b863d589037a9ace3b590165ea1c0c5ac72bf600b7c88c1e435f41932c1132aae1bfa0bb68e46b96ccb12c3415e4d82af717d8a2959d3f95eae5dc7d70144ce1b73b403b7eb6e0b973c2d38487e58fd6e145491b110080fb14ac915a0411fc78f19e09a399ddee0d20c63a75d8f930f1694544ad2dc01bb71b214cb885500844365e95cd9942c7276e7fd8a2750ec6dded3dcdc2f351782310b0eadc077db59abca0f0cd26776e2e7acb9f3bce40b1fa5221fd1561226c6263cc5ff474cf03cceff28abc65c9cbae594f725c80e12d96c9b86c3400e529bfe184056e257c07940bb664636f689e8d2027c834681f8f878b73445261034e946bb2d901b4b878f8b27bb8608c11016739b3f8a19e54ab8c7abacd936cfeba200f3645a98b65adb0dd3692b69ce0b3ae10e7176b9a4b0d83f04065b1042b4bcb646a34b75c550f92fc34b8b2b1db0fa0d3172db23ba92727c80bcd306320d0ff411bf858525fde13bc8e0370f84c8401e9c2e6a0820dc11d63176a0eb1b828bc5376867b275579112b7013358da40317e7bab6e98401e9c2e7a00edc71ce80105a3220a87bea2792fa340d66c59002f02b0a09349ed1ed284070808b972fac2b9077a4dcb6fc37093799a652858016c99142b227500c844fa97ec22e3f9d3b1e982f14bcd999a7453e89ce5ef5c55f1c7f8f74ba904186cd67828200"
+		extraData := "0xd883010209846765746888676f312e31392e38856c696e7578000000dc55905c071284214b9b9c85549ab3d2b972df0deef66ac2c98e82934ca974fdcd97f3309de967d3c9c43fa711a8d673af5d75465844bf8969c8d1948d903748ac7b8b1720fa64e50c35552c16704d214347f29fa77f77da6d75d7c752b742ad4855bae330426b823e742da31f816cc83bc16d69a9134be0cfb4a1d17ec34f1b5b32d5c20440b8536b1e88f0f247788386d0ed6c748e03a53160b4b30ed3748cc5000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000980a75ecd1309ea12fa2ed87a8744fbfc9b863d589037a9ace3b590165ea1c0c5ac72bf600b7c88c1e435f41932c1132aae1bfa0bb68e46b96ccb12c3415e4d82af717d8a2959d3f95eae5dc7d70144ce1b73b403b7eb6e0b973c2d38487e58fd6e145491b110080fb14ac915a0411fc78f19e09a399ddee0d20c63a75d8f930f1694544ad2dc01bb71b214cb885500844365e95cd9942c7276e7fd8a2750ec6dded3dcdc2f351782310b0eadc077db59abca0f0cd26776e2e7acb9f3bce40b1fa5221fd1561226c6263cc5ff474cf03cceff28abc65c9cbae594f725c80e12d96c9b86c3400e529bfe184056e257c07940bb664636f689e8d2027c834681f8f878b73445261034e946bb2d901b4b878f8b27bb8608c11016739b3f8a19e54ab8c7abacd936cfeba200f3645a98b65adb0dd3692b69ce0b3ae10e7176b9a4b0d83f04065b1042b4bcb646a34b75c550f92fc34b8b2b1db0fa0d3172db23ba92727c80bcd306320d0ff411bf858525fde13bc8e0370f84c8401e9c2e6a0820dc11d63176a0eb1b828bc5376867b275579112b7013358da40317e7bab6e98401e9c2e7a00edc71ce80105a3220a87bea2792fa340d66c59002f02b0a09349ed1ed28407080048b972fac2b9077a4dcb6fc37093799a652858016c99142b227500c844fa97ec22e3f9d3b1e982f14bcd999a7453e89ce5ef5c55f1c7f8f74ba904186cd67828200"
 		extra, err := parseExtra(extraData)
 		assert.NoError(t, err)
 		{
@@ -105,4 +140,53 @@ func TestExtraParse(t *testing.T) {
 			}
 		}
 	}
+
+	// case 6, |---Extra Vanity---|---Validators Number and Validators Bytes---|---Turn Length---|---Vote Attestation---|---Extra Seal---|
+	{
+		extraData := "0xd883010209846765746888676f312e31392e38856c696e7578000000dc55905c071284214b9b9c85549ab3d2b972df0deef66ac2c98e82934ca974fdcd97f3309de967d3c9c43fa711a8d673af5d75465844bf8969c8d1948d903748ac7b8b1720fa64e50c35552c16704d214347f29fa77f77da6d75d7c752b742ad4855bae330426b823e742da31f816cc83bc16d69a9134be0cfb4a1d17ec34f1b5b32d5c20440b8536b1e88f0f247788386d0ed6c748e03a53160b4b30ed3748cc5000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000980a75ecd1309ea12fa2ed87a8744fbfc9b863d589037a9ace3b590165ea1c0c5ac72bf600b7c88c1e435f41932c1132aae1bfa0bb68e46b96ccb12c3415e4d82af717d8a2959d3f95eae5dc7d70144ce1b73b403b7eb6e0b973c2d38487e58fd6e145491b110080fb14ac915a0411fc78f19e09a399ddee0d20c63a75d8f930f1694544ad2dc01bb71b214cb885500844365e95cd9942c7276e7fd8a2750ec6dded3dcdc2f351782310b0eadc077db59abca0f0cd26776e2e7acb9f3bce40b1fa5221fd1561226c6263cc5ff474cf03cceff28abc65c9cbae594f725c80e12d96c9b86c3400e529bfe184056e257c07940bb664636f689e8d2027c834681f8f878b73445261034e946bb2d901b4b87804f8b27bb8608c11016739b3f8a19e54ab8c7abacd936cfeba200f3645a98b65adb0dd3692b69ce0b3ae10e7176b9a4b0d83f04065b1042b4bcb646a34b75c550f92fc34b8b2b1db0fa0d3172db23ba92727c80bcd306320d0ff411bf858525fde13bc8e0370f84c8401e9c2e6a0820dc11d63176a0eb1b828bc5376867b275579112b7013358da40317e7bab6e98401e9c2e7a00edc71ce80105a3220a87bea2792fa340d66c59002f02b0a09349ed1ed28407080048b972fac2b9077a4dcb6fc37093799a652858016c99142b227500c844fa97ec22e3f9d3b1e982f14bcd999a7453e89ce5ef5c55f1c7f8f74ba904186cd67828200"
+		extra, err := parseExtra(extraData)
+		assert.NoError(t, err)
+		{
+			var have = common.Bytes2Hex(extra.Validators[0].Address[:])
+			var want = "1284214b9b9c85549ab3d2b972df0deef66ac2c9"
+			if have != want {
+				t.Fatalf("extra.Validators[0].Address mismatch, have %s, want %s", have, want)
+			}
+		}
+		{
+			var have = common.Bytes2Hex(extra.Validators[0].BLSPublicKey[:])
+			var want = "8e82934ca974fdcd97f3309de967d3c9c43fa711a8d673af5d75465844bf8969c8d1948d903748ac7b8b1720fa64e50c"
+			if have != want {
+				t.Fatalf("extra.Validators[0].BLSPublicKey mismatch, have %s, want %s", have, want)
+			}
+		}
+		{
+			var have = extra.Validators[0].VoteIncluded
+			var want = true
+			if have != want {
+				t.Fatalf("extra.Validators[0].VoteIncluded mismatch, have %t, want %t", have, want)
+			}
+		}
+		{
+			var have = common.Bytes2Hex(extra.Data.TargetHash[:])
+			var want = "0edc71ce80105a3220a87bea2792fa340d66c59002f02b0a09349ed1ed284070"
+			if have != want {
+				t.Fatalf("extra.Data.TargetHash mismatch, have %s, want %s", have, want)
+			}
+		}
+		{
+			var have = extra.Data.TargetNumber
+			var want = uint64(32096999)
+			if have != want {
+				t.Fatalf("extra.Data.TargetNumber mismatch, have %d, want %d", have, want)
+			}
+		}
+		{
+			var have = extra.TurnLength
+			var want = uint8(4)
+			if *have != want {
+				t.Fatalf("extra.TurnLength mismatch, have %d, want %d", *have, want)
+			}
+		}
+	}
 }

cmd/extradump/main.go

@@ -24,7 +24,7 @@ const (
 	BLSPublicKeyLength = 48
 
 	// follow order in extra field
-	// |---Extra Vanity---|---Validators Number and Validators Bytes (or Empty)---|---Vote Attestation (or Empty)---|---Extra Seal---|
+	// |---Extra Vanity---|---Validators Number and Validators Bytes (or Empty)---|---Turn Length (or Empty)---|---Vote Attestation (or Empty)---|---Extra Seal---|
 	extraVanityLength    = 32 // Fixed number of extra-data prefix bytes reserved for signer vanity
 	validatorNumberSize  = 1  // Fixed number of extra prefix bytes reserved for validator number after Luban
 	validatorBytesLength = common.AddressLength + types.BLSPublicKeyLength
@@ -35,6 +35,7 @@ type Extra struct {
 	ExtraVanity   string
 	ValidatorSize uint8
 	Validators    validatorsAscending
+	TurnLength    *uint8
 	*types.VoteAttestation
 	ExtraSeal []byte
 }
@@ -113,6 +114,15 @@ func parseExtra(hexData string) (*Extra, error) {
 			sort.Sort(extra.Validators)
 			data = data[validatorBytesTotalLength-validatorNumberSize:]
 			dataLength = len(data)
+
+			// parse TurnLength
+			if dataLength > 0 {
+				if data[0] != '\xf8' {
+					extra.TurnLength = &data[0]
+					data = data[1:]
+					dataLength = len(data)
+				}
+			}
 		}
 
 		// parse Vote Attestation
@@ -148,6 +158,10 @@ func prettyExtra(extra Extra) {
 		}
 	}
 
+	if extra.TurnLength != nil {
+		fmt.Printf("TurnLength	:	%d\n", *extra.TurnLength)
+	}
+
 	if extra.VoteAttestation != nil {
 		fmt.Printf("Attestation	:\n")
 		fmt.Printf("\tVoteAddressSet	:	%b,	%d\n", extra.VoteAddressSet, bitset.From([]uint64{uint64(extra.VoteAddressSet)}).Count())

cmd/faucet/faucet.go

@@ -49,6 +49,7 @@ import (
 	"github.com/ethereum/go-ethereum/log"
 	"github.com/ethereum/go-ethereum/params"
 	"github.com/gorilla/websocket"
+	"golang.org/x/time/rate"
 )
 
 var (
@@ -216,6 +217,8 @@ type faucet struct {
 
 	bep2eInfos map[string]bep2eInfo
 	bep2eAbi   abi.ABI
+
+	limiter *IPRateLimiter
 }
 
 // wsConn wraps a websocket connection with a write mutex as the underlying
@@ -235,6 +238,12 @@ func newFaucet(genesis *core.Genesis, url string, ks *keystore.KeyStore, index [
 		return nil, err
 	}
 
+	// Allow 1 request per minute with burst of 5, and cache up to 1000 IPs
+	limiter, err := NewIPRateLimiter(rate.Limit(1.0), 5, 1000)
+	if err != nil {
+		return nil, err
+	}
+
 	return &faucet{
 		config:     genesis.Config,
 		client:     client,
@@ -245,6 +254,7 @@ func newFaucet(genesis *core.Genesis, url string, ks *keystore.KeyStore, index [
 		update:     make(chan struct{}, 1),
 		bep2eInfos: bep2eInfos,
 		bep2eAbi:   bep2eAbi,
+		limiter:    limiter,
 	}, nil
 }
 
@@ -272,6 +282,20 @@ func (f *faucet) webHandler(w http.ResponseWriter, r *http.Request) {
 
 // apiHandler handles requests for Ether grants and transaction statuses.
 func (f *faucet) apiHandler(w http.ResponseWriter, r *http.Request) {
+	ip := r.RemoteAddr
+	if len(r.Header.Get("X-Forwarded-For")) > 0 {
+		ips := strings.Split(r.Header.Get("X-Forwarded-For"), ",")
+		if len(ips) > 0 {
+			ip = strings.TrimSpace(ips[len(ips)-1])
+		}
+	}
+
+	if !f.limiter.GetLimiter(ip).Allow() {
+		log.Warn("Too many requests from client: ", "client", ip)
+		http.Error(w, "Too many requests", http.StatusTooManyRequests)
+		return
+	}
+
 	upgrader := websocket.Upgrader{CheckOrigin: func(r *http.Request) bool { return true }}
 	conn, err := upgrader.Upgrade(w, r, nil)
 	if err != nil {
@@ -625,24 +649,27 @@ func (f *faucet) loop() {
 			balance := new(big.Int).Div(f.balance, ether)
 
 			for _, conn := range f.conns {
-				if err := send(conn, map[string]interface{}{
-					"funds":    balance,
-					"funded":   f.nonce,
-					"requests": f.reqs,
-				}, time.Second); err != nil {
-					log.Warn("Failed to send stats to client", "err", err)
-					conn.conn.Close()
-					continue
-				}
-				if err := send(conn, head, time.Second); err != nil {
-					log.Warn("Failed to send header to client", "err", err)
-					conn.conn.Close()
-				}
+				go func(conn *wsConn) {
+					if err := send(conn, map[string]interface{}{
+						"funds":    balance,
+						"funded":   f.nonce,
+						"requests": f.reqs,
+					}, time.Second); err != nil {
+						log.Warn("Failed to send stats to client", "err", err)
+						conn.conn.Close()
+						return // Exit the goroutine if the first send fails
+					}
+
+					if err := send(conn, head, time.Second); err != nil {
+						log.Warn("Failed to send header to client", "err", err)
+						conn.conn.Close()
+					}
+				}(conn)
 			}
 			f.lock.RUnlock()
 		}
 	}()
-	// Wait for various events and assing to the appropriate background threads
+	// Wait for various events and assign to the appropriate background threads
 	for {
 		select {
 		case head := <-heads:
@@ -656,10 +683,12 @@ func (f *faucet) loop() {
 			// Pending requests updated, stream to clients
 			f.lock.RLock()
 			for _, conn := range f.conns {
-				if err := send(conn, map[string]interface{}{"requests": f.reqs}, time.Second); err != nil {
-					log.Warn("Failed to send requests to client", "err", err)
-					conn.conn.Close()
-				}
+				go func(conn *wsConn) {
+					if err := send(conn, map[string]interface{}{"requests": f.reqs}, time.Second); err != nil {
+						log.Warn("Failed to send requests to client", "err", err)
+						conn.conn.Close()
+					}
+				}(conn)
 			}
 			f.lock.RUnlock()
 		}

cmd/faucet/rate_limiter.go

@@ -0,0 +1,44 @@
+package main
+
+import (
+	lru "github.com/hashicorp/golang-lru"
+	"golang.org/x/time/rate"
+)
+
+type IPRateLimiter struct {
+	ips *lru.Cache // LRU cache to store IP addresses and their associated rate limiters
+	r   rate.Limit // the rate limit, e.g., 5 requests per second
+	b   int        // the burst size, e.g., allowing a burst of 10 requests at once. The rate limiter gets into action
+	// only after this number exceeds
+}
+
+func NewIPRateLimiter(r rate.Limit, b int, size int) (*IPRateLimiter, error) {
+	cache, err := lru.New(size)
+	if err != nil {
+		return nil, err
+	}
+
+	i := &IPRateLimiter{
+		ips: cache,
+		r:   r,
+		b:   b,
+	}
+
+	return i, nil
+}
+
+func (i *IPRateLimiter) addIP(ip string) *rate.Limiter {
+	limiter := rate.NewLimiter(i.r, i.b)
+
+	i.ips.Add(ip, limiter)
+
+	return limiter
+}
+
+func (i *IPRateLimiter) GetLimiter(ip string) *rate.Limiter {
+	if limiter, exists := i.ips.Get(ip); exists {
+		return limiter.(*rate.Limiter)
+	}
+
+	return i.addIP(ip)
+}

cmd/geth/chaincmd.go

@@ -62,10 +62,10 @@ var (
 		ArgsUsage: "<genesisPath>",
 		Flags: flags.Merge([]cli.Flag{
 			utils.CachePreimagesFlag,
-			utils.OverrideCancun,
-			utils.OverrideHaber,
+			utils.OverridePassedForkTime,
 			utils.OverrideBohr,
 			utils.OverrideVerkle,
+			utils.MultiDataBaseFlag,
 		}, utils.DatabaseFlags),
 		Description: `
 The init command initializes a new genesis block and definition for the network.
@@ -254,13 +254,9 @@ func initGenesis(ctx *cli.Context) error {
 	defer stack.Close()
 
 	var overrides core.ChainOverrides
-	if ctx.IsSet(utils.OverrideCancun.Name) {
-		v := ctx.Uint64(utils.OverrideCancun.Name)
-		overrides.OverrideCancun = &v
-	}
-	if ctx.IsSet(utils.OverrideHaber.Name) {
-		v := ctx.Uint64(utils.OverrideHaber.Name)
-		overrides.OverrideHaber = &v
+	if ctx.IsSet(utils.OverridePassedForkTime.Name) {
+		v := ctx.Uint64(utils.OverridePassedForkTime.Name)
+		overrides.OverridePassedForkTime = &v
 	}
 	if ctx.IsSet(utils.OverrideBohr.Name) {
 		v := ctx.Uint64(utils.OverrideBohr.Name)
@@ -769,7 +765,7 @@ func parseDumpConfig(ctx *cli.Context, stack *node.Node) (*state.DumpConfig, eth
 		arg := ctx.Args().First()
 		if hashish(arg) {
 			hash := common.HexToHash(arg)
-			if number := rawdb.ReadHeaderNumber(db.BlockStore(), hash); number != nil {
+			if number := rawdb.ReadHeaderNumber(db, hash); number != nil {
 				header = rawdb.ReadHeader(db, hash, *number)
 			} else {
 				return nil, nil, common.Hash{}, fmt.Errorf("block %x not found", hash)

cmd/geth/config.go

@@ -185,13 +185,9 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
 		params.RialtoGenesisHash = common.HexToHash(v)
 	}
 
-	if ctx.IsSet(utils.OverrideCancun.Name) {
-		v := ctx.Uint64(utils.OverrideCancun.Name)
-		cfg.Eth.OverrideCancun = &v
-	}
-	if ctx.IsSet(utils.OverrideHaber.Name) {
-		v := ctx.Uint64(utils.OverrideHaber.Name)
-		cfg.Eth.OverrideHaber = &v
+	if ctx.IsSet(utils.OverridePassedForkTime.Name) {
+		v := ctx.Uint64(utils.OverridePassedForkTime.Name)
+		cfg.Eth.OverridePassedForkTime = &v
 	}
 	if ctx.IsSet(utils.OverrideBohr.Name) {
 		v := ctx.Uint64(utils.OverrideBohr.Name)
@@ -214,6 +210,9 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
 	if ctx.IsSet(utils.OverrideBreatheBlockInterval.Name) {
 		params.BreatheBlockInterval = ctx.Uint64(utils.OverrideBreatheBlockInterval.Name)
 	}
+	if ctx.IsSet(utils.OverrideFixedTurnLength.Name) {
+		params.FixedTurnLength = ctx.Uint64(utils.OverrideFixedTurnLength.Name)
+	}
 
 	backend, eth := utils.RegisterEthService(stack, &cfg.Eth)
 

cmd/geth/dbcmd.go

@@ -397,8 +397,8 @@ func inspectTrie(ctx *cli.Context) error {
 	var headerBlockHash common.Hash
 	if ctx.NArg() >= 1 {
 		if ctx.Args().Get(0) == "latest" {
-			headerHash := rawdb.ReadHeadHeaderHash(db.BlockStore())
-			blockNumber = *(rawdb.ReadHeaderNumber(db.BlockStore(), headerHash))
+			headerHash := rawdb.ReadHeadHeaderHash(db)
+			blockNumber = *(rawdb.ReadHeaderNumber(db, headerHash))
 		} else if ctx.Args().Get(0) == "snapshot" {
 			trieRootHash = rawdb.ReadSnapshotRoot(db)
 			blockNumber = math.MaxUint64
@@ -406,7 +406,7 @@ func inspectTrie(ctx *cli.Context) error {
 			var err error
 			blockNumber, err = strconv.ParseUint(ctx.Args().Get(0), 10, 64)
 			if err != nil {
-				return fmt.Errorf("failed to Parse blocknum, Args[0]: %v, err: %v", ctx.Args().Get(0), err)
+				return fmt.Errorf("failed to parse blocknum, Args[0]: %v, err: %v", ctx.Args().Get(0), err)
 			}
 		}
 
@@ -417,26 +417,26 @@ func inspectTrie(ctx *cli.Context) error {
 			var err error
 			jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64)
 			if err != nil {
-				return fmt.Errorf("failed to Parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
+				return fmt.Errorf("failed to parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
 			}
 			topN = 10
 		} else {
 			var err error
 			jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64)
 			if err != nil {
-				return fmt.Errorf("failed to Parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
+				return fmt.Errorf("failed to parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
 			}
 
 			topN, err = strconv.ParseUint(ctx.Args().Get(2), 10, 64)
 			if err != nil {
-				return fmt.Errorf("failed to Parse topn, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
+				return fmt.Errorf("failed to parse topn, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
 			}
 		}
 
 		if blockNumber != math.MaxUint64 {
 			headerBlockHash = rawdb.ReadCanonicalHash(db, blockNumber)
 			if headerBlockHash == (common.Hash{}) {
-				return errors.New("ReadHeadBlockHash empry hash")
+				return errors.New("ReadHeadBlockHash empty hash")
 			}
 			blockHeader := rawdb.ReadHeader(db, headerBlockHash, blockNumber)
 			trieRootHash = blockHeader.Root
@@ -508,7 +508,7 @@ func ancientInspect(ctx *cli.Context) error {
 	stack, _ := makeConfigNode(ctx)
 	defer stack.Close()
 
-	db := utils.MakeChainDatabase(ctx, stack, true, true)
+	db := utils.MakeChainDatabase(ctx, stack, true, false)
 	defer db.Close()
 	return rawdb.AncientInspect(db)
 }
@@ -1212,7 +1212,7 @@ func showMetaData(ctx *cli.Context) error {
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Error accessing ancients: %v", err)
 	}
-	data := rawdb.ReadChainMetadataFromMultiDatabase(db)
+	data := rawdb.ReadChainMetadata(db)
 	data = append(data, []string{"frozen", fmt.Sprintf("%d items", ancients)})
 	data = append(data, []string{"snapshotGenerator", snapshot.ParseGeneratorStatus(rawdb.ReadSnapshotGenerator(db))})
 	if b := rawdb.ReadHeadBlock(db); b != nil {
@@ -1255,7 +1255,7 @@ func hbss2pbss(ctx *cli.Context) error {
 	defer stack.Close()
 
 	db := utils.MakeChainDatabase(ctx, stack, false, false)
-	db.Sync()
+	db.BlockStore().Sync()
 	stateDiskDb := db.StateStore()
 	defer db.Close()
 
@@ -1273,8 +1273,8 @@ func hbss2pbss(ctx *cli.Context) error {
 		log.Info("hbss2pbss triedb", "scheme", triedb.Scheme())
 		defer triedb.Close()
 
-		headerHash := rawdb.ReadHeadHeaderHash(db.BlockStore())
-		blockNumber := rawdb.ReadHeaderNumber(db.BlockStore(), headerHash)
+		headerHash := rawdb.ReadHeadHeaderHash(db)
+		blockNumber := rawdb.ReadHeaderNumber(db, headerHash)
 		if blockNumber == nil {
 			log.Error("read header number failed.")
 			return fmt.Errorf("read header number failed")

cmd/geth/main.go

@@ -72,14 +72,14 @@ var (
 		utils.USBFlag,
 		utils.SmartCardDaemonPathFlag,
 		utils.RialtoHash,
-		utils.OverrideCancun,
-		utils.OverrideHaber,
+		utils.OverridePassedForkTime,
 		utils.OverrideBohr,
 		utils.OverrideVerkle,
 		utils.OverrideFullImmutabilityThreshold,
 		utils.OverrideMinBlocksForBlobRequests,
 		utils.OverrideDefaultExtraReserveForBlobRequests,
 		utils.OverrideBreatheBlockInterval,
+		utils.OverrideFixedTurnLength,
 		utils.EnablePersonal,
 		utils.TxPoolLocalsFlag,
 		utils.TxPoolNoLocalsFlag,
@@ -127,6 +127,7 @@ var (
 		utils.CacheSnapshotFlag,
 		// utils.CacheNoPrefetchFlag,
 		utils.CachePreimagesFlag,
+		utils.MultiDataBaseFlag,
 		utils.PersistDiffFlag,
 		utils.DiffBlockFlag,
 		utils.PruneAncientDataFlag,
@@ -336,9 +337,6 @@ func prepare(ctx *cli.Context) {
   5. Networking is disabled; there is no listen-address, the maximum number of peers is set
      to 0, and discovery is disabled.
 `)
-
-	case !ctx.IsSet(utils.NetworkIdFlag.Name):
-		log.Info("Starting Geth on BSC mainnet...")
 	}
 	// If we're a full node on mainnet without --cache specified, bump default cache allowance
 	if !ctx.IsSet(utils.CacheFlag.Name) && !ctx.IsSet(utils.NetworkIdFlag.Name) {

cmd/geth/pruneblock_test.go

@@ -155,6 +155,12 @@ func BlockchainCreator(t *testing.T, chaindbPath, AncientPath string, blockRemai
 	triedb := triedb.NewDatabase(db, nil)
 	defer triedb.Close()
 
+	if err = db.SetupFreezerEnv(&ethdb.FreezerEnv{
+		ChainCfg:         gspec.Config,
+		BlobExtraReserve: params.DefaultExtraReserveForBlobRequests,
+	}); err != nil {
+		t.Fatalf("Failed to create chain: %v", err)
+	}
 	genesis := gspec.MustCommit(db, triedb)
 	// Initialize a fresh chain with only a genesis block
 	blockchain, err := core.NewBlockChain(db, config, gspec, nil, engine, vm.Config{}, nil, nil)

cmd/jsutils/README.md

@@ -24,4 +24,24 @@ testnet validators version
 ### 2.Get Transaction Count
 ```bash
 node gettxcount.js --rpc ${url} --startNum ${start} --endNum ${end} --miner ${miner} (optional)
-```
+```
+
+### 3. Get Performance
+```bash
+node get_perf.js --rpc ${url} --startNum ${start} --endNum ${end}
+```
+output as following
+```bash
+Get the performance between [ 19470 , 19670 )
+txCountPerBlock = 3142.81 txCountTotal = 628562 BlockCount = 200 avgBlockTime = 3.005 inturnBlocksRatio = 0.975 justifiedBlocksRatio = 0.98
+txCountPerSecond = 1045.8602329450914 avgGasUsedPerBlock = 250.02062627 avgGasUsedPerSecond =  83.20153952412646
+```
+
+### 4. Get validators slash count
+```bash
+use the latest block 
+node getslashcount.js --Rpc ${ArchiveRpc} 
+use a block number
+node getslashcount.js --Rpc ${ArchiveRpc} --Num ${blockNum}
+```
+

cmd/jsutils/get_perf.js

@@ -0,0 +1,70 @@
+import { ethers } from "ethers";
+import program from "commander";
+
+program.option("--rpc <rpc>", "Rpc");
+program.option("--startNum <startNum>", "start num")
+program.option("--endNum <endNum>", "end num")
+program.parse(process.argv);
+
+const provider = new ethers.JsonRpcProvider(program.rpc)
+
+const main = async () => {
+    let txCountTotal = 0;
+    let gasUsedTotal = 0;
+    let inturnBlocks = 0;
+    let justifiedBlocks = 0;
+    let turnLength = await provider.send("parlia_getTurnLength", [
+        ethers.toQuantity(program.startNum)]);
+    for (let i = program.startNum; i < program.endNum; i++) {
+        let txCount = await provider.send("eth_getBlockTransactionCountByNumber", [
+            ethers.toQuantity(i)]);
+        txCountTotal += ethers.toNumber(txCount)
+
+        let header = await provider.send("eth_getHeaderByNumber", [
+            ethers.toQuantity(i)]);
+        let gasUsed = eval(eval(header.gasUsed).toString(10))
+        gasUsedTotal += gasUsed
+        let difficulty = eval(eval(header.difficulty).toString(10))
+        if (difficulty == 2) {
+            inturnBlocks += 1
+        }
+        let timestamp = eval(eval(header.timestamp).toString(10))
+
+        let justifiedNumber = await provider.send("parlia_getJustifiedNumber", [
+            ethers.toQuantity(i)]);
+        if (justifiedNumber + 1 == i) {
+            justifiedBlocks += 1
+        } else {
+            console.log("justified unexpected", "BlockNumber =", i,"justifiedNumber",justifiedNumber)    
+        }
+        console.log("BlockNumber =", i, "mod =", i%turnLength, "miner =", header.miner , "difficulty =", difficulty, "txCount =", ethers.toNumber(txCount), "gasUsed", gasUsed, "timestamp", timestamp)
+    }
+
+    let blockCount = program.endNum - program.startNum
+    let txCountPerBlock = txCountTotal/blockCount
+
+    let startHeader = await provider.send("eth_getHeaderByNumber", [
+        ethers.toQuantity(program.startNum)]);
+    let startTime = eval(eval(startHeader.timestamp).toString(10))
+    let endHeader = await provider.send("eth_getHeaderByNumber", [
+        ethers.toQuantity(program.endNum)]);
+    let endTime = eval(eval(endHeader.timestamp).toString(10))
+    let timeCost = endTime - startTime
+    let avgBlockTime = timeCost/blockCount
+    let inturnBlocksRatio = inturnBlocks/blockCount
+    let justifiedBlocksRatio = justifiedBlocks/blockCount
+    let tps = txCountTotal/timeCost
+    let M = 1000000
+    let avgGasUsedPerBlock = gasUsedTotal/blockCount/M
+    let avgGasUsedPerSecond = gasUsedTotal/timeCost/M
+
+    console.log("Get the performance between [", program.startNum, ",", program.endNum, ")");
+    console.log("txCountPerBlock =", txCountPerBlock, "txCountTotal =", txCountTotal, "BlockCount =", blockCount, "avgBlockTime =", avgBlockTime, "inturnBlocksRatio =", inturnBlocksRatio, "justifiedBlocksRatio =", justifiedBlocksRatio);
+    console.log("txCountPerSecond =", tps, "avgGasUsedPerBlock =", avgGasUsedPerBlock, "avgGasUsedPerSecond =", avgGasUsedPerSecond);
+};
+
+main().then(() => process.exit(0))
+    .catch((error) => {
+        console.error(error);
+        process.exit(1);
+    });

cmd/jsutils/getchainstatus.js

@@ -0,0 +1,164 @@
+import { ethers } from "ethers";
+import program from "commander";
+
+// Global Options:
+program.option("--rpc <rpc>", "Rpc");
+// GetTxCount Options:
+program.option("--startNum <startNum>", "start num")
+program.option("--endNum <endNum>", "end num")
+program.option("--miner <miner>", "miner", "")
+// GetVersion Options:
+program.option("--num <Num>", "validator num", 21)
+// GetTopAddr Options:
+program.option("--topNum <Num>", "top num of address to be displayed", 20)
+
+program.parse(process.argv);
+
+const provider = new ethers.JsonRpcProvider(program.rpc)
+
+function printUsage() {
+    console.log("Usage:");
+    console.log("  node getchainstatus.js --help");
+    console.log("  node getchainstatus.js [subcommand] [options]");
+    console.log("\nSubcommands:");
+    console.log("  GetTxCount: find the block with max tx size of a range");
+    console.log("  GetVersion: dump validators' binary version, based on Header.Extra");
+    console.log("  GetTopAddr: get hottest $topNum target address within a block range");
+    console.log("\nOptions:");
+    console.log("  --rpc       specify the url of RPC endpoint");
+    console.log("  --startNum  the start block number, for command GetTxCount");
+    console.log("  --endNum    the end block number, for command GetTxCount");
+    console.log("  --miner     the miner address, for command GetTxCount");
+    console.log("  --num       the number of blocks to be checked, for command GetVersion");
+    console.log("  --topNum    the topNum of blocks to be checked, for command GetVersion");
+    console.log("\nExample:");
+    // mainnet https://bsc-mainnet.nodereal.io/v1/454e504917db4f82b756bd0cf6317dce
+    console.log("  node getchainstatus.js GetTxCount --rpc https://bsc-testnet-dataseed.bnbchain.org --startNum 40000001  --endNum 40000005")
+    console.log("  node getchainstatus.js GetVersion --rpc https://bsc-testnet-dataseed.bnbchain.org --num 21")
+    console.log("  node getchainstatus.js GetTopAddr --rpc https://bsc-testnet-dataseed.bnbchain.org --startNum 40000001  --endNum 40000010 --topNum 10")
+}
+
+// 1.cmd: "GetTxCount", usage:
+// node getchainstatus.js GetTxCount --rpc https://bsc-testnet-dataseed.bnbchain.org \
+//      --startNum 40000001  --endNum 40000005 \
+//      --miner(optional): specified: find the max txCounter from the specified validator,
+//                         not specified: find the max txCounter from all validators
+async function getTxCount()  {
+    let txCount = 0;
+    let num = 0;
+    console.log("Find the max txs count between", program.startNum, "and", program.endNum);
+    for (let i = program.startNum; i < program.endNum; i++) {
+        if (program.miner !== "") {
+            let blockData = await provider.getBlock(Number(i))
+            if (program.miner !== blockData.miner) {
+                continue
+            }
+        }
+        let x = await provider.send("eth_getBlockTransactionCountByNumber", [
+            ethers.toQuantity(i)]);
+        let a = ethers.toNumber(x)
+        if (a > txCount) {
+            num = i;
+            txCount = a;
+        }
+    }
+    console.log("BlockNum = ", num, "TxCount =", txCount);
+}
+
+// 2.cmd: "GetVersion", usage:
+// node getchainstatus.js GetVersion \
+//      --rpc https://bsc-testnet-dataseed.bnbchain.org \
+//       --num(optional): defualt 21, the number of blocks that will be checked
+async function getBinaryVersion()  {
+    const blockNum = await provider.getBlockNumber();
+    console.log(blockNum);
+    for (let i = 0; i < program.num; i++) {
+        let blockData = await provider.getBlock(blockNum - i);
+        // 1.get Geth client version
+        let major = ethers.toNumber(ethers.dataSlice(blockData.extraData, 2, 3))
+        let minor = ethers.toNumber(ethers.dataSlice(blockData.extraData, 3, 4))
+        let patch = ethers.toNumber(ethers.dataSlice(blockData.extraData, 4, 5))
+
+        // 2.get minimum txGasPrice based on the last non-zero-gasprice transaction
+        let lastGasPrice = 0
+        for  (let txIndex = blockData.transactions.length - 1; txIndex >= 0; txIndex--) {
+            let txHash = blockData.transactions[txIndex]
+            let txData =  await provider.getTransaction(txHash);
+            if (txData.gasPrice == 0) {
+                continue
+            }
+            lastGasPrice = txData.gasPrice
+            break
+        }
+        console.log(blockData.miner, "version =", major + "." + minor + "." + patch, " MinGasPrice = " + lastGasPrice)
+    }
+};
+
+// 3.cmd: "GetTopAddr", usage:
+// node getchainstatus.js GetTopAddr \
+//      --rpc https://bsc-testnet-dataseed.bnbchain.org \
+//      --startNum 40000001  --endNum 40000005 \
+//      --topNum(optional): the top num of address to be displayed, default 20
+function getTopKElements(map, k) {
+    let entries = Array.from(map.entries());
+    entries.sort((a, b) => b[1] - a[1]);
+    return entries.slice(0, k);
+}
+
+async function getTopAddr()  {
+    let countMap = new Map();
+    let totalTxs = 0
+    console.log("Find the top target address, between", program.startNum, "and", program.endNum);
+    for (let i = program.startNum; i <= program.endNum; i++) {
+        let blockData = await provider.getBlock(Number(i), true)
+        totalTxs += blockData.transactions.length
+        for  (let txIndex = blockData.transactions.length - 1; txIndex >= 0; txIndex--) {
+            let txData = await blockData.getTransaction(txIndex)
+            if (txData.to == null) {
+                console.log("Contract creation,txHash:", txData.hash)
+                continue
+            }
+            let toAddr = txData.to;
+            if (countMap.has(toAddr)) {
+                countMap.set(toAddr, countMap.get(toAddr) + 1);
+            } else {
+                countMap.set(toAddr, 1);
+            }
+        }
+        console.log("progress:", (program.endNum-i), "blocks left", "totalTxs", totalTxs)
+    }
+    let tops = getTopKElements(countMap, program.topNum)
+    tops.forEach((value, key) => {
+        // value: [ '0x40661F989826CC641Ce1601526Bb16a4221412c8', 71 ]
+        console.log(key+":", value[0], " ", value[1], " ", ((value[1]*100)/totalTxs).toFixed(2)+"%");
+      });
+};
+
+const main = async () => {
+    if (process.argv.length <= 2) {
+        console.error('invalid process.argv.length', process.argv.length);
+        printUsage()
+        return
+    }
+    const cmd =  process.argv[2]
+    if (cmd === "--help") {
+        printUsage()
+        return
+    }
+    if (cmd === "GetTxCount") {
+        await getTxCount()
+    } else if (cmd === "GetVersion") {
+        await getBinaryVersion()
+    } else if (cmd === "GetTopAddr") {
+        await getTopAddr()
+    } else {
+        console.log("unsupported cmd", cmd);
+        printUsage()
+    }
+}
+
+main().then(() => process.exit(0))
+    .catch((error) => {
+        console.error(error);
+        process.exit(1);
+    });

cmd/jsutils/getslashcount.js

@@ -0,0 +1,119 @@
+import { ethers } from "ethers";
+import program from "commander";
+
+program.option("--Rpc <Rpc>", "Rpc");
+program.option("--Num <Num>", "num", 0)
+program.parse(process.argv);
+
+const provider = new ethers.JsonRpcProvider(program.Rpc);
+
+const slashAbi = [
+    "function getSlashIndicator(address validatorAddr) external view returns (uint256, uint256)"
+]
+const validatorSetAbi = [
+    "function getLivingValidators() external view returns (address[], bytes[])"
+]
+const stakeHubAbi = [
+    "function getValidatorDescription(address validatorAddr) external view returns (tuple(string, string, string, string))",
+    "function consensusToOperator(address consensusAddr) public view returns (address)"
+]
+const addrValidatorSet = '0x0000000000000000000000000000000000001000';
+const validatorSet = new ethers.Contract(addrValidatorSet, validatorSetAbi, provider);
+
+const addrSlash = '0x0000000000000000000000000000000000001001';
+const slashIndicator = new ethers.Contract(addrSlash, slashAbi,  provider)
+
+const addrStakeHub = '0x0000000000000000000000000000000000002002';
+const stakeHub = new ethers.Contract(addrStakeHub, stakeHubAbi, provider)
+
+const validatorMap = new Map([
+    //BSC
+    ["0x37e9627A91DD13e453246856D58797Ad6583D762", "LegendII"],
+    ["0xB4647b856CB9C3856d559C885Bed8B43e0846a47", "CertiK"],
+    ["0x75B851a27D7101438F45fce31816501193239A83", "Figment"],
+    ["0x502aECFE253E6AA0e8D2A06E12438FFeD0Fe16a0", "BscScan"],
+    ["0xCa503a7eD99eca485da2E875aedf7758472c378C", "InfStones"],
+    ["0x5009317FD4F6F8FeEa9dAe41E5F0a4737BB7A7D5", "NodeReal"],
+    ["0x1cFDBd2dFf70C6e2e30df5012726F87731F38164", "Tranchess"],
+    ["0xF8de5e61322302b2c6e0a525cC842F10332811bf", "Namelix"],
+    ["0xCcB42A9b8d6C46468900527Bc741938E78AB4577", "Turing"],
+    ["0x9f1b7FAE54BE07F4FEE34Eb1aaCb39A1F7B6FC92", "TWStaking"],
+    ["0x7E1FdF03Eb3aC35BF0256694D7fBe6B6d7b3E0c8","LegendIII"],
+    ["0x7b501c7944185130DD4aD73293e8Aa84eFfDcee7","MathW"],
+    ["0x58567F7A51a58708C8B40ec592A38bA64C0697De","Legend"],
+    ["0x460A252B4fEEFA821d3351731220627D7B7d1F3d","Defibit"],
+    ["0x8A239732871AdC8829EA2f47e94087C5FBad47b6","The48Club"],
+    ["0xD3b0d838cCCEAe7ebF1781D11D1bB741DB7Fe1A7","BNBEve"],
+    ["0xF8B99643fAfC79d9404DE68E48C4D49a3936f787","Avengers"],
+    ["0x4e5acf9684652BEa56F2f01b7101a225Ee33d23f","HashKey"],
+    ["0x9bb56C2B4DBE5a06d79911C9899B6f817696ACFc","Feynman"],
+    ["0xbdcc079BBb23C1D9a6F36AA31309676C258aBAC7","Fuji"],
+    ["0x38944092685a336CB6B9ea58836436709a2adC89","Shannon"],
+    ["0xfC1004C0f296Ec3Df4F6762E9EabfcF20EB304a2","Aoraki"],
+    ["0xa0884bb00E5F23fE2427f0E5eC9E51F812848563","Coda"],
+    ["0xe7776De78740f28a96412eE5cbbB8f90896b11A5","Ankr"],
+    ["0xA2D969E82524001Cb6a2357dBF5922B04aD2FCD8","Pexmons"],
+    ["0x5cf810AB8C718ac065b45f892A5BAdAB2B2946B9","Zen"],
+    ["0x4d15D9BCd0c2f33E7510c0de8b42697CA558234a","LegendVII"],
+    ["0x1579ca96EBd49A0B173f86C372436ab1AD393380","LegendV"],
+    ["0xd1F72d433f362922f6565FC77c25e095B29141c8","LegendVI"],
+    ["0xf9814D93b4d904AaA855cBD4266D6Eb0Ec1Aa478","Legend8"],
+    ["0x025a4e09Ea947b8d695f53ddFDD48ddB8F9B06b7","Ciscox"],
+    ["0xE9436F6F30b4B01b57F2780B2898f3820EbD7B98","LegendIV"],
+    ["0xC2d534F079444E6E7Ff9DabB3FD8a26c607932c8","Axion"],
+    ["0x9F7110Ba7EdFda83Fc71BeA6BA3c0591117b440D","LegendIX"],
+    ["0xB997Bf1E3b96919fBA592c1F61CE507E165Ec030","Seoraksan"],
+    ["0x286C1b674d48cFF67b4096b6c1dc22e769581E91","Sigm8"],
+    ["0x73A26778ef9509a6E94b55310eE7233795a9EB25","Coinlix"],
+    ["0x18c44f4FBEde9826C7f257d500A65a3D5A8edebc","Nozti"],
+    ["0xA100FCd08cE722Dc68Ddc3b54237070Cb186f118","Tiollo"],
+    ["0x0F28847cfdbf7508B13Ebb9cEb94B2f1B32E9503","Raptas"],
+    ["0xfD85346c8C991baC16b9c9157e6bdfDACE1cD7d7","Glorin"],
+    ["0x978F05CED39A4EaFa6E8FD045Fe2dd6Da836c7DF","NovaX"],
+    ["0xd849d1dF66bFF1c2739B4399425755C2E0fAbbAb","Nexa"],
+    ["0xA015d9e9206859c13201BB3D6B324d6634276534","Star"],
+    ["0x5ADde0151BfAB27f329e5112c1AeDeed7f0D3692","Veri"],
+    //Chapel
+    ["0x08265dA01E1A65d62b903c7B34c08cB389bF3D99","Ararat"],
+    ["0x7f5f2cF1aec83bF0c74DF566a41aa7ed65EA84Ea","Kita"],
+    ["0x53387F3321FD69d1E030BB921230dFb188826AFF","Fuji"],
+    ["0x76D76ee8823dE52A1A431884c2ca930C5e72bff3","Seoraksan"],
+    ["0xd447b49CD040D20BC21e49ffEa6487F5638e4346","Everest"],
+    ["0x1a3d9D7A717D64e6088aC937d5aAcDD3E20ca963","Elbrus"],
+    ["0x40D3256EB0BaBE89f0ea54EDAa398513136612f5","Bloxroute"],
+    ["0xF9a1Db0d6f22Bd78ffAECCbc8F47c83Df9FBdbCf","Test"]
+]);
+
+
+const main = async () => {
+    let blockNum = ethers.getNumber(program.Num)
+    if (blockNum === 0) {
+       blockNum = await provider.getBlockNumber()
+    }
+    let block = await provider.getBlock(blockNum)
+    console.log("At block", blockNum, "time", block.date)
+    const data = await validatorSet.getLivingValidators({blockTag:blockNum})
+    let totalSlash = 0
+    for (let i = 0; i < data[0].length; i++) {
+        let addr = data[0][i];
+        var val
+        if (!validatorMap.has(addr)) {
+            let opAddr = await stakeHub.consensusToOperator(addr, {blockTag:blockNum})
+            let value = await stakeHub.getValidatorDescription(opAddr, {blockTag:blockNum})
+            val = value[0]
+            console.log(addr, val)
+        } else {
+            val = validatorMap.get(addr)
+        }
+        let info = await slashIndicator.getSlashIndicator(addr, {blockTag:blockNum})
+        let count = ethers.toNumber(info[1])
+        totalSlash += count
+        console.log("Slash:", count, addr, val)
+    }
+    console.log("Total slash count", totalSlash)
+};
+main().then(() => process.exit(0))
+    .catch((error) => {
+        console.error(error);
+        process.exit(1);
+    });

cmd/jsutils/gettxcount.js

@@ -1,41 +0,0 @@
-import { ethers } from "ethers";
-import program from "commander";
-
-program.option("--rpc <rpc>", "Rpc");
-program.option("--startNum <startNum>", "start num")
-program.option("--endNum <endNum>", "end num")
-// --miner:
-//   specified: find the max txCounter from the specified validator
-//   not specified: find the max txCounter from all validators
-program.option("--miner <miner>", "miner", "")
-program.parse(process.argv);
-
-const provider = new ethers.JsonRpcProvider(program.rpc)
-
-const main = async () => {
-    let txCount = 0;
-    let num = 0;
-    console.log("Find the max txs count between", program.startNum, "and", program.endNum);
-    for (let i = program.startNum; i < program.endNum; i++) {
-        if (program.miner !== "") {
-            let blockData = await provider.getBlock(Number(i))
-            if (program.miner !== blockData.miner) {
-                continue
-            }
-        }
-        let x = await provider.send("eth_getBlockTransactionCountByNumber", [
-            ethers.toQuantity(i)]);
-        let a = ethers.toNumber(x)
-        if (a > txCount) {
-            num = i;
-            txCount = a;
-        }
-    }
-    console.log("BlockNum = ", num, "TxCount =", txCount);
-};
-
-main().then(() => process.exit(0))
-    .catch((error) => {
-        console.error(error);
-        process.exit(1);
-    });

cmd/jsutils/getvalidatorversion.js

@@ -1,38 +0,0 @@
-import { ethers } from "ethers";
-import program from "commander";
-
-program.option("--Rpc <Rpc>", "Rpc");
-program.option("--Num <Num>", "validator num", 21)
-program.parse(process.argv);
-
-const provider = new ethers.JsonRpcProvider(program.Rpc);
-
-const main = async () => {
-    const blockNum = await provider.getBlockNumber();
-    console.log(blockNum);
-    for (let i = 0; i < program.Num; i++) {
-        let blockData = await provider.getBlock(blockNum - i);
-        // 1.get Geth client version
-        let major = ethers.toNumber(ethers.dataSlice(blockData.extraData, 2, 3))
-        let minor = ethers.toNumber(ethers.dataSlice(blockData.extraData, 3, 4))
-        let patch = ethers.toNumber(ethers.dataSlice(blockData.extraData, 4, 5))
-
-        // 2.get minimum txGasPrice based on the last non-zero-gasprice transaction
-        let lastGasPrice = 0
-        for  (let txIndex = blockData.transactions.length - 1; txIndex >= 0; txIndex--) {
-            let txHash = blockData.transactions[txIndex]
-            let txData =  await provider.getTransaction(txHash);
-            if (txData.gasPrice == 0) {
-                continue
-            }
-            lastGasPrice = txData.gasPrice
-            break
-        }
-        console.log(blockData.miner, "version =", major + "." + minor + "." + patch, " MinGasPrice = " + lastGasPrice)
-    }
-};
-main().then(() => process.exit(0))
-    .catch((error) => {
-        console.error(error);
-        process.exit(1);
-    });

cmd/utils/flags.go

@@ -305,14 +305,9 @@ var (
 		Usage:    "Manually specify the Rialto Genesis Hash, to trigger builtin network logic",
 		Category: flags.EthCategory,
 	}
-	OverrideCancun = &cli.Uint64Flag{
-		Name:     "override.cancun",
-		Usage:    "Manually specify the Cancun fork timestamp, overriding the bundled setting",
-		Category: flags.EthCategory,
-	}
-	OverrideHaber = &cli.Uint64Flag{
-		Name:     "override.haber",
-		Usage:    "Manually specify the Haber fork timestamp, overriding the bundled setting",
+	OverridePassedForkTime = &cli.Uint64Flag{
+		Name:     "override.passedforktime",
+		Usage:    "Manually specify the hard fork timestamp except the last one, overriding the bundled setting",
 		Category: flags.EthCategory,
 	}
 	OverrideBohr = &cli.Uint64Flag{
@@ -349,6 +344,12 @@ var (
 		Value:    params.BreatheBlockInterval,
 		Category: flags.EthCategory,
 	}
+	OverrideFixedTurnLength = &cli.Uint64Flag{
+		Name:     "override.fixedturnlength",
+		Usage:    "It use fixed or random values for turn length instead of reading from the contract, only for testing purpose",
+		Value:    params.FixedTurnLength,
+		Category: flags.EthCategory,
+	}
 	SyncModeFlag = &flags.TextMarshalerFlag{
 		Name:     "syncmode",
 		Usage:    `Blockchain sync mode ("snap" or "full")`,
@@ -1163,7 +1164,6 @@ var (
 		DBEngineFlag,
 		StateSchemeFlag,
 		HttpHeaderFlag,
-		MultiDataBaseFlag,
 	}
 )
 
@@ -2083,7 +2083,7 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) {
 		}
 		cfg.Genesis = core.DefaultBSCGenesisBlock()
 		SetDNSDiscoveryDefaults(cfg, params.BSCGenesisHash)
-	case ctx.Bool(ChapelFlag.Name):
+	case ctx.Bool(ChapelFlag.Name) || cfg.NetworkId == 97:
 		if !ctx.IsSet(NetworkIdFlag.Name) {
 			cfg.NetworkId = 97
 		}

cmd/utils/history_test.go

@@ -83,7 +83,7 @@ func TestHistoryImportAndExport(t *testing.T) {
 		t.Fatalf("unable to initialize chain: %v", err)
 	}
 	if _, err := chain.InsertChain(blocks); err != nil {
-		t.Fatalf("error insterting chain: %v", err)
+		t.Fatalf("error inserting chain: %v", err)
 	}
 
 	// Make temp directory for era files.

consensus/consensus.go

@@ -59,6 +59,9 @@ type ChainHeaderReader interface {
 	// GetHighestVerifiedHeader retrieves the highest header verified.
 	GetHighestVerifiedHeader() *types.Header
 
+	// GetVerifiedBlockByHash retrieves the highest verified block.
+	GetVerifiedBlockByHash(hash common.Hash) *types.Header
+
 	// ChasingHead return the best chain head of peers.
 	ChasingHead() *types.Header
 }

consensus/parlia/abi.go

@@ -2306,6 +2306,19 @@ const validatorSetABI = `
     ],
     "stateMutability": "view"
   },
+  {
+    "inputs": [],
+    "name": "getTurnLength",
+    "outputs": [
+      {
+        "internalType": "uint256",
+        "name": "",
+        "type": "uint256"
+      }
+    ],
+    "stateMutability": "view",
+    "type": "function"
+  },
   {
     "type": "function",
     "name": "getValidators",

consensus/parlia/api.go

@@ -31,13 +31,7 @@ type API struct {
 
 // GetSnapshot retrieves the state snapshot at a given block.
 func (api *API) GetSnapshot(number *rpc.BlockNumber) (*Snapshot, error) {
-	// Retrieve the requested block number (or current if none requested)
-	var header *types.Header
-	if number == nil || *number == rpc.LatestBlockNumber {
-		header = api.chain.CurrentHeader()
-	} else {
-		header = api.chain.GetHeaderByNumber(uint64(number.Int64()))
-	}
+	header := api.getHeader(number)
 	// Ensure we have an actually valid block and return its snapshot
 	if header == nil {
 		return nil, errUnknownBlock
@@ -56,13 +50,7 @@ func (api *API) GetSnapshotAtHash(hash common.Hash) (*Snapshot, error) {
 
 // GetValidators retrieves the list of validators at the specified block.
 func (api *API) GetValidators(number *rpc.BlockNumber) ([]common.Address, error) {
-	// Retrieve the requested block number (or current if none requested)
-	var header *types.Header
-	if number == nil || *number == rpc.LatestBlockNumber {
-		header = api.chain.CurrentHeader()
-	} else {
-		header = api.chain.GetHeaderByNumber(uint64(number.Int64()))
-	}
+	header := api.getHeader(number)
 	// Ensure we have an actually valid block and return the validators from its snapshot
 	if header == nil {
 		return nil, errUnknownBlock
@@ -86,3 +74,65 @@ func (api *API) GetValidatorsAtHash(hash common.Hash) ([]common.Address, error)
 	}
 	return snap.validators(), nil
 }
+
+func (api *API) GetJustifiedNumber(number *rpc.BlockNumber) (uint64, error) {
+	header := api.getHeader(number)
+	// Ensure we have an actually valid block and return the validators from its snapshot
+	if header == nil {
+		return 0, errUnknownBlock
+	}
+	snap, err := api.parlia.snapshot(api.chain, header.Number.Uint64(), header.Hash(), nil)
+	if err != nil || snap.Attestation == nil {
+		return 0, err
+	}
+	return snap.Attestation.TargetNumber, nil
+}
+
+func (api *API) GetTurnLength(number *rpc.BlockNumber) (uint8, error) {
+	header := api.getHeader(number)
+	// Ensure we have an actually valid block and return the validators from its snapshot
+	if header == nil {
+		return 0, errUnknownBlock
+	}
+	snap, err := api.parlia.snapshot(api.chain, header.Number.Uint64(), header.Hash(), nil)
+	if err != nil || snap.TurnLength == 0 {
+		return 0, err
+	}
+	return snap.TurnLength, nil
+}
+
+func (api *API) GetFinalizedNumber(number *rpc.BlockNumber) (uint64, error) {
+	header := api.getHeader(number)
+	// Ensure we have an actually valid block and return the validators from its snapshot
+	if header == nil {
+		return 0, errUnknownBlock
+	}
+	snap, err := api.parlia.snapshot(api.chain, header.Number.Uint64(), header.Hash(), nil)
+	if err != nil || snap.Attestation == nil {
+		return 0, err
+	}
+	return snap.Attestation.SourceNumber, nil
+}
+
+func (api *API) getHeader(number *rpc.BlockNumber) (header *types.Header) {
+	currentHeader := api.chain.CurrentHeader()
+
+	if number == nil || *number == rpc.LatestBlockNumber {
+		header = currentHeader // current if none requested
+	} else if *number == rpc.SafeBlockNumber {
+		justifiedNumber, _, err := api.parlia.GetJustifiedNumberAndHash(api.chain, []*types.Header{currentHeader})
+		if err != nil {
+			return nil
+		}
+		header = api.chain.GetHeaderByNumber(justifiedNumber)
+	} else if *number == rpc.FinalizedBlockNumber {
+		header = api.parlia.GetFinalizedHeader(api.chain, currentHeader)
+	} else if *number == rpc.PendingBlockNumber {
+		return nil // no pending blocks on bsc
+	} else if *number == rpc.EarliestBlockNumber {
+		header = api.chain.GetHeaderByNumber(0)
+	} else {
+		header = api.chain.GetHeaderByNumber(uint64(number.Int64()))
+	}
+	return
+}

consensus/parlia/bohrFork.go

@@ -0,0 +1,91 @@
+package parlia
+
+import (
+	"context"
+	"errors"
+	"math/big"
+	mrand "math/rand"
+
+	"github.com/ethereum/go-ethereum/common"
+	"github.com/ethereum/go-ethereum/common/hexutil"
+	"github.com/ethereum/go-ethereum/common/math"
+	"github.com/ethereum/go-ethereum/consensus"
+	"github.com/ethereum/go-ethereum/core/systemcontracts"
+	"github.com/ethereum/go-ethereum/core/types"
+	"github.com/ethereum/go-ethereum/internal/ethapi"
+	"github.com/ethereum/go-ethereum/log"
+	"github.com/ethereum/go-ethereum/params"
+	"github.com/ethereum/go-ethereum/rpc"
+)
+
+func (p *Parlia) getTurnLength(chain consensus.ChainHeaderReader, header *types.Header) (*uint8, error) {
+	parent := chain.GetHeaderByHash(header.ParentHash)
+	if parent == nil {
+		return nil, errors.New("parent not found")
+	}
+
+	var turnLength uint8
+	if p.chainConfig.IsBohr(parent.Number, parent.Time) {
+		turnLengthFromContract, err := p.getTurnLengthFromContract(parent)
+		if err != nil {
+			return nil, err
+		}
+		if turnLengthFromContract == nil {
+			return nil, errors.New("unexpected error when getTurnLengthFromContract")
+		}
+		turnLength = uint8(turnLengthFromContract.Int64())
+	} else {
+		turnLength = defaultTurnLength
+	}
+	log.Debug("getTurnLength", "turnLength", turnLength)
+
+	return &turnLength, nil
+}
+
+func (p *Parlia) getTurnLengthFromContract(header *types.Header) (turnLength *big.Int, err error) {
+	// mock to get turnLength from the contract
+	if params.FixedTurnLength >= 1 && params.FixedTurnLength <= 9 {
+		if params.FixedTurnLength == 2 {
+			return p.getRandTurnLength(header)
+		}
+		return big.NewInt(int64(params.FixedTurnLength)), nil
+	}
+
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	method := "getTurnLength"
+	toAddress := common.HexToAddress(systemcontracts.ValidatorContract)
+	gas := (hexutil.Uint64)(uint64(math.MaxUint64 / 2))
+
+	data, err := p.validatorSetABI.Pack(method)
+	if err != nil {
+		log.Error("Unable to pack tx for getTurnLength", "error", err)
+		return nil, err
+	}
+	msgData := (hexutil.Bytes)(data)
+
+	blockNr := rpc.BlockNumberOrHashWithHash(header.Hash(), false)
+	result, err := p.ethAPI.Call(ctx, ethapi.TransactionArgs{
+		Gas:  &gas,
+		To:   &toAddress,
+		Data: &msgData,
+	}, &blockNr, nil, nil)
+	if err != nil {
+		return nil, err
+	}
+
+	if err := p.validatorSetABI.UnpackIntoInterface(&turnLength, method, result); err != nil {
+		return nil, err
+	}
+
+	return turnLength, nil
+}
+
+// getRandTurnLength returns a random valid value, used to test switching turn length
+func (p *Parlia) getRandTurnLength(header *types.Header) (turnLength *big.Int, err error) {
+	turnLengths := [8]uint8{1, 3, 4, 5, 6, 7, 8, 9}
+	r := mrand.New(mrand.NewSource(int64(header.Time)))
+	lengthIndex := int(r.Int31n(int32(len(turnLengths))))
+	return big.NewInt(int64(turnLengths[lengthIndex])), nil
+}

consensus/parlia/parlia.go

@@ -6,6 +6,7 @@ import (
 	"encoding/hex"
 	"errors"
 	"fmt"
+	"io"
 	"math"
 	"math/big"
 	"math/rand"
@@ -53,11 +54,13 @@ const (
 	inMemoryHeaders    = 86400 // Number of recent headers to keep in memory for double sign detection,
 
 	checkpointInterval = 1024        // Number of blocks after which to save the snapshot to the database
-	defaultEpochLength = uint64(100) // Default number of blocks of checkpoint to update validatorSet from contract
+	defaultEpochLength = uint64(200) // Default number of blocks of checkpoint to update validatorSet from contract
+	defaultTurnLength  = uint8(1)    // Default consecutive number of blocks a validator receives priority for block production
 
 	extraVanity      = 32 // Fixed number of extra-data prefix bytes reserved for signer vanity
 	extraSeal        = 65 // Fixed number of extra-data suffix bytes reserved for signer seal
 	nextForkHashSize = 4  // Fixed number of extra-data suffix bytes reserved for nextForkHash.
+	turnLengthSize   = 1  // Fixed number of extra-data suffix bytes reserved for turnLength
 
 	validatorBytesLengthBeforeLuban = common.AddressLength
 	validatorBytesLength            = common.AddressLength + types.BLSPublicKeyLength
@@ -126,6 +129,10 @@ var (
 	// invalid list of validators (i.e. non divisible by 20 bytes).
 	errInvalidSpanValidators = errors.New("invalid validator list on sprint end block")
 
+	// errInvalidTurnLength is returned if a block contains an
+	// invalid length of turn (i.e. no data left after parsing validators).
+	errInvalidTurnLength = errors.New("invalid turnLength")
+
 	// errInvalidMixDigest is returned if a block's mix digest is non-zero.
 	errInvalidMixDigest = errors.New("non-zero mix digest")
 
@@ -136,6 +143,10 @@ var (
 	// list of validators different than the one the local node calculated.
 	errMismatchingEpochValidators = errors.New("mismatching validator list on epoch block")
 
+	// errMismatchingEpochTurnLength is returned if a sprint block contains a
+	// turn length different than the one the local node calculated.
+	errMismatchingEpochTurnLength = errors.New("mismatching turn length on epoch block")
+
 	// errInvalidDifficulty is returned if the difficulty of a block is missing.
 	errInvalidDifficulty = errors.New("invalid difficulty")
 
@@ -307,6 +318,10 @@ func New(
 	return c
 }
 
+func (p *Parlia) Period() uint64 {
+	return p.config.Period
+}
+
 func (p *Parlia) IsSystemTransaction(tx *types.Transaction, header *types.Header) (bool, error) {
 	// deploy a contract
 	if tx.To() == nil {
@@ -365,6 +380,7 @@ func (p *Parlia) VerifyHeaders(chain consensus.ChainHeaderReader, headers []*typ
 // On luban fork, we introduce vote attestation into the header's extra field, so extra format is different from before.
 // Before luban fork: |---Extra Vanity---|---Validators Bytes (or Empty)---|---Extra Seal---|
 // After luban fork:  |---Extra Vanity---|---Validators Number and Validators Bytes (or Empty)---|---Vote Attestation (or Empty)---|---Extra Seal---|
+// After bohr fork:   |---Extra Vanity---|---Validators Number and Validators Bytes (or Empty)---|---Turn Length (or Empty)---|---Vote Attestation (or Empty)---|---Extra Seal---|
 func getValidatorBytesFromHeader(header *types.Header, chainConfig *params.ChainConfig, parliaConfig *params.ParliaConfig) []byte {
 	if len(header.Extra) <= extraVanity+extraSeal {
 		return nil
@@ -381,11 +397,15 @@ func getValidatorBytesFromHeader(header *types.Header, chainConfig *params.Chain
 		return nil
 	}
 	num := int(header.Extra[extraVanity])
-	if num == 0 || len(header.Extra) <= extraVanity+extraSeal+num*validatorBytesLength {
-		return nil
-	}
 	start := extraVanity + validatorNumberSize
 	end := start + num*validatorBytesLength
+	extraMinLen := end + extraSeal
+	if chainConfig.IsBohr(header.Number, header.Time) {
+		extraMinLen += turnLengthSize
+	}
+	if num == 0 || len(header.Extra) < extraMinLen {
+		return nil
+	}
 	return header.Extra[start:end]
 }
 
@@ -404,11 +424,14 @@ func getVoteAttestationFromHeader(header *types.Header, chainConfig *params.Chai
 		attestationBytes = header.Extra[extraVanity : len(header.Extra)-extraSeal]
 	} else {
 		num := int(header.Extra[extraVanity])
-		if len(header.Extra) <= extraVanity+extraSeal+validatorNumberSize+num*validatorBytesLength {
+		start := extraVanity + validatorNumberSize + num*validatorBytesLength
+		if chainConfig.IsBohr(header.Number, header.Time) {
+			start += turnLengthSize
+		}
+		end := len(header.Extra) - extraSeal
+		if end <= start {
 			return nil, nil
 		}
-		start := extraVanity + validatorNumberSize + num*validatorBytesLength
-		end := len(header.Extra) - extraSeal
 		attestationBytes = header.Extra[start:end]
 	}
 
@@ -600,15 +623,11 @@ func (p *Parlia) verifyHeader(chain consensus.ChainHeaderReader, header *types.H
 			return fmt.Errorf("invalid excessBlobGas: have %d, expected nil", header.ExcessBlobGas)
 		case header.BlobGasUsed != nil:
 			return fmt.Errorf("invalid blobGasUsed: have %d, expected nil", header.BlobGasUsed)
-		case header.ParentBeaconRoot != nil:
-			return fmt.Errorf("invalid parentBeaconRoot, have %#x, expected nil", header.ParentBeaconRoot)
 		case header.WithdrawalsHash != nil:
 			return fmt.Errorf("invalid WithdrawalsHash, have %#x, expected nil", header.WithdrawalsHash)
 		}
 	} else {
 		switch {
-		case header.ParentBeaconRoot != nil:
-			return fmt.Errorf("invalid parentBeaconRoot, have %#x, expected nil", header.ParentBeaconRoot)
 		case !header.EmptyWithdrawalsHash():
 			return errors.New("header has wrong WithdrawalsHash")
 		}
@@ -617,6 +636,17 @@ func (p *Parlia) verifyHeader(chain consensus.ChainHeaderReader, header *types.H
 		}
 	}
 
+	bohr := chain.Config().IsBohr(header.Number, header.Time)
+	if !bohr {
+		if header.ParentBeaconRoot != nil {
+			return fmt.Errorf("invalid parentBeaconRoot, have %#x, expected nil", header.ParentBeaconRoot)
+		}
+	} else {
+		if header.ParentBeaconRoot == nil || *header.ParentBeaconRoot != (common.Hash{}) {
+			return fmt.Errorf("invalid parentBeaconRoot, have %#x, expected zero hash", header.ParentBeaconRoot)
+		}
+	}
+
 	// All basic checks passed, verify cascading fields
 	return p.verifyCascadingFields(chain, header, parents)
 }
@@ -709,13 +739,28 @@ func (p *Parlia) snapshot(chain consensus.ChainHeaderReader, number uint64, hash
 			}
 		}
 
-		// If we're at the genesis, snapshot the initial state.
-		if number == 0 {
-			checkpoint := chain.GetHeaderByNumber(number)
-			if checkpoint != nil {
-				// get checkpoint data
-				hash := checkpoint.Hash()
-
+		// If we're at the genesis, snapshot the initial state. Alternatively if we have
+		// piled up more headers than allowed to be reorged (chain reinit from a freezer),
+		// consider the checkpoint trusted and snapshot it.
+		// An offset `p.config.Epoch - 1` can ensure getting the right validators.
+		if number == 0 || ((number+1)%p.config.Epoch == 0 && (len(headers) > int(params.FullImmutabilityThreshold))) {
+			var (
+				checkpoint *types.Header
+				blockHash  common.Hash
+			)
+			if number == 0 {
+				checkpoint = chain.GetHeaderByNumber(0)
+				if checkpoint != nil {
+					blockHash = checkpoint.Hash()
+				}
+			} else {
+				checkpoint = chain.GetHeaderByNumber(number + 1 - p.config.Epoch)
+				blockHeader := chain.GetHeaderByNumber(number)
+				if blockHeader != nil {
+					blockHash = blockHeader.Hash()
+				}
+			}
+			if checkpoint != nil && blockHash != (common.Hash{}) {
 				// get validators from headers
 				validators, voteAddrs, err := parseValidators(checkpoint, p.chainConfig, p.config)
 				if err != nil {
@@ -723,11 +768,27 @@ func (p *Parlia) snapshot(chain consensus.ChainHeaderReader, number uint64, hash
 				}
 
 				// new snapshot
-				snap = newSnapshot(p.config, p.signatures, number, hash, validators, voteAddrs, p.ethAPI)
+				snap = newSnapshot(p.config, p.signatures, number, blockHash, validators, voteAddrs, p.ethAPI)
+
+				// get turnLength from headers and use that for new turnLength
+				turnLength, err := parseTurnLength(checkpoint, p.chainConfig, p.config)
+				if err != nil {
+					return nil, err
+				}
+				if turnLength != nil {
+					snap.TurnLength = *turnLength
+				}
+
+				// snap.Recents is currently empty, which affects the following:
+				// a. The function SignRecently - This is acceptable since an empty snap.Recents results in a more lenient check.
+				// b. The function blockTimeVerifyForRamanujanFork - This is also acceptable as it won't be invoked during `snap.apply`.
+				// c. This may cause a mismatch in the slash systemtx, but the transaction list is not verified during `snap.apply`.
+
+				// snap.Attestation is nil, but Snapshot.updateAttestation will handle it correctly.
 				if err := snap.store(p.db); err != nil {
 					return nil, err
 				}
-				log.Info("Stored checkpoint snapshot to disk", "number", number, "hash", hash)
+				log.Info("Stored checkpoint snapshot to disk", "number", number, "hash", blockHash)
 				break
 			}
 		}
@@ -884,6 +945,24 @@ func (p *Parlia) prepareValidators(header *types.Header) error {
 	return nil
 }
 
+func (p *Parlia) prepareTurnLength(chain consensus.ChainHeaderReader, header *types.Header) error {
+	if header.Number.Uint64()%p.config.Epoch != 0 ||
+		!p.chainConfig.IsBohr(header.Number, header.Time) {
+		return nil
+	}
+
+	turnLength, err := p.getTurnLength(chain, header)
+	if err != nil {
+		return err
+	}
+
+	if turnLength != nil {
+		header.Extra = append(header.Extra, *turnLength)
+	}
+
+	return nil
+}
+
 func (p *Parlia) assembleVoteAttestation(chain consensus.ChainHeaderReader, header *types.Header) error {
 	if !p.chainConfig.IsLuban(header.Number) || header.Number.Uint64() < 2 {
 		return nil
@@ -1015,6 +1094,9 @@ func (p *Parlia) Prepare(chain consensus.ChainHeaderReader, header *types.Header
 		return err
 	}
 
+	if err := p.prepareTurnLength(chain, header); err != nil {
+		return err
+	}
 	// add extra seal space
 	header.Extra = append(header.Extra, make([]byte, extraSeal)...)
 
@@ -1065,6 +1147,30 @@ func (p *Parlia) verifyValidators(header *types.Header) error {
 	return nil
 }
 
+func (p *Parlia) verifyTurnLength(chain consensus.ChainHeaderReader, header *types.Header) error {
+	if header.Number.Uint64()%p.config.Epoch != 0 ||
+		!p.chainConfig.IsBohr(header.Number, header.Time) {
+		return nil
+	}
+
+	turnLengthFromHeader, err := parseTurnLength(header, p.chainConfig, p.config)
+	if err != nil {
+		return err
+	}
+	if turnLengthFromHeader != nil {
+		turnLength, err := p.getTurnLength(chain, header)
+		if err != nil {
+			return err
+		}
+		if turnLength != nil && *turnLength == *turnLengthFromHeader {
+			log.Debug("verifyTurnLength", "turnLength", *turnLength)
+			return nil
+		}
+	}
+
+	return errMismatchingEpochTurnLength
+}
+
 func (p *Parlia) distributeFinalityReward(chain consensus.ChainHeaderReader, state *state.StateDB, header *types.Header,
 	cx core.ChainContext, txs *[]*types.Transaction, receipts *[]*types.Receipt, systemTxs *[]*types.Transaction,
 	usedGas *uint64, mining bool) error {
@@ -1159,6 +1265,10 @@ func (p *Parlia) Finalize(chain consensus.ChainHeaderReader, header *types.Heade
 		return err
 	}
 
+	if err := p.verifyTurnLength(chain, header); err != nil {
+		return err
+	}
+
 	cx := chainContext{Chain: chain, parlia: p}
 
 	parent := chain.GetHeaderByHash(header.ParentHash)
@@ -1185,7 +1295,7 @@ func (p *Parlia) Finalize(chain consensus.ChainHeaderReader, header *types.Heade
 		}
 	}
 	if header.Difficulty.Cmp(diffInTurn) != 0 {
-		spoiledVal := snap.supposeValidator()
+		spoiledVal := snap.inturnValidator()
 		signedRecently := false
 		if p.chainConfig.IsPlato(header.Number) {
 			signedRecently = snap.SignRecently(spoiledVal)
@@ -1276,7 +1386,7 @@ func (p *Parlia) FinalizeAndAssemble(chain consensus.ChainHeaderReader, header *
 		if err != nil {
 			return nil, nil, err
 		}
-		spoiledVal := snap.supposeValidator()
+		spoiledVal := snap.inturnValidator()
 		signedRecently := false
 		if p.chainConfig.IsPlato(header.Number) {
 			signedRecently = snap.SignRecently(spoiledVal)
@@ -1358,7 +1468,7 @@ func (p *Parlia) IsActiveValidatorAt(chain consensus.ChainHeaderReader, header *
 func (p *Parlia) VerifyVote(chain consensus.ChainHeaderReader, vote *types.VoteEnvelope) error {
 	targetNumber := vote.Data.TargetNumber
 	targetHash := vote.Data.TargetHash
-	header := chain.GetHeaderByHash(targetHash)
+	header := chain.GetVerifiedBlockByHash(targetHash)
 	if header == nil {
 		log.Warn("BlockHeader at current voteBlockNumber is nil", "targetNumber", targetNumber, "targetHash", targetHash)
 		return errors.New("BlockHeader at current voteBlockNumber is nil")
@@ -1429,10 +1539,13 @@ func (p *Parlia) Delay(chain consensus.ChainReader, header *types.Header, leftOv
 		delay = delay - *leftOver
 	}
 
-	// The blocking time should be no more than half of period
-	half := time.Duration(p.config.Period) * time.Second / 2
-	if delay > half {
-		delay = half
+	// The blocking time should be no more than half of period when snap.TurnLength == 1
+	timeForMining := time.Duration(p.config.Period) * time.Second / 2
+	if !snap.lastBlockInOneTurn(header.Number.Uint64()) {
+		timeForMining = time.Duration(p.config.Period) * time.Second * 2 / 3
+	}
+	if delay > timeForMining {
+		delay = timeForMining
 	}
 	return &delay
 }
@@ -1590,11 +1703,35 @@ func CalcDifficulty(snap *Snapshot, signer common.Address) *big.Int {
 	return new(big.Int).Set(diffNoTurn)
 }
 
+func encodeSigHeaderWithoutVoteAttestation(w io.Writer, header *types.Header, chainId *big.Int) {
+	err := rlp.Encode(w, []interface{}{
+		chainId,
+		header.ParentHash,
+		header.UncleHash,
+		header.Coinbase,
+		header.Root,
+		header.TxHash,
+		header.ReceiptHash,
+		header.Bloom,
+		header.Difficulty,
+		header.Number,
+		header.GasLimit,
+		header.GasUsed,
+		header.Time,
+		header.Extra[:extraVanity], // this will panic if extra is too short, should check before calling encodeSigHeaderWithoutVoteAttestation
+		header.MixDigest,
+		header.Nonce,
+	})
+	if err != nil {
+		panic("can't encode: " + err.Error())
+	}
+}
+
 // SealHash returns the hash of a block without vote attestation prior to it being sealed.
 // So it's not the real hash of a block, just used as unique id to distinguish task
 func (p *Parlia) SealHash(header *types.Header) (hash common.Hash) {
 	hasher := sha3.NewLegacyKeccak256()
-	types.EncodeSigHeaderWithoutVoteAttestation(hasher, header, p.chainConfig.ChainID)
+	encodeSigHeaderWithoutVoteAttestation(hasher, header, p.chainConfig.ChainID)
 	hasher.Sum(hash[:0])
 	return hash
 }
@@ -1824,7 +1961,7 @@ func (p *Parlia) applyTransaction(
 		// move to next
 		*receivedTxs = (*receivedTxs)[1:]
 	}
-	state.SetTxContext(expectedTx.Hash(), len(*txs), 0)
+	state.SetTxContext(expectedTx.Hash(), len(*txs))
 	gasUsed, err := applyMessage(msg, state, header, p.chainConfig, chainContext)
 	if err != nil {
 		return err
@@ -1900,42 +2037,40 @@ func (p *Parlia) GetFinalizedHeader(chain consensus.ChainHeaderReader, header *t
 // ===========================     utility function        ==========================
 func (p *Parlia) backOffTime(snap *Snapshot, header *types.Header, val common.Address) uint64 {
 	if snap.inturn(val) {
+		log.Debug("backOffTime", "blockNumber", header.Number, "in turn validator", val)
 		return 0
 	} else {
 		delay := initialBackOffTime
 		validators := snap.validators()
 		if p.chainConfig.IsPlanck(header.Number) {
-			// reverse the key/value of snap.Recents to get recentsMap
-			recentsMap := make(map[common.Address]uint64, len(snap.Recents))
-			bound := uint64(0)
-			if n, limit := header.Number.Uint64(), uint64(len(validators)/2+1); n > limit {
-				bound = n - limit
-			}
-			for seen, recent := range snap.Recents {
-				if seen <= bound {
-					continue
-				}
-				recentsMap[recent] = seen
+			counts := snap.countRecents()
+			for addr, seenTimes := range counts {
+				log.Debug("backOffTime", "blockNumber", header.Number, "validator", addr, "seenTimes", seenTimes)
 			}
 
 			// The backOffTime does not matter when a validator has signed recently.
-			if _, ok := recentsMap[val]; ok {
+			if snap.signRecentlyByCounts(val, counts) {
 				return 0
 			}
 
-			inTurnAddr := validators[(snap.Number+1)%uint64(len(validators))]
-			if _, ok := recentsMap[inTurnAddr]; ok {
+			inTurnAddr := snap.inturnValidator()
+			if snap.signRecentlyByCounts(inTurnAddr, counts) {
 				log.Debug("in turn validator has recently signed, skip initialBackOffTime",
 					"inTurnAddr", inTurnAddr)
 				delay = 0
 			}
 
-			// Exclude the recently signed validators
+			// Exclude the recently signed validators and the in turn validator
 			temp := make([]common.Address, 0, len(validators))
 			for _, addr := range validators {
-				if _, ok := recentsMap[addr]; ok {
+				if snap.signRecentlyByCounts(addr, counts) {
 					continue
 				}
+				if p.chainConfig.IsBohr(header.Number, header.Time) {
+					if addr == inTurnAddr {
+						continue
+					}
+				}
 				temp = append(temp, addr)
 			}
 			validators = temp
@@ -1953,7 +2088,11 @@ func (p *Parlia) backOffTime(snap *Snapshot, header *types.Header, val common.Ad
 			return 0
 		}
 
-		s := rand.NewSource(int64(snap.Number))
+		randSeed := snap.Number
+		if p.chainConfig.IsBohr(header.Number, header.Time) {
+			randSeed = header.Number.Uint64() / uint64(snap.TurnLength)
+		}
+		s := rand.NewSource(int64(randSeed))
 		r := rand.New(s)
 		n := len(validators)
 		backOffSteps := make([]uint64, 0, n)

consensus/parlia/parlia_test.go

@@ -22,22 +22,44 @@ func TestImpactOfValidatorOutOfService(t *testing.T) {
 	testCases := []struct {
 		totalValidators int
 		downValidators  int
+		turnLength      int
 	}{
-		{3, 1},
-		{5, 2},
-		{10, 1},
-		{10, 4},
-		{21, 1},
-		{21, 3},
-		{21, 5},
-		{21, 10},
+		{3, 1, 1},
+		{5, 2, 1},
+		{10, 1, 2},
+		{10, 4, 2},
+		{21, 1, 3},
+		{21, 3, 3},
+		{21, 5, 4},
+		{21, 10, 5},
 	}
 	for _, tc := range testCases {
-		simulateValidatorOutOfService(tc.totalValidators, tc.downValidators)
+		simulateValidatorOutOfService(tc.totalValidators, tc.downValidators, tc.turnLength)
 	}
 }
 
-func simulateValidatorOutOfService(totalValidators int, downValidators int) {
+// refer Snapshot.SignRecently
+func signRecently(idx int, recents map[uint64]int, turnLength int) bool {
+	recentSignTimes := 0
+	for _, signIdx := range recents {
+		if signIdx == idx {
+			recentSignTimes += 1
+		}
+	}
+	return recentSignTimes >= turnLength
+}
+
+// refer Snapshot.minerHistoryCheckLen
+func minerHistoryCheckLen(totalValidators int, turnLength int) uint64 {
+	return uint64(totalValidators/2+1)*uint64(turnLength) - 1
+}
+
+// refer Snapshot.inturnValidator
+func inturnValidator(totalValidators int, turnLength int, height int) int {
+	return height / turnLength % totalValidators
+}
+
+func simulateValidatorOutOfService(totalValidators int, downValidators int, turnLength int) {
 	downBlocks := 10000
 	recoverBlocks := 10000
 	recents := make(map[uint64]int)
@@ -55,12 +77,7 @@ func simulateValidatorOutOfService(totalValidators int, downValidators int) {
 		delete(validators, down[i])
 	}
 	isRecentSign := func(idx int) bool {
-		for _, signIdx := range recents {
-			if signIdx == idx {
-				return true
-			}
-		}
-		return false
+		return signRecently(idx, recents, turnLength)
 	}
 	isInService := func(idx int) bool {
 		return validators[idx]
@@ -68,10 +85,10 @@ func simulateValidatorOutOfService(totalValidators int, downValidators int) {
 
 	downDelay := uint64(0)
 	for h := 1; h <= downBlocks; h++ {
-		if limit := uint64(totalValidators/2 + 1); uint64(h) >= limit {
+		if limit := minerHistoryCheckLen(totalValidators, turnLength) + 1; uint64(h) >= limit {
 			delete(recents, uint64(h)-limit)
 		}
-		proposer := h % totalValidators
+		proposer := inturnValidator(totalValidators, turnLength, h)
 		if !isInService(proposer) || isRecentSign(proposer) {
 			candidates := make(map[int]bool, totalValidators/2)
 			for v := range validators {
@@ -99,10 +116,10 @@ func simulateValidatorOutOfService(totalValidators int, downValidators int) {
 	recoverDelay := uint64(0)
 	lastseen := downBlocks
 	for h := downBlocks + 1; h <= downBlocks+recoverBlocks; h++ {
-		if limit := uint64(totalValidators/2 + 1); uint64(h) >= limit {
+		if limit := minerHistoryCheckLen(totalValidators, turnLength) + 1; uint64(h) >= limit {
 			delete(recents, uint64(h)-limit)
 		}
-		proposer := h % totalValidators
+		proposer := inturnValidator(totalValidators, turnLength, h)
 		if !isInService(proposer) || isRecentSign(proposer) {
 			lastseen = h
 			candidates := make(map[int]bool, totalValidators/2)

consensus/parlia/snapshot.go

@@ -22,6 +22,7 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
+	"math"
 	"sort"
 
 	lru "github.com/hashicorp/golang-lru"
@@ -43,6 +44,7 @@ type Snapshot struct {
 
 	Number           uint64                            `json:"number"`                // Block number where the snapshot was created
 	Hash             common.Hash                       `json:"hash"`                  // Block hash where the snapshot was created
+	TurnLength       uint8                             `json:"turn_length"`           // Length of `turn`, meaning the consecutive number of blocks a validator receives priority for block production
 	Validators       map[common.Address]*ValidatorInfo `json:"validators"`            // Set of authorized validators at this moment
 	Recents          map[uint64]common.Address         `json:"recents"`               // Set of recent validators for spam protections
 	RecentForkHashes map[uint64]string                 `json:"recent_fork_hashes"`    // Set of recent forkHash
@@ -72,6 +74,7 @@ func newSnapshot(
 		sigCache:         sigCache,
 		Number:           number,
 		Hash:             hash,
+		TurnLength:       defaultTurnLength,
 		Recents:          make(map[uint64]common.Address),
 		RecentForkHashes: make(map[uint64]string),
 		Validators:       make(map[common.Address]*ValidatorInfo),
@@ -114,6 +117,10 @@ func loadSnapshot(config *params.ParliaConfig, sigCache *lru.ARCCache, db ethdb.
 	if err := json.Unmarshal(blob, snap); err != nil {
 		return nil, err
 	}
+	if snap.TurnLength == 0 { // no TurnLength field in old snapshots
+		snap.TurnLength = defaultTurnLength
+	}
+
 	snap.config = config
 	snap.sigCache = sigCache
 	snap.ethAPI = ethAPI
@@ -138,6 +145,7 @@ func (s *Snapshot) copy() *Snapshot {
 		sigCache:         s.sigCache,
 		Number:           s.Number,
 		Hash:             s.Hash,
+		TurnLength:       s.TurnLength,
 		Validators:       make(map[common.Address]*ValidatorInfo),
 		Recents:          make(map[uint64]common.Address),
 		RecentForkHashes: make(map[uint64]string),
@@ -210,17 +218,45 @@ func (s *Snapshot) updateAttestation(header *types.Header, chainConfig *params.C
 	}
 }
 
-func (s *Snapshot) SignRecently(validator common.Address) bool {
-	for seen, recent := range s.Recents {
-		if recent == validator {
-			if limit := uint64(len(s.Validators)/2 + 1); s.Number+1 < limit || seen > s.Number+1-limit {
-				return true
-			}
-		}
+func (s *Snapshot) versionHistoryCheckLen() uint64 {
+	return uint64(len(s.Validators)) * uint64(s.TurnLength)
+}
+
+func (s *Snapshot) minerHistoryCheckLen() uint64 {
+	return (uint64(len(s.Validators))/2+1)*uint64(s.TurnLength) - 1
+}
+
+func (s *Snapshot) countRecents() map[common.Address]uint8 {
+	leftHistoryBound := uint64(0) // the bound is excluded
+	checkHistoryLength := s.minerHistoryCheckLen()
+	if s.Number > checkHistoryLength {
+		leftHistoryBound = s.Number - checkHistoryLength
 	}
+	counts := make(map[common.Address]uint8, len(s.Validators))
+	for seen, recent := range s.Recents {
+		if seen <= leftHistoryBound || recent == (common.Address{}) /*when seen == `epochKey`*/ {
+			continue
+		}
+		counts[recent] += 1
+	}
+	return counts
+}
+
+func (s *Snapshot) signRecentlyByCounts(validator common.Address, counts map[common.Address]uint8) bool {
+	if seenTimes, ok := counts[validator]; ok && seenTimes >= s.TurnLength {
+		if seenTimes > s.TurnLength {
+			log.Warn("produce more blocks than expected!", "validator", validator, "seenTimes", seenTimes)
+		}
+		return true
+	}
+
 	return false
 }
 
+func (s *Snapshot) SignRecently(validator common.Address) bool {
+	return s.signRecentlyByCounts(validator, s.countRecents())
+}
+
 func (s *Snapshot) apply(headers []*types.Header, chain consensus.ChainHeaderReader, parents []*types.Header, chainConfig *params.ChainConfig) (*Snapshot, error) {
 	// Allow passing in no headers for cleaner code
 	if len(headers) == 0 {
@@ -247,10 +283,10 @@ func (s *Snapshot) apply(headers []*types.Header, chain consensus.ChainHeaderRea
 	for _, header := range headers {
 		number := header.Number.Uint64()
 		// Delete the oldest validator from the recent list to allow it signing again
-		if limit := uint64(len(snap.Validators)/2 + 1); number >= limit {
+		if limit := snap.minerHistoryCheckLen() + 1; number >= limit {
 			delete(snap.Recents, number-limit)
 		}
-		if limit := uint64(len(snap.Validators)); number >= limit {
+		if limit := snap.versionHistoryCheckLen(); number >= limit {
 			delete(snap.RecentForkHashes, number-limit)
 		}
 		// Resolve the authorization key and check against signers
@@ -261,19 +297,47 @@ func (s *Snapshot) apply(headers []*types.Header, chain consensus.ChainHeaderRea
 		if _, ok := snap.Validators[validator]; !ok {
 			return nil, errUnauthorizedValidator(validator.String())
 		}
-		for _, recent := range snap.Recents {
-			if recent == validator {
+		if chainConfig.IsBohr(header.Number, header.Time) {
+			if snap.SignRecently(validator) {
 				return nil, errRecentlySigned
 			}
+		} else {
+			for _, recent := range snap.Recents {
+				if recent == validator {
+					return nil, errRecentlySigned
+				}
+			}
 		}
 		snap.Recents[number] = validator
+		snap.RecentForkHashes[number] = hex.EncodeToString(header.Extra[extraVanity-nextForkHashSize : extraVanity])
+		snap.updateAttestation(header, chainConfig, s.config)
 		// change validator set
-		if number > 0 && number%s.config.Epoch == uint64(len(snap.Validators)/2) {
-			checkpointHeader := FindAncientHeader(header, uint64(len(snap.Validators)/2), chain, parents)
+		if number > 0 && number%s.config.Epoch == snap.minerHistoryCheckLen() {
+			epochKey := math.MaxUint64 - header.Number.Uint64()/s.config.Epoch // impossible used as a block number
+			if chainConfig.IsBohr(header.Number, header.Time) {
+				// after switching the validator set, snap.Validators may become larger,
+				// then the unexpected second switch will happen, just skip it.
+				if _, ok := snap.Recents[epochKey]; ok {
+					continue
+				}
+			}
+
+			checkpointHeader := FindAncientHeader(header, snap.minerHistoryCheckLen(), chain, parents)
 			if checkpointHeader == nil {
 				return nil, consensus.ErrUnknownAncestor
 			}
 
+			oldVersionsLen := snap.versionHistoryCheckLen()
+			// get turnLength from headers and use that for new turnLength
+			turnLength, err := parseTurnLength(checkpointHeader, chainConfig, s.config)
+			if err != nil {
+				return nil, err
+			}
+			if turnLength != nil {
+				snap.TurnLength = *turnLength
+				log.Debug("validator set switch", "turnLength", *turnLength)
+			}
+
 			// get validators from headers and use that for new validator set
 			newValArr, voteAddrs, err := parseValidators(checkpointHeader, chainConfig, s.config)
 			if err != nil {
@@ -289,18 +353,18 @@ func (s *Snapshot) apply(headers []*types.Header, chain consensus.ChainHeaderRea
 					}
 				}
 			}
-			oldLimit := len(snap.Validators)/2 + 1
-			newLimit := len(newVals)/2 + 1
-			if newLimit < oldLimit {
-				for i := 0; i < oldLimit-newLimit; i++ {
-					delete(snap.Recents, number-uint64(newLimit)-uint64(i))
-				}
-			}
-			oldLimit = len(snap.Validators)
-			newLimit = len(newVals)
-			if newLimit < oldLimit {
-				for i := 0; i < oldLimit-newLimit; i++ {
-					delete(snap.RecentForkHashes, number-uint64(newLimit)-uint64(i))
+			if chainConfig.IsBohr(header.Number, header.Time) {
+				// BEP-404: Clear Miner History when Switching Validators Set
+				snap.Recents = make(map[uint64]common.Address)
+				snap.Recents[epochKey] = common.Address{}
+				log.Debug("Recents are cleared up", "blockNumber", number)
+			} else {
+				oldLimit := len(snap.Validators)/2 + 1
+				newLimit := len(newVals)/2 + 1
+				if newLimit < oldLimit {
+					for i := 0; i < oldLimit-newLimit; i++ {
+						delete(snap.Recents, number-uint64(newLimit)-uint64(i))
+					}
 				}
 			}
 			snap.Validators = newVals
@@ -310,11 +374,10 @@ func (s *Snapshot) apply(headers []*types.Header, chain consensus.ChainHeaderRea
 					snap.Validators[val].Index = idx + 1 // offset by 1
 				}
 			}
+			for i := snap.versionHistoryCheckLen(); i < oldVersionsLen; i++ {
+				delete(snap.RecentForkHashes, number-i)
+			}
 		}
-
-		snap.updateAttestation(header, chainConfig, s.config)
-
-		snap.RecentForkHashes[number] = hex.EncodeToString(header.Extra[extraVanity-nextForkHashSize : extraVanity])
 	}
 	snap.Number += uint64(len(headers))
 	snap.Hash = headers[len(headers)-1].Hash()
@@ -331,17 +394,20 @@ func (s *Snapshot) validators() []common.Address {
 	return validators
 }
 
-// inturn returns if a validator at a given block height is in-turn or not.
-func (s *Snapshot) inturn(validator common.Address) bool {
-	validators := s.validators()
-	offset := (s.Number + 1) % uint64(len(validators))
-	return validators[offset] == validator
+// lastBlockInOneTurn returns if the block at height `blockNumber` is the last block in current turn.
+func (s *Snapshot) lastBlockInOneTurn(blockNumber uint64) bool {
+	return (blockNumber+1)%uint64(s.TurnLength) == 0
 }
 
-// inturnValidator returns the validator at a given block height.
+// inturn returns if a validator at a given block height is in-turn or not.
+func (s *Snapshot) inturn(validator common.Address) bool {
+	return s.inturnValidator() == validator
+}
+
+// inturnValidator returns the validator for the following block height.
 func (s *Snapshot) inturnValidator() common.Address {
 	validators := s.validators()
-	offset := (s.Number + 1) % uint64(len(validators))
+	offset := (s.Number + 1) / uint64(s.TurnLength) % uint64(len(validators))
 	return validators[offset]
 }
 
@@ -379,12 +445,6 @@ func (s *Snapshot) indexOfVal(validator common.Address) int {
 	return -1
 }
 
-func (s *Snapshot) supposeValidator() common.Address {
-	validators := s.validators()
-	index := (s.Number + 1) % uint64(len(validators))
-	return validators[index]
-}
-
 func parseValidators(header *types.Header, chainConfig *params.ChainConfig, parliaConfig *params.ParliaConfig) ([]common.Address, []types.BLSPublicKey, error) {
 	validatorsBytes := getValidatorBytesFromHeader(header, chainConfig, parliaConfig)
 	if len(validatorsBytes) == 0 {
@@ -410,6 +470,24 @@ func parseValidators(header *types.Header, chainConfig *params.ChainConfig, parl
 	return cnsAddrs, voteAddrs, nil
 }
 
+func parseTurnLength(header *types.Header, chainConfig *params.ChainConfig, parliaConfig *params.ParliaConfig) (*uint8, error) {
+	if header.Number.Uint64()%parliaConfig.Epoch != 0 ||
+		!chainConfig.IsBohr(header.Number, header.Time) {
+		return nil, nil
+	}
+
+	if len(header.Extra) <= extraVanity+extraSeal {
+		return nil, errInvalidSpanValidators
+	}
+	num := int(header.Extra[extraVanity])
+	pos := extraVanity + validatorNumberSize + num*validatorBytesLength
+	if len(header.Extra) <= pos {
+		return nil, errInvalidTurnLength
+	}
+	turnLength := header.Extra[pos]
+	return &turnLength, nil
+}
+
 func FindAncientHeader(header *types.Header, ite uint64, chain consensus.ChainHeaderReader, candidateParents []*types.Header) *types.Header {
 	ancient := header
 	for i := uint64(1); i <= ite; i++ {

core/blockchain.go

@@ -259,23 +259,25 @@ type BlockChain struct {
 	triesInMemory uint64
 	txIndexer     *txIndexer // Transaction indexer, might be nil if not enabled
 
-	hc                  *HeaderChain
-	rmLogsFeed          event.Feed
-	chainFeed           event.Feed
-	chainSideFeed       event.Feed
-	chainHeadFeed       event.Feed
-	chainBlockFeed      event.Feed
-	logsFeed            event.Feed
-	blockProcFeed       event.Feed
-	finalizedHeaderFeed event.Feed
-	scope               event.SubscriptionScope
-	genesisBlock        *types.Block
+	hc                       *HeaderChain
+	rmLogsFeed               event.Feed
+	chainFeed                event.Feed
+	chainSideFeed            event.Feed
+	chainHeadFeed            event.Feed
+	chainBlockFeed           event.Feed
+	logsFeed                 event.Feed
+	blockProcFeed            event.Feed
+	finalizedHeaderFeed      event.Feed
+	highestVerifiedBlockFeed event.Feed
+	scope                    event.SubscriptionScope
+	genesisBlock             *types.Block
 
 	// This mutex synchronizes chain write operations.
 	// Readers don't need to take it, they can just read the database.
 	chainmu *syncx.ClosableMutex
 
 	highestVerifiedHeader atomic.Pointer[types.Header]
+	highestVerifiedBlock  atomic.Pointer[types.Header]
 	currentBlock          atomic.Pointer[types.Header] // Current head of the chain
 	currentSnapBlock      atomic.Pointer[types.Header] // Current head of snap-sync
 	currentFinalBlock     atomic.Pointer[types.Header] // Latest (consensus) finalized block
@@ -301,6 +303,7 @@ type BlockChain struct {
 	diffLayerFreezerBlockLimit uint64
 
 	wg            sync.WaitGroup
+	dbWg          sync.WaitGroup
 	quit          chan struct{} // shutdown signal, closed in Stop.
 	stopping      atomic.Bool   // false if chain is running, true when stopped
 	procInterrupt atomic.Bool   // interrupt signaler for block processing
@@ -399,6 +402,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
 	}
 
 	bc.highestVerifiedHeader.Store(nil)
+	bc.highestVerifiedBlock.Store(nil)
 	bc.currentBlock.Store(nil)
 	bc.currentSnapBlock.Store(nil)
 	bc.chasingHead.Store(nil)
@@ -461,8 +465,8 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
 		}
 	}
 	// Ensure that a previous crash in SetHead doesn't leave extra ancients
-	if frozen, err := bc.db.ItemAmountInAncient(); err == nil && frozen > 0 {
-		frozen, err = bc.db.Ancients()
+	if frozen, err := bc.db.BlockStore().ItemAmountInAncient(); err == nil && frozen > 0 {
+		frozen, err = bc.db.BlockStore().Ancients()
 		if err != nil {
 			return nil, err
 		}
@@ -656,20 +660,13 @@ func (bc *BlockChain) cacheDiffLayer(diffLayer *types.DiffLayer, diffLayerCh cha
 	}
 }
 
-func (bc *BlockChain) cacheBlock(hash common.Hash, block *types.Block) {
-	bc.blockCache.Add(hash, block)
-	if bc.chainConfig.IsCancun(block.Number(), block.Time()) {
-		bc.sidecarsCache.Add(hash, block.Sidecars())
-	}
-}
-
 // empty returns an indicator whether the blockchain is empty.
 // Note, it's a special case that we connect a non-empty ancient
 // database with an empty node, so that we can plugin the ancient
 // into node seamlessly.
 func (bc *BlockChain) empty() bool {
 	genesis := bc.genesisBlock.Hash()
-	for _, hash := range []common.Hash{rawdb.ReadHeadBlockHash(bc.db.BlockStore()), rawdb.ReadHeadHeaderHash(bc.db.BlockStore()), rawdb.ReadHeadFastBlockHash(bc.db)} {
+	for _, hash := range []common.Hash{rawdb.ReadHeadBlockHash(bc.db), rawdb.ReadHeadHeaderHash(bc.db), rawdb.ReadHeadFastBlockHash(bc.db)} {
 		if hash != genesis {
 			return false
 		}
@@ -705,7 +702,7 @@ func (bc *BlockChain) getFinalizedNumber(header *types.Header) uint64 {
 // assumes that the chain manager mutex is held.
 func (bc *BlockChain) loadLastState() error {
 	// Restore the last known head block
-	head := rawdb.ReadHeadBlockHash(bc.db.BlockStore())
+	head := rawdb.ReadHeadBlockHash(bc.db)
 	if head == (common.Hash{}) {
 		// Corrupt or empty database, init from scratch
 		log.Warn("Empty database, resetting chain")
@@ -727,7 +724,7 @@ func (bc *BlockChain) loadLastState() error {
 
 	// Restore the last known head header
 	headHeader := headBlock.Header()
-	if head := rawdb.ReadHeadHeaderHash(bc.db.BlockStore()); head != (common.Hash{}) {
+	if head := rawdb.ReadHeadHeaderHash(bc.db); head != (common.Hash{}) {
 		if header := bc.GetHeaderByHash(head); header != nil {
 			headHeader = header
 		}
@@ -1106,7 +1103,7 @@ func (bc *BlockChain) setHeadBeyondRoot(head uint64, time uint64, root common.Ha
 		// intent afterwards is full block importing, delete the chain segment
 		// between the stateful-block and the sethead target.
 		var wipe bool
-		frozen, _ := bc.db.Ancients()
+		frozen, _ := bc.db.BlockStore().Ancients()
 		if headNumber+1 < frozen {
 			wipe = pivot == nil || headNumber >= *pivot
 		}
@@ -1115,11 +1112,11 @@ func (bc *BlockChain) setHeadBeyondRoot(head uint64, time uint64, root common.Ha
 	// Rewind the header chain, deleting all block bodies until then
 	delFn := func(db ethdb.KeyValueWriter, hash common.Hash, num uint64) {
 		// Ignore the error here since light client won't hit this path
-		frozen, _ := bc.db.Ancients()
+		frozen, _ := bc.db.BlockStore().Ancients()
 		if num+1 <= frozen {
 			// Truncate all relative data(header, total difficulty, body, receipt
 			// and canonical hash) from ancient store.
-			if _, err := bc.db.TruncateHead(num); err != nil {
+			if _, err := bc.db.BlockStore().TruncateHead(num); err != nil {
 				log.Crit("Failed to truncate ancient data", "number", num, "err", err)
 			}
 			// Remove the hash <-> number mapping from the active store.
@@ -1137,7 +1134,7 @@ func (bc *BlockChain) setHeadBeyondRoot(head uint64, time uint64, root common.Ha
 	// If SetHead was only called as a chain reparation method, try to skip
 	// touching the header chain altogether, unless the freezer is broken
 	if repair {
-		if target, force := updateFn(bc.db, bc.CurrentBlock()); force {
+		if target, force := updateFn(bc.db.BlockStore(), bc.CurrentBlock()); force {
 			bc.hc.SetHead(target.Number.Uint64(), updateFn, delFn)
 		}
 	} else {
@@ -1298,19 +1295,33 @@ func (bc *BlockChain) ExportN(w io.Writer, first uint64, last uint64) error {
 //
 // Note, this function assumes that the `mu` mutex is held!
 func (bc *BlockChain) writeHeadBlock(block *types.Block) {
-	// Add the block to the canonical chain number scheme and mark as the head
-	rawdb.WriteCanonicalHash(bc.db.BlockStore(), block.Hash(), block.NumberU64())
-	rawdb.WriteHeadHeaderHash(bc.db.BlockStore(), block.Hash())
-	rawdb.WriteHeadBlockHash(bc.db.BlockStore(), block.Hash())
+	bc.dbWg.Add(2)
+	defer bc.dbWg.Wait()
+	go func() {
+		defer bc.dbWg.Done()
+		// Add the block to the canonical chain number scheme and mark as the head
+		blockBatch := bc.db.BlockStore().NewBatch()
+		rawdb.WriteCanonicalHash(blockBatch, block.Hash(), block.NumberU64())
+		rawdb.WriteHeadHeaderHash(blockBatch, block.Hash())
+		rawdb.WriteHeadBlockHash(blockBatch, block.Hash())
+		rawdb.WriteHeadFastBlockHash(blockBatch, block.Hash())
+		// Flush the whole batch into the disk, exit the node if failed
+		if err := blockBatch.Write(); err != nil {
+			log.Crit("Failed to update chain indexes and markers in block db", "err", err)
+		}
+	}()
+	go func() {
+		defer bc.dbWg.Done()
 
-	batch := bc.db.NewBatch()
-	rawdb.WriteHeadFastBlockHash(batch, block.Hash())
-	rawdb.WriteTxLookupEntriesByBlock(batch, block)
+		batch := bc.db.NewBatch()
+		rawdb.WriteTxLookupEntriesByBlock(batch, block)
+
+		// Flush the whole batch into the disk, exit the node if failed
+		if err := batch.Write(); err != nil {
+			log.Crit("Failed to update chain indexes in chain db", "err", err)
+		}
+	}()
 
-	// Flush the whole batch into the disk, exit the node if failed
-	if err := batch.Write(); err != nil {
-		log.Crit("Failed to update chain indexes and markers", "err", err)
-	}
 	// Update all in-memory chain markers in the last step
 	bc.hc.SetCurrentHeader(block.Header())
 
@@ -1382,7 +1393,7 @@ func (bc *BlockChain) Stop() {
 		if !bc.cacheConfig.TrieDirtyDisabled {
 			triedb := bc.triedb
 			var once sync.Once
-			for _, offset := range []uint64{0, 1, TriesInMemory - 1} {
+			for _, offset := range []uint64{0, 1, bc.TriesInMemory() - 1} {
 				if number := bc.CurrentBlock().Number.Uint64(); number > offset {
 					recent := bc.GetBlockByNumber(number - offset)
 					log.Info("Writing cached state to disk", "block", recent.Number(), "hash", recent.Hash(), "root", recent.Root())
@@ -1531,7 +1542,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
 			} else if !reorg {
 				return false
 			}
-			rawdb.WriteHeadFastBlockHash(bc.db, head.Hash())
+			rawdb.WriteHeadFastBlockHash(bc.db.BlockStore(), head.Hash())
 			bc.currentSnapBlock.Store(head.Header())
 			headFastBlockGauge.Update(int64(head.NumberU64()))
 			return true
@@ -1548,9 +1559,9 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
 
 		// Ensure genesis is in ancients.
 		if first.NumberU64() == 1 {
-			if frozen, _ := bc.db.Ancients(); frozen == 0 {
+			if frozen, _ := bc.db.BlockStore().Ancients(); frozen == 0 {
 				td := bc.genesisBlock.Difficulty()
-				writeSize, err := rawdb.WriteAncientBlocks(bc.db, []*types.Block{bc.genesisBlock}, []types.Receipts{nil}, td)
+				writeSize, err := rawdb.WriteAncientBlocks(bc.db.BlockStore(), []*types.Block{bc.genesisBlock}, []types.Receipts{nil}, td)
 				if err != nil {
 					log.Error("Error writing genesis to ancients", "err", err)
 					return 0, err
@@ -1568,7 +1579,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
 
 		// Write all chain data to ancients.
 		td := bc.GetTd(first.Hash(), first.NumberU64())
-		writeSize, err := rawdb.WriteAncientBlocksWithBlobs(bc.db, blockChain, receiptChain, td)
+		writeSize, err := rawdb.WriteAncientBlocksWithBlobs(bc.db.BlockStore(), blockChain, receiptChain, td)
 		if err != nil {
 			log.Error("Error importing chain data to ancients", "err", err)
 			return 0, err
@@ -1576,7 +1587,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
 		size += writeSize
 
 		// Sync the ancient store explicitly to ensure all data has been flushed to disk.
-		if err := bc.db.Sync(); err != nil {
+		if err := bc.db.BlockStore().Sync(); err != nil {
 			return 0, err
 		}
 		// Update the current snap block because all block data is now present in DB.
@@ -1584,7 +1595,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
 		if !updateHead(blockChain[len(blockChain)-1]) {
 			// We end up here if the header chain has reorg'ed, and the blocks/receipts
 			// don't match the canonical chain.
-			if _, err := bc.db.TruncateHead(previousSnapBlock + 1); err != nil {
+			if _, err := bc.db.BlockStore().TruncateHead(previousSnapBlock + 1); err != nil {
 				log.Error("Can't truncate ancient store after failed insert", "err", err)
 			}
 			return 0, errSideChainReceipts
@@ -1604,7 +1615,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
 			rawdb.DeleteBlockWithoutNumber(blockBatch, block.Hash(), block.NumberU64())
 		}
 		// Delete side chain hash-to-number mappings.
-		for _, nh := range rawdb.ReadAllHashesInRange(bc.db, first.NumberU64(), last.NumberU64()) {
+		for _, nh := range rawdb.ReadAllHashesInRange(bc.db.BlockStore(), first.NumberU64(), last.NumberU64()) {
 			if _, canon := canonHashes[nh.Hash]; !canon {
 				rawdb.DeleteHeader(blockBatch, nh.Hash, nh.Number)
 			}
@@ -1774,7 +1785,6 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
 	wg := sync.WaitGroup{}
 	wg.Add(1)
 	go func() {
-		rawdb.WritePreimages(bc.db, state.Preimages())
 		blockBatch := bc.db.BlockStore().NewBatch()
 		rawdb.WriteTd(blockBatch, block.Hash(), block.NumberU64(), externTd)
 		rawdb.WriteBlock(blockBatch, block)
@@ -1783,10 +1793,20 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
 		if bc.chainConfig.IsCancun(block.Number(), block.Time()) {
 			rawdb.WriteBlobSidecars(blockBatch, block.Hash(), block.NumberU64(), block.Sidecars())
 		}
-		rawdb.WritePreimages(blockBatch, state.Preimages())
+		if bc.db.StateStore() != nil {
+			rawdb.WritePreimages(bc.db.StateStore(), state.Preimages())
+		} else {
+			rawdb.WritePreimages(blockBatch, state.Preimages())
+		}
 		if err := blockBatch.Write(); err != nil {
 			log.Crit("Failed to write block into disk", "err", err)
 		}
+		bc.hc.tdCache.Add(block.Hash(), externTd)
+		bc.blockCache.Add(block.Hash(), block)
+		bc.cacheReceipts(block.Hash(), receipts, block)
+		if bc.chainConfig.IsCancun(block.Number(), block.Time()) {
+			bc.sidecarsCache.Add(block.Hash(), block.Sidecars())
+		}
 		wg.Done()
 	}()
 
@@ -1811,7 +1831,7 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
 
 		// Flush limits are not considered for the first TriesInMemory blocks.
 		current := block.NumberU64()
-		if current <= TriesInMemory {
+		if current <= bc.TriesInMemory() {
 			return nil
 		}
 		// If we exceeded our memory allowance, flush matured singleton nodes to disk
@@ -1909,14 +1929,19 @@ func (bc *BlockChain) WriteBlockAndSetHead(block *types.Block, receipts []*types
 // writeBlockAndSetHead is the internal implementation of WriteBlockAndSetHead.
 // This function expects the chain mutex to be held.
 func (bc *BlockChain) writeBlockAndSetHead(block *types.Block, receipts []*types.Receipt, logs []*types.Log, state *state.StateDB, emitHeadEvent bool) (status WriteStatus, err error) {
-	if err := bc.writeBlockWithState(block, receipts, state); err != nil {
-		return NonStatTy, err
-	}
 	currentBlock := bc.CurrentBlock()
 	reorg, err := bc.forker.ReorgNeededWithFastFinality(currentBlock, block.Header())
 	if err != nil {
 		return NonStatTy, err
 	}
+	if reorg {
+		bc.highestVerifiedBlock.Store(types.CopyHeader(block.Header()))
+		bc.highestVerifiedBlockFeed.Send(HighestVerifiedBlockEvent{Header: block.Header()})
+	}
+
+	if err := bc.writeBlockWithState(block, receipts, state); err != nil {
+		return NonStatTy, err
+	}
 	if reorg {
 		// Reorganise the chain if the parent is not the head block
 		if block.ParentHash() != currentBlock.Hash() {
@@ -2264,8 +2289,6 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
 		vtime := time.Since(vstart)
 		proctime := time.Since(start) // processing + validation
 
-		bc.cacheBlock(block.Hash(), block)
-
 		// Update the metrics touched during block processing and validation
 		accountReadTimer.Update(statedb.AccountReads)                   // Account reads are complete(in processing)
 		storageReadTimer.Update(statedb.StorageReads)                   // Storage reads are complete(in processing)
@@ -2297,8 +2320,6 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
 			return it.index, err
 		}
 
-		bc.cacheReceipts(block.Hash(), receipts, block)
-
 		// Update the metrics touched during block commit
 		accountCommitTimer.Update(statedb.AccountCommits)   // Account commits are complete, we can mark them
 		storageCommitTimer.Update(statedb.StorageCommits)   // Storage commits are complete, we can mark them
@@ -2307,6 +2328,7 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
 
 		blockWriteTimer.Update(time.Since(wstart) - statedb.AccountCommits - statedb.StorageCommits - statedb.SnapshotCommits - statedb.TrieDBCommits)
 		blockInsertTimer.UpdateSince(start)
+
 		// Report the import stats before returning the various results
 		stats.processed++
 		stats.usedGas += usedGas
@@ -2377,26 +2399,11 @@ func (bc *BlockChain) updateHighestVerifiedHeader(header *types.Header) {
 	if header == nil || header.Number == nil {
 		return
 	}
-	currentHeader := bc.highestVerifiedHeader.Load()
-	if currentHeader == nil {
+	currentBlock := bc.CurrentBlock()
+	reorg, err := bc.forker.ReorgNeededWithFastFinality(currentBlock, header)
+	if err == nil && reorg {
 		bc.highestVerifiedHeader.Store(types.CopyHeader(header))
-		return
-	}
-
-	newParentTD := bc.GetTd(header.ParentHash, header.Number.Uint64()-1)
-	if newParentTD == nil {
-		newParentTD = big.NewInt(0)
-	}
-	oldParentTD := bc.GetTd(currentHeader.ParentHash, currentHeader.Number.Uint64()-1)
-	if oldParentTD == nil {
-		oldParentTD = big.NewInt(0)
-	}
-	newTD := big.NewInt(0).Add(newParentTD, header.Difficulty)
-	oldTD := big.NewInt(0).Add(oldParentTD, currentHeader.Difficulty)
-
-	if newTD.Cmp(oldTD) > 0 {
-		bc.highestVerifiedHeader.Store(types.CopyHeader(header))
-		return
+		log.Trace("updateHighestVerifiedHeader", "number", header.Number.Uint64(), "hash", header.Hash())
 	}
 }
 

core/blockchain_reader.go

@@ -98,6 +98,15 @@ func (bc *BlockChain) GetHeaderByHash(hash common.Hash) *types.Header {
 	return bc.hc.GetHeaderByHash(hash)
 }
 
+// GetVerifiedBlockByHash retrieves the header of a verified block, it may be only in memory.
+func (bc *BlockChain) GetVerifiedBlockByHash(hash common.Hash) *types.Header {
+	highestVerifiedBlock := bc.highestVerifiedBlock.Load()
+	if highestVerifiedBlock != nil && highestVerifiedBlock.Hash() == hash {
+		return highestVerifiedBlock
+	}
+	return bc.hc.GetHeaderByHash(hash)
+}
+
 // GetHeaderByNumber retrieves a block header from the database by number,
 // caching it (associated with its hash) if found.
 func (bc *BlockChain) GetHeaderByNumber(number uint64) *types.Header {
@@ -231,7 +240,7 @@ func (bc *BlockChain) GetReceiptsByHash(hash common.Hash) types.Receipts {
 	if receipts, ok := bc.receiptsCache.Get(hash); ok {
 		return receipts
 	}
-	number := rawdb.ReadHeaderNumber(bc.db.BlockStore(), hash)
+	number := rawdb.ReadHeaderNumber(bc.db, hash)
 	if number == nil {
 		return nil
 	}
@@ -486,6 +495,11 @@ func (bc *BlockChain) SubscribeChainHeadEvent(ch chan<- ChainHeadEvent) event.Su
 	return bc.scope.Track(bc.chainHeadFeed.Subscribe(ch))
 }
 
+// SubscribeHighestVerifiedBlockEvent registers a subscription of HighestVerifiedBlockEvent.
+func (bc *BlockChain) SubscribeHighestVerifiedHeaderEvent(ch chan<- HighestVerifiedBlockEvent) event.Subscription {
+	return bc.scope.Track(bc.highestVerifiedBlockFeed.Subscribe(ch))
+}
+
 // SubscribeChainBlockEvent registers a subscription of ChainBlockEvent.
 func (bc *BlockChain) SubscribeChainBlockEvent(ch chan<- ChainHeadEvent) event.Subscription {
 	return bc.scope.Track(bc.chainBlockFeed.Subscribe(ch))
@@ -514,7 +528,7 @@ func (bc *BlockChain) SubscribeFinalizedHeaderEvent(ch chan<- FinalizedHeaderEve
 
 // AncientTail retrieves the tail the ancients blocks
 func (bc *BlockChain) AncientTail() (uint64, error) {
-	tail, err := bc.db.Tail()
+	tail, err := bc.db.BlockStore().Tail()
 	if err != nil {
 		return 0, err
 	}

core/blockchain_repair_test.go

@@ -26,6 +26,8 @@ import (
 	"testing"
 	"time"
 
+	"github.com/ethereum/go-ethereum/ethdb"
+
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/consensus/ethash"
 	"github.com/ethereum/go-ethereum/core/rawdb"
@@ -1795,6 +1797,13 @@ func testRepairWithScheme(t *testing.T, tt *rewindTest, snapshots bool, scheme s
 		config.SnapshotWait = true
 	}
 	config.TriesInMemory = 128
+
+	if err = db.SetupFreezerEnv(&ethdb.FreezerEnv{
+		ChainCfg:         gspec.Config,
+		BlobExtraReserve: params.DefaultExtraReserveForBlobRequests,
+	}); err != nil {
+		t.Fatalf("Failed to create chain: %v", err)
+	}
 	chain, err := NewBlockChain(db, config, gspec, nil, engine, vm.Config{}, nil, nil)
 	if err != nil {
 		t.Fatalf("Failed to create chain: %v", err)

core/blockchain_sethead_test.go

@@ -27,6 +27,8 @@ import (
 	"testing"
 	"time"
 
+	"github.com/ethereum/go-ethereum/ethdb"
+
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/consensus/ethash"
 	"github.com/ethereum/go-ethereum/core/rawdb"
@@ -1998,6 +2000,13 @@ func testSetHeadWithScheme(t *testing.T, tt *rewindTest, snapshots bool, scheme
 		config.SnapshotWait = true
 	}
 	config.TriesInMemory = 128
+
+	if err = db.SetupFreezerEnv(&ethdb.FreezerEnv{
+		ChainCfg:         gspec.Config,
+		BlobExtraReserve: params.DefaultExtraReserveForBlobRequests,
+	}); err != nil {
+		t.Fatalf("Failed to create chain: %v", err)
+	}
 	chain, err := NewBlockChain(db, config, gspec, nil, engine, vm.Config{}, nil, nil)
 	if err != nil {
 		t.Fatalf("Failed to create chain: %v", err)

core/chain_indexer.go

@@ -227,8 +227,8 @@ func (c *ChainIndexer) eventLoop(currentHeader *types.Header, events chan ChainH
 				// Reorg to the common ancestor if needed (might not exist in light sync mode, skip reorg then)
 				// TODO(karalabe, zsfelfoldi): This seems a bit brittle, can we detect this case explicitly?
 
-				if rawdb.ReadCanonicalHash(c.chainDb.BlockStore(), prevHeader.Number.Uint64()) != prevHash {
-					if h := rawdb.FindCommonAncestor(c.chainDb.BlockStore(), prevHeader, header); h != nil {
+				if rawdb.ReadCanonicalHash(c.chainDb, prevHeader.Number.Uint64()) != prevHash {
+					if h := rawdb.FindCommonAncestor(c.chainDb, prevHeader, header); h != nil {
 						c.newHead(h.Number.Uint64(), true)
 					}
 				}

core/chain_makers.go

@@ -119,7 +119,7 @@ func (b *BlockGen) addTx(bc *BlockChain, vmConfig vm.Config, tx *types.Transacti
 	if b.gasPool == nil {
 		b.SetCoinbase(common.Address{})
 	}
-	b.statedb.SetTxContext(tx.Hash(), len(b.txs), 0)
+	b.statedb.SetTxContext(tx.Hash(), len(b.txs))
 	receipt, err := ApplyTransaction(b.cm.config, bc, &b.header.Coinbase, b.gasPool, b.statedb, b.header, tx, &b.header.GasUsed, vmConfig, NewReceiptBloomGenerator())
 	if err != nil {
 		panic(err)
@@ -486,7 +486,7 @@ func (cm *chainMaker) makeHeader(parent *types.Block, state *state.StateDB, engi
 		if cm.config.Parlia != nil {
 			header.WithdrawalsHash = &types.EmptyWithdrawalsHash
 		}
-		if cm.config.Parlia == nil {
+		if cm.config.Parlia == nil || cm.config.IsBohr(header.Number, header.Time) {
 			header.ParentBeaconRoot = new(common.Hash)
 		}
 	}
@@ -621,6 +621,10 @@ func (cm *chainMaker) GetHighestVerifiedHeader() *types.Header {
 	panic("not supported")
 }
 
+func (cm *chainMaker) GetVerifiedBlockByHash(hash common.Hash) *types.Header {
+	return cm.GetHeaderByHash(hash)
+}
+
 func (cm *chainMaker) ChasingHead() *types.Header {
 	panic("not supported")
 }

core/data_availability_test.go

@@ -365,6 +365,10 @@ func (r *mockDAHeaderReader) GetHighestVerifiedHeader() *types.Header {
 	panic("not supported")
 }
 
+func (r *mockDAHeaderReader) GetVerifiedBlockByHash(hash common.Hash) *types.Header {
+	panic("not supported")
+}
+
 func createMockDATx(config *params.ChainConfig, sidecar *types.BlobTxSidecar) *types.Transaction {
 	if sidecar == nil {
 		tx := &types.DynamicFeeTx{

core/events.go

@@ -50,3 +50,5 @@ type ChainSideEvent struct {
 }
 
 type ChainHeadEvent struct{ Block *types.Block }
+
+type HighestVerifiedBlockEvent struct{ Header *types.Header }

core/forkchoice.go

@@ -86,9 +86,16 @@ func (f *ForkChoice) ReorgNeeded(current *types.Header, extern *types.Header) (b
 		localTD  = f.chain.GetTd(current.Hash(), current.Number.Uint64())
 		externTd = f.chain.GetTd(extern.Hash(), extern.Number.Uint64())
 	)
-	if localTD == nil || externTd == nil {
+	if localTD == nil {
 		return false, errors.New("missing td")
 	}
+	if externTd == nil {
+		ptd := f.chain.GetTd(extern.ParentHash, extern.Number.Uint64()-1)
+		if ptd == nil {
+			return false, consensus.ErrUnknownAncestor
+		}
+		externTd = new(big.Int).Add(ptd, extern.Difficulty)
+	}
 	// Accept the new header as the chain head if the transition
 	// is already triggered. We assume all the headers after the
 	// transition come from the trusted consensus layer.
@@ -114,7 +121,12 @@ func (f *ForkChoice) ReorgNeeded(current *types.Header, extern *types.Header) (b
 		if f.preserve != nil {
 			currentPreserve, externPreserve = f.preserve(current), f.preserve(extern)
 		}
-		reorg = !currentPreserve && (externPreserve || f.rand.Float64() < 0.5)
+		doubleSign := (extern.Coinbase == current.Coinbase)
+		reorg = !currentPreserve && (externPreserve ||
+			extern.Time < current.Time ||
+			extern.Time == current.Time &&
+				((doubleSign && extern.Hash().Cmp(current.Hash()) < 0) ||
+					(!doubleSign && f.rand.Float64() < 0.5)))
 	}
 	return reorg, nil
 }

core/genesis.go

@@ -216,10 +216,9 @@ func (e *GenesisMismatchError) Error() string {
 // ChainOverrides contains the changes to chain config
 // Typically, these modifications involve hardforks that are not enabled on the BSC mainnet, intended for testing purposes.
 type ChainOverrides struct {
-	OverrideCancun *uint64
-	OverrideHaber  *uint64
-	OverrideBohr   *uint64
-	OverrideVerkle *uint64
+	OverridePassedForkTime *uint64
+	OverrideBohr           *uint64
+	OverrideVerkle         *uint64
 }
 
 // SetupGenesisBlock writes or updates the genesis block in db.
@@ -245,11 +244,14 @@ func SetupGenesisBlockWithOverride(db ethdb.Database, triedb *triedb.Database, g
 	}
 	applyOverrides := func(config *params.ChainConfig) {
 		if config != nil {
-			if overrides != nil && overrides.OverrideCancun != nil {
-				config.CancunTime = overrides.OverrideCancun
-			}
-			if overrides != nil && overrides.OverrideHaber != nil {
-				config.HaberTime = overrides.OverrideHaber
+			if overrides != nil && overrides.OverridePassedForkTime != nil {
+				config.ShanghaiTime = overrides.OverridePassedForkTime
+				config.KeplerTime = overrides.OverridePassedForkTime
+				config.FeynmanTime = overrides.OverridePassedForkTime
+				config.FeynmanFixTime = overrides.OverridePassedForkTime
+				config.CancunTime = overrides.OverridePassedForkTime
+				config.HaberTime = overrides.OverridePassedForkTime
+				config.HaberFixTime = overrides.OverridePassedForkTime
 			}
 			if overrides != nil && overrides.OverrideBohr != nil {
 				config.BohrTime = overrides.OverrideBohr
@@ -452,7 +454,7 @@ func (g *Genesis) ToBlock() *types.Block {
 			// EIP-4788: The parentBeaconBlockRoot of the genesis block is always
 			// the zero hash. This is because the genesis block does not have a parent
 			// by definition.
-			if conf.Parlia == nil {
+			if conf.Parlia == nil || conf.IsBohr(num, g.Timestamp) {
 				head.ParentBeaconRoot = new(common.Hash)
 			}
 
@@ -498,7 +500,7 @@ func (g *Genesis) Commit(db ethdb.Database, triedb *triedb.Database) (*types.Blo
 	rawdb.WriteReceipts(db.BlockStore(), block.Hash(), block.NumberU64(), nil)
 	rawdb.WriteCanonicalHash(db.BlockStore(), block.Hash(), block.NumberU64())
 	rawdb.WriteHeadBlockHash(db.BlockStore(), block.Hash())
-	rawdb.WriteHeadFastBlockHash(db, block.Hash())
+	rawdb.WriteHeadFastBlockHash(db.BlockStore(), block.Hash())
 	rawdb.WriteHeadHeaderHash(db.BlockStore(), block.Hash())
 	rawdb.WriteChainConfig(db, block.Hash(), config)
 	return block, nil

core/headerchain.go

@@ -97,7 +97,7 @@ func NewHeaderChain(chainDb ethdb.Database, config *params.ChainConfig, engine c
 		return nil, ErrNoGenesis
 	}
 	hc.currentHeader.Store(hc.genesisHeader)
-	if head := rawdb.ReadHeadBlockHash(chainDb.BlockStore()); head != (common.Hash{}) {
+	if head := rawdb.ReadHeadBlockHash(chainDb); head != (common.Hash{}) {
 		if chead := hc.GetHeaderByHash(head); chead != nil {
 			hc.currentHeader.Store(chead)
 		}
@@ -144,7 +144,7 @@ func (hc *HeaderChain) GetBlockNumber(hash common.Hash) *uint64 {
 	if cached, ok := hc.numberCache.Get(hash); ok {
 		return &cached
 	}
-	number := rawdb.ReadHeaderNumber(hc.chainDb.BlockStore(), hash)
+	number := rawdb.ReadHeaderNumber(hc.chainDb, hash)
 	if number != nil {
 		hc.numberCache.Add(hash, *number)
 	}
@@ -436,6 +436,10 @@ func (hc *HeaderChain) GetHighestVerifiedHeader() *types.Header {
 	return nil
 }
 
+func (hc *HeaderChain) GetVerifiedBlockByHash(hash common.Hash) *types.Header {
+	return hc.GetHeaderByHash(hash)
+}
+
 func (hc *HeaderChain) ChasingHead() *types.Header {
 	return nil
 }
@@ -668,7 +672,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
 		// first then remove the relative data from the database.
 		//
 		// Update head first(head fast block, head full block) before deleting the data.
-		markerBatch := hc.chainDb.NewBatch()
+		markerBatch := hc.chainDb.BlockStore().NewBatch()
 		if updateFn != nil {
 			newHead, force := updateFn(markerBatch, parent)
 			if force && ((headTime > 0 && newHead.Time < headTime) || (headTime == 0 && newHead.Number.Uint64() < headBlock)) {
@@ -677,7 +681,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
 			}
 		}
 		// Update head header then.
-		rawdb.WriteHeadHeaderHash(hc.chainDb.BlockStore(), parentHash)
+		rawdb.WriteHeadHeaderHash(markerBatch, parentHash)
 		if err := markerBatch.Write(); err != nil {
 			log.Crit("Failed to update chain markers", "error", err)
 		}
@@ -691,7 +695,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
 		// we don't end up with dangling daps in the database
 		var nums []uint64
 		if origin {
-			for n := num + 1; len(rawdb.ReadAllHashes(hc.chainDb, n)) > 0; n++ {
+			for n := num + 1; len(rawdb.ReadAllHashes(hc.chainDb.BlockStore(), n)) > 0; n++ {
 				nums = append([]uint64{n}, nums...) // suboptimal, but we don't really expect this path
 			}
 			origin = false
@@ -701,7 +705,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
 		// Remove the related data from the database on all sidechains
 		for _, num := range nums {
 			// Gather all the side fork hashes
-			hashes := rawdb.ReadAllHashes(hc.chainDb, num)
+			hashes := rawdb.ReadAllHashes(hc.chainDb.BlockStore(), num)
 			if len(hashes) == 0 {
 				// No hashes in the database whatsoever, probably frozen already
 				hashes = append(hashes, hdr.Hash())

core/rawdb/accessors_chain.go

@@ -34,6 +34,15 @@ import (
 	"golang.org/x/exp/slices"
 )
 
+// Support Multi-Database Based on Data Pattern, the Chaindata will be divided into three stores: BlockStore, StateStore, and ChainStore,
+// according to data schema and read/write behavior. When using the following data interfaces, you should take note of the following:
+//
+// 1) Block-Related Data: For CanonicalHash, Header, Body, Td, Receipts, and BlobSidecars, the Write, Delete, and Iterator
+// 	operations should carefully ensure that the database being used is BlockStore.
+// 2) Meta-Related Data: For HeaderNumber, HeadHeaderHash, HeadBlockHash, HeadFastBlockHash, and FinalizedBlockHash, the
+// 	Write and Delete operations should carefully ensure that the database being used is BlockStore.
+// 3) Ancient Data: When using a multi-database, Ancient data will use the BlockStore.
+
 // ReadCanonicalHash retrieves the hash assigned to a canonical block number.
 func ReadCanonicalHash(db ethdb.Reader, number uint64) common.Hash {
 	var data []byte
@@ -144,8 +153,8 @@ func ReadAllCanonicalHashes(db ethdb.Iteratee, from uint64, to uint64, limit int
 }
 
 // ReadHeaderNumber returns the header number assigned to a hash.
-func ReadHeaderNumber(db ethdb.KeyValueReader, hash common.Hash) *uint64 {
-	data, _ := db.Get(headerNumberKey(hash))
+func ReadHeaderNumber(db ethdb.MultiDatabaseReader, hash common.Hash) *uint64 {
+	data, _ := db.BlockStoreReader().Get(headerNumberKey(hash))
 	if len(data) != 8 {
 		return nil
 	}
@@ -170,8 +179,8 @@ func DeleteHeaderNumber(db ethdb.KeyValueWriter, hash common.Hash) {
 }
 
 // ReadHeadHeaderHash retrieves the hash of the current canonical head header.
-func ReadHeadHeaderHash(db ethdb.KeyValueReader) common.Hash {
-	data, _ := db.Get(headHeaderKey)
+func ReadHeadHeaderHash(db ethdb.MultiDatabaseReader) common.Hash {
+	data, _ := db.BlockStoreReader().Get(headHeaderKey)
 	if len(data) == 0 {
 		return common.Hash{}
 	}
@@ -186,8 +195,8 @@ func WriteHeadHeaderHash(db ethdb.KeyValueWriter, hash common.Hash) {
 }
 
 // ReadHeadBlockHash retrieves the hash of the current canonical head block.
-func ReadHeadBlockHash(db ethdb.KeyValueReader) common.Hash {
-	data, _ := db.Get(headBlockKey)
+func ReadHeadBlockHash(db ethdb.MultiDatabaseReader) common.Hash {
+	data, _ := db.BlockStoreReader().Get(headBlockKey)
 	if len(data) == 0 {
 		return common.Hash{}
 	}
@@ -202,8 +211,8 @@ func WriteHeadBlockHash(db ethdb.KeyValueWriter, hash common.Hash) {
 }
 
 // ReadHeadFastBlockHash retrieves the hash of the current fast-sync head block.
-func ReadHeadFastBlockHash(db ethdb.KeyValueReader) common.Hash {
-	data, _ := db.Get(headFastBlockKey)
+func ReadHeadFastBlockHash(db ethdb.MultiDatabaseReader) common.Hash {
+	data, _ := db.BlockStoreReader().Get(headFastBlockKey)
 	if len(data) == 0 {
 		return common.Hash{}
 	}
@@ -218,8 +227,8 @@ func WriteHeadFastBlockHash(db ethdb.KeyValueWriter, hash common.Hash) {
 }
 
 // ReadFinalizedBlockHash retrieves the hash of the finalized block.
-func ReadFinalizedBlockHash(db ethdb.KeyValueReader) common.Hash {
-	data, _ := db.Get(headFinalizedBlockKey)
+func ReadFinalizedBlockHash(db ethdb.MultiDatabaseReader) common.Hash {
+	data, _ := db.BlockStoreReader().Get(headFinalizedBlockKey)
 	if len(data) == 0 {
 		return common.Hash{}
 	}
@@ -297,7 +306,7 @@ func ReadHeaderRange(db ethdb.Reader, number uint64, count uint64) []rlp.RawValu
 		// It's ok to request block 0, 1 item
 		count = number + 1
 	}
-	limit, _ := db.Ancients()
+	limit, _ := db.BlockStoreReader().Ancients()
 	// First read live blocks
 	if i >= limit {
 		// If we need to read live blocks, we need to figure out the hash first
@@ -317,7 +326,7 @@ func ReadHeaderRange(db ethdb.Reader, number uint64, count uint64) []rlp.RawValu
 		return rlpHeaders
 	}
 	// read remaining from ancients, cap at 2M
-	data, err := db.AncientRange(ChainFreezerHeaderTable, i+1-count, count, 2*1024*1024)
+	data, err := db.BlockStoreReader().AncientRange(ChainFreezerHeaderTable, i+1-count, count, 2*1024*1024)
 	if err != nil {
 		log.Error("Failed to read headers from freezer", "err", err)
 		return rlpHeaders
@@ -468,7 +477,7 @@ func ReadCanonicalBodyRLP(db ethdb.Reader, number uint64) rlp.RawValue {
 		// Block is not in ancients, read from leveldb by hash and number.
 		// Note: ReadCanonicalHash cannot be used here because it also
 		// calls ReadAncients internally.
-		hash, _ := db.Get(headerHashKey(number))
+		hash, _ := db.BlockStoreReader().Get(headerHashKey(number))
 		data, _ = db.BlockStoreReader().Get(blockBodyKey(number, common.BytesToHash(hash)))
 		return nil
 	})
@@ -516,6 +525,13 @@ func WriteBody(db ethdb.KeyValueWriter, hash common.Hash, number uint64, body *t
 	WriteBodyRLP(db, hash, number, data)
 }
 
+// DeleteBody removes all block body data associated with a hash.
+func DeleteBody(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
+	if err := db.Delete(blockBodyKey(number, hash)); err != nil {
+		log.Crit("Failed to delete block body", "err", err)
+	}
+}
+
 func WriteDiffLayer(db ethdb.KeyValueWriter, hash common.Hash, layer *types.DiffLayer) {
 	data, err := rlp.EncodeToBytes(layer)
 	if err != nil {
@@ -554,13 +570,6 @@ func DeleteDiffLayer(db ethdb.KeyValueWriter, blockHash common.Hash) {
 	}
 }
 
-// DeleteBody removes all block body data associated with a hash.
-func DeleteBody(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
-	if err := db.Delete(blockBodyKey(number, hash)); err != nil {
-		log.Crit("Failed to delete block body", "err", err)
-	}
-}
-
 // ReadTdRLP retrieves a block's total difficulty corresponding to the hash in RLP encoding.
 func ReadTdRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue {
 	var data []byte
@@ -884,7 +893,7 @@ func WriteAncientBlocks(db ethdb.AncientWriter, blocks []*types.Block, receipts
 // ReadBlobSidecarsRLP retrieves all the transaction blobs belonging to a block in RLP encoding.
 func ReadBlobSidecarsRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue {
 	var data []byte
-	db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
+	db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
 		// Check if the data is in ancients
 		if isCanon(reader, number, hash) {
 			data, _ = reader.Ancient(ChainFreezerBlobSidecarTable, number)
@@ -1093,24 +1102,24 @@ func FindCommonAncestor(db ethdb.Reader, a, b *types.Header) *types.Header {
 
 // ReadHeadHeader returns the current canonical head header.
 func ReadHeadHeader(db ethdb.Reader) *types.Header {
-	headHeaderHash := ReadHeadHeaderHash(db.BlockStoreReader())
+	headHeaderHash := ReadHeadHeaderHash(db)
 	if headHeaderHash == (common.Hash{}) {
 		return nil
 	}
-	headHeaderNumber := ReadHeaderNumber(db.BlockStoreReader(), headHeaderHash)
+	headHeaderNumber := ReadHeaderNumber(db, headHeaderHash)
 	if headHeaderNumber == nil {
 		return nil
 	}
-	return ReadHeader(db.BlockStoreReader(), headHeaderHash, *headHeaderNumber)
+	return ReadHeader(db, headHeaderHash, *headHeaderNumber)
 }
 
 // ReadHeadBlock returns the current canonical head block.
 func ReadHeadBlock(db ethdb.Reader) *types.Block {
-	headBlockHash := ReadHeadBlockHash(db.BlockStoreReader())
+	headBlockHash := ReadHeadBlockHash(db)
 	if headBlockHash == (common.Hash{}) {
 		return nil
 	}
-	headBlockNumber := ReadHeaderNumber(db.BlockStoreReader(), headBlockHash)
+	headBlockNumber := ReadHeaderNumber(db, headBlockHash)
 	if headBlockNumber == nil {
 		return nil
 	}

core/rawdb/accessors_indexes.go

@@ -42,7 +42,7 @@ func ReadTxLookupEntry(db ethdb.Reader, hash common.Hash) *uint64 {
 	}
 	// Database v4-v5 tx lookup format just stores the hash
 	if len(data) == common.HashLength {
-		return ReadHeaderNumber(db.BlockStoreReader(), common.BytesToHash(data))
+		return ReadHeaderNumber(db, common.BytesToHash(data))
 	}
 	// Finally try database v3 tx lookup format
 	var entry LegacyTxLookupEntry

core/rawdb/ancient_utils.go

@@ -18,6 +18,8 @@ package rawdb
 
 import (
 	"fmt"
+	"io"
+	"os"
 	"path/filepath"
 
 	"github.com/ethereum/go-ethereum/common"
@@ -98,6 +100,18 @@ func inspectFreezers(db ethdb.Database) ([]freezerInfo, error) {
 			if err != nil {
 				return nil, err
 			}
+
+			file, err := os.Open(filepath.Join(datadir, StateFreezerName))
+			if err != nil {
+				return nil, err
+			}
+			defer file.Close()
+			// if state freezer folder has been pruned, there is no need for inspection
+			_, err = file.Readdirnames(1)
+			if err == io.EOF {
+				continue
+			}
+
 			f, err := NewStateFreezer(datadir, true, 0)
 			if err != nil {
 				return nil, err

core/rawdb/chain_freezer.go

@@ -58,7 +58,8 @@ type chainFreezer struct {
 	wg      sync.WaitGroup
 	trigger chan chan struct{} // Manual blocking freeze trigger, test determinism
 
-	freezeEnv atomic.Value
+	freezeEnv    atomic.Value
+	waitEnvTimes int
 
 	multiDatabase bool
 }
@@ -91,7 +92,7 @@ func (f *chainFreezer) Close() error {
 
 // readHeadNumber returns the number of chain head block. 0 is returned if the
 // block is unknown or not available yet.
-func (f *chainFreezer) readHeadNumber(db ethdb.KeyValueReader) uint64 {
+func (f *chainFreezer) readHeadNumber(db ethdb.Reader) uint64 {
 	hash := ReadHeadBlockHash(db)
 	if hash == (common.Hash{}) {
 		log.Error("Head block is not reachable")
@@ -107,7 +108,7 @@ func (f *chainFreezer) readHeadNumber(db ethdb.KeyValueReader) uint64 {
 
 // readFinalizedNumber returns the number of finalized block. 0 is returned
 // if the block is unknown or not available yet.
-func (f *chainFreezer) readFinalizedNumber(db ethdb.KeyValueReader) uint64 {
+func (f *chainFreezer) readFinalizedNumber(db ethdb.Reader) uint64 {
 	hash := ReadFinalizedBlockHash(db)
 	if hash == (common.Hash{}) {
 		return 0
@@ -122,7 +123,7 @@ func (f *chainFreezer) readFinalizedNumber(db ethdb.KeyValueReader) uint64 {
 
 // freezeThreshold returns the threshold for chain freezing. It's determined
 // by formula: max(finality, HEAD-params.FullImmutabilityThreshold).
-func (f *chainFreezer) freezeThreshold(db ethdb.KeyValueReader) (uint64, error) {
+func (f *chainFreezer) freezeThreshold(db ethdb.Reader) (uint64, error) {
 	var (
 		head      = f.readHeadNumber(db)
 		final     = f.readFinalizedNumber(db)
@@ -178,19 +179,6 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
 			}
 		}
 
-		// check freezer env first, it must wait a while when the env is necessary
-		err := f.checkFreezerEnv()
-		if err == missFreezerEnvErr {
-			log.Warn("Freezer need related env, may wait for a while", "err", err)
-			backoff = true
-			continue
-		}
-		if err != nil {
-			log.Error("Freezer check FreezerEnv err", "err", err)
-			backoff = true
-			continue
-		}
-
 		var (
 			frozen    uint64
 			threshold uint64
@@ -200,6 +188,7 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
 			hash   common.Hash
 			number *uint64
 			head   *types.Header
+			err    error
 		)
 
 		// use finalized block as the chain freeze indicator was used for multiDatabase feature, if multiDatabase is false, keep 9W blocks in db
@@ -282,6 +271,18 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
 				last = first + freezerBatchLimit
 			}
 		}
+
+		// check env first before chain freeze, it must wait when the env is necessary
+		if err := f.checkFreezerEnv(); err != nil {
+			f.waitEnvTimes++
+			if f.waitEnvTimes%30 == 0 {
+				log.Warn("Freezer need related env, may wait for a while, and it's not a issue when non-import block", "err", err)
+				return
+			}
+			backoff = true
+			continue
+		}
+
 		// Seems we have data ready to be frozen, process in usable batches
 		var (
 			start = time.Now()
@@ -544,14 +545,7 @@ func (f *chainFreezer) checkFreezerEnv() error {
 	if exist {
 		return nil
 	}
-	blobFrozen, err := f.TableAncients(ChainFreezerBlobSidecarTable)
-	if err != nil {
-		return err
-	}
-	if blobFrozen > 0 {
-		return missFreezerEnvErr
-	}
-	return nil
+	return missFreezerEnvErr
 }
 
 func isCancun(env *ethdb.FreezerEnv, num *big.Int, time uint64) bool {

core/rawdb/chain_iterator.go

@@ -35,16 +35,16 @@ import (
 // injects into the database the block hash->number mappings.
 func InitDatabaseFromFreezer(db ethdb.Database) {
 	// If we can't access the freezer or it's empty, abort
-	frozen, err := db.ItemAmountInAncient()
+	frozen, err := db.BlockStore().ItemAmountInAncient()
 	if err != nil || frozen == 0 {
 		return
 	}
 	var (
-		batch  = db.NewBatch()
+		batch  = db.BlockStore().NewBatch()
 		start  = time.Now()
 		logged = start.Add(-7 * time.Second) // Unindex during import is fast, don't double log
 		hash   common.Hash
-		offset = db.AncientOffSet()
+		offset = db.BlockStore().AncientOffSet()
 	)
 	for i := uint64(0) + offset; i < frozen+offset; i++ {
 		// We read 100K hashes at a time, for a total of 3.2M
@@ -52,7 +52,7 @@ func InitDatabaseFromFreezer(db ethdb.Database) {
 		if i+count > frozen+offset {
 			count = frozen + offset - i
 		}
-		data, err := db.AncientRange(ChainFreezerHashTable, i, count, 32*count)
+		data, err := db.BlockStore().AncientRange(ChainFreezerHashTable, i, count, 32*count)
 		if err != nil {
 			log.Crit("Failed to init database from freezer", "err", err)
 		}
@@ -81,7 +81,7 @@ func InitDatabaseFromFreezer(db ethdb.Database) {
 	batch.Reset()
 
 	WriteHeadHeaderHash(db.BlockStore(), hash)
-	WriteHeadFastBlockHash(db, hash)
+	WriteHeadFastBlockHash(db.BlockStore(), hash)
 	log.Info("Initialized database from freezer", "blocks", frozen, "elapsed", common.PrettyDuration(time.Since(start)))
 }
 
@@ -100,7 +100,7 @@ func iterateTransactions(db ethdb.Database, from uint64, to uint64, reverse bool
 		number uint64
 		rlp    rlp.RawValue
 	}
-	if offset := db.AncientOffSet(); offset > from {
+	if offset := db.BlockStore().AncientOffSet(); offset > from {
 		from = offset
 	}
 	if to <= from {
@@ -122,7 +122,7 @@ func iterateTransactions(db ethdb.Database, from uint64, to uint64, reverse bool
 		}
 		defer close(rlpCh)
 		for n != end {
-			data := ReadCanonicalBodyRLP(db.BlockStore(), n)
+			data := ReadCanonicalBodyRLP(db, n)
 			// Feed the block to the aggregator, or abort on interrupt
 			select {
 			case rlpCh <- &numberRlp{n, data}:
@@ -187,7 +187,7 @@ func iterateTransactions(db ethdb.Database, from uint64, to uint64, reverse bool
 // signal received.
 func indexTransactions(db ethdb.Database, from uint64, to uint64, interrupt chan struct{}, hook func(uint64) bool, report bool) {
 	// short circuit for invalid range
-	if offset := db.AncientOffSet(); offset > from {
+	if offset := db.BlockStore().AncientOffSet(); offset > from {
 		from = offset
 	}
 	if from >= to {
@@ -286,7 +286,7 @@ func indexTransactionsForTesting(db ethdb.Database, from uint64, to uint64, inte
 // signal received.
 func unindexTransactions(db ethdb.Database, from uint64, to uint64, interrupt chan struct{}, hook func(uint64) bool, report bool) {
 	// short circuit for invalid range
-	if offset := db.AncientOffSet(); offset > from {
+	if offset := db.BlockStore().AncientOffSet(); offset > from {
 		from = offset
 	}
 	if from >= to {

core/rawdb/database.go

@@ -61,8 +61,10 @@ func (frdb *freezerdb) BlockStoreReader() ethdb.Reader {
 }
 
 func (frdb *freezerdb) BlockStoreWriter() ethdb.Writer {
-	//TODO implement me
-	panic("implement me")
+	if frdb.blockStore == nil {
+		return frdb
+	}
+	return frdb.blockStore
 }
 
 // AncientDatadir returns the path of root ancient directory.
@@ -116,6 +118,13 @@ func (frdb *freezerdb) StateStore() ethdb.Database {
 	return frdb.stateStore
 }
 
+func (frdb *freezerdb) GetStateStore() ethdb.Database {
+	if frdb.stateStore != nil {
+		return frdb.stateStore
+	}
+	return frdb
+}
+
 func (frdb *freezerdb) SetStateStore(state ethdb.Database) {
 	if frdb.stateStore != nil {
 		frdb.stateStore.Close()
@@ -193,7 +202,7 @@ func (db *nofreezedb) Ancients() (uint64, error) {
 	return 0, errNotSupported
 }
 
-// Ancients returns an error as we don't have a backing chain freezer.
+// ItemAmountInAncient returns an error as we don't have a backing chain freezer.
 func (db *nofreezedb) ItemAmountInAncient() (uint64, error) {
 	return 0, errNotSupported
 }
@@ -254,6 +263,13 @@ func (db *nofreezedb) SetStateStore(state ethdb.Database) {
 	db.stateStore = state
 }
 
+func (db *nofreezedb) GetStateStore() ethdb.Database {
+	if db.stateStore != nil {
+		return db.stateStore
+	}
+	return db
+}
+
 func (db *nofreezedb) StateStoreReader() ethdb.Reader {
 	if db.stateStore != nil {
 		return db.stateStore
@@ -331,6 +347,111 @@ func NewDatabase(db ethdb.KeyValueStore) ethdb.Database {
 	return &nofreezedb{KeyValueStore: db}
 }
 
+type emptyfreezedb struct {
+	ethdb.KeyValueStore
+}
+
+// HasAncient returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) HasAncient(kind string, number uint64) (bool, error) {
+	return false, nil
+}
+
+// Ancient returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) Ancient(kind string, number uint64) ([]byte, error) {
+	return nil, nil
+}
+
+// AncientRange returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) AncientRange(kind string, start, max, maxByteSize uint64) ([][]byte, error) {
+	return nil, nil
+}
+
+// Ancients returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) Ancients() (uint64, error) {
+	return 0, nil
+}
+
+// ItemAmountInAncient returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) ItemAmountInAncient() (uint64, error) {
+	return 0, nil
+}
+
+// Tail returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) Tail() (uint64, error) {
+	return 0, nil
+}
+
+// AncientSize returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) AncientSize(kind string) (uint64, error) {
+	return 0, nil
+}
+
+// ModifyAncients returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) ModifyAncients(func(ethdb.AncientWriteOp) error) (int64, error) {
+	return 0, nil
+}
+
+// TruncateHead returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) TruncateHead(items uint64) (uint64, error) {
+	return 0, nil
+}
+
+// TruncateTail returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) TruncateTail(items uint64) (uint64, error) {
+	return 0, nil
+}
+
+// TruncateTableTail returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) TruncateTableTail(kind string, tail uint64) (uint64, error) {
+	return 0, nil
+}
+
+// ResetTable returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) ResetTable(kind string, startAt uint64, onlyEmpty bool) error {
+	return nil
+}
+
+// Sync returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) Sync() error {
+	return nil
+}
+
+func (db *emptyfreezedb) DiffStore() ethdb.KeyValueStore        { return db }
+func (db *emptyfreezedb) SetDiffStore(diff ethdb.KeyValueStore) {}
+func (db *emptyfreezedb) StateStore() ethdb.Database            { return db }
+func (db *emptyfreezedb) GetStateStore() ethdb.Database         { return db }
+func (db *emptyfreezedb) SetStateStore(state ethdb.Database)    {}
+func (db *emptyfreezedb) StateStoreReader() ethdb.Reader        { return db }
+func (db *emptyfreezedb) BlockStore() ethdb.Database            { return db }
+func (db *emptyfreezedb) SetBlockStore(block ethdb.Database)    {}
+func (db *emptyfreezedb) HasSeparateBlockStore() bool           { return false }
+func (db *emptyfreezedb) BlockStoreReader() ethdb.Reader        { return db }
+func (db *emptyfreezedb) BlockStoreWriter() ethdb.Writer        { return db }
+func (db *emptyfreezedb) ReadAncients(fn func(reader ethdb.AncientReaderOp) error) (err error) {
+	return nil
+}
+func (db *emptyfreezedb) AncientOffSet() uint64 { return 0 }
+
+// MigrateTable returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) MigrateTable(kind string, convert convertLegacyFn) error {
+	return nil
+}
+
+// AncientDatadir returns nil for pruned db that we don't have a backing chain freezer.
+func (db *emptyfreezedb) AncientDatadir() (string, error) {
+	return "", nil
+}
+func (db *emptyfreezedb) SetupFreezerEnv(env *ethdb.FreezerEnv) error {
+	return nil
+}
+
+// NewEmptyFreezeDB is used for CLI such as `geth db inspect` in pruned db that we don't
+// have a backing chain freezer.
+// WARNING: it must be only used in the above case.
+func NewEmptyFreezeDB(db ethdb.KeyValueStore) ethdb.Database {
+	return &emptyfreezedb{KeyValueStore: db}
+}
+
 // NewFreezerDb only create a freezer without statedb.
 func NewFreezerDb(db ethdb.KeyValueStore, frz, namespace string, readonly bool, newOffSet uint64) (*Freezer, error) {
 	// Create the idle freezer instance, this operation should be atomic to avoid mismatch between offset and acientDB.
@@ -380,6 +501,12 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 		offset = ReadOffSetOfCurrentAncientFreezer(db)
 	}
 
+	// This case is used for someone who wants to execute geth db inspect CLI in a pruned db
+	if !disableFreeze && readonly && ReadAncientType(db) == PruneFreezerType {
+		log.Warn("Disk db is pruned, using an empty freezer db for CLI")
+		return NewEmptyFreezeDB(db), nil
+	}
+
 	if pruneAncientData && !disableFreeze && !readonly {
 		frdb, err := newPrunedFreezer(resolveChainFreezerDir(ancient), db, offset)
 		if err != nil {
@@ -408,8 +535,17 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 
 	// Create the idle freezer instance
 	frdb, err := newChainFreezer(resolveChainFreezerDir(ancient), namespace, readonly, offset, multiDatabase)
+
+	// We are creating the freezerdb here because the validation logic for db and freezer below requires certain interfaces
+	// that need a database type. Therefore, we are pre-creating it for subsequent use.
+	freezerDb := &freezerdb{
+		ancientRoot:    ancient,
+		KeyValueStore:  db,
+		AncientStore:   frdb,
+		AncientFreezer: frdb,
+	}
 	if err != nil {
-		printChainMetadata(db)
+		printChainMetadata(freezerDb)
 		return nil, err
 	}
 
@@ -445,10 +581,10 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 			// the freezer and the key-value store.
 			frgenesis, err := frdb.Ancient(ChainFreezerHashTable, 0)
 			if err != nil {
-				printChainMetadata(db)
+				printChainMetadata(freezerDb)
 				return nil, fmt.Errorf("failed to retrieve genesis from ancient %v", err)
 			} else if !bytes.Equal(kvgenesis, frgenesis) {
-				printChainMetadata(db)
+				printChainMetadata(freezerDb)
 				return nil, fmt.Errorf("genesis mismatch: %#x (leveldb) != %#x (ancients)", kvgenesis, frgenesis)
 			}
 			// Key-value store and freezer belong to the same network. Ensure that they
@@ -456,7 +592,7 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 			if kvhash, _ := db.Get(headerHashKey(frozen)); len(kvhash) == 0 {
 				// Subsequent header after the freezer limit is missing from the database.
 				// Reject startup if the database has a more recent head.
-				if head := *ReadHeaderNumber(db, ReadHeadHeaderHash(db)); head > frozen-1 {
+				if head := *ReadHeaderNumber(freezerDb, ReadHeadHeaderHash(freezerDb)); head > frozen-1 {
 					// Find the smallest block stored in the key-value store
 					// in range of [frozen, head]
 					var number uint64
@@ -466,7 +602,7 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 						}
 					}
 					// We are about to exit on error. Print database metadata before exiting
-					printChainMetadata(db)
+					printChainMetadata(freezerDb)
 					return nil, fmt.Errorf("gap in the chain between ancients [0 - #%d] and leveldb [#%d - #%d] ",
 						frozen-1, number, head)
 				}
@@ -481,11 +617,11 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 			// store, otherwise we'll end up missing data. We check block #1 to decide
 			// if we froze anything previously or not, but do take care of databases with
 			// only the genesis block.
-			if ReadHeadHeaderHash(db) != common.BytesToHash(kvgenesis) {
+			if ReadHeadHeaderHash(freezerDb) != common.BytesToHash(kvgenesis) {
 				// Key-value store contains more data than the genesis block, make sure we
 				// didn't freeze anything yet.
 				if kvblob, _ := db.Get(headerHashKey(1)); len(kvblob) == 0 {
-					printChainMetadata(db)
+					printChainMetadata(freezerDb)
 					return nil, errors.New("ancient chain segments already extracted, please set --datadir.ancient to the correct path")
 				}
 				// Block #1 is still in the database, we're allowed to init a new freezer
@@ -507,12 +643,7 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
 			frdb.wg.Done()
 		}()
 	}
-	return &freezerdb{
-		ancientRoot:    ancient,
-		KeyValueStore:  db,
-		AncientStore:   frdb,
-		AncientFreezer: frdb,
-	}, nil
+	return freezerDb, nil
 }
 
 // NewMemoryDatabase creates an ephemeral in-memory key-value database without a
@@ -634,7 +765,7 @@ func Open(o OpenOptions) (ethdb.Database, error) {
 	}
 	if ReadAncientType(kvdb) == PruneFreezerType {
 		if !o.PruneAncientData {
-			log.Warn("Disk db is pruned")
+			log.Warn("NOTICE: You're opening a pruned disk db!")
 		}
 	}
 	if len(o.AncientsDirectory) == 0 {
@@ -763,7 +894,7 @@ func DataTypeByKey(key []byte) DataType {
 				return StateDataType
 			}
 		}
-		for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey} {
+		for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey, headBlockKey, headFastBlockKey} {
 			if bytes.Equal(key, meta) {
 				return BlockDataType
 			}
@@ -978,7 +1109,7 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
 				hashNumPairings.Add(size)
 			default:
 				var accounted bool
-				for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey} {
+				for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey, headBlockKey, headFastBlockKey} {
 					if bytes.Equal(key, meta) {
 						metadata.Add(size)
 						accounted = true
@@ -1128,7 +1259,7 @@ func DeleteTrieState(db ethdb.Database) error {
 }
 
 // printChainMetadata prints out chain metadata to stderr.
-func printChainMetadata(db ethdb.KeyValueStore) {
+func printChainMetadata(db ethdb.Reader) {
 	fmt.Fprintf(os.Stderr, "Chain metadata\n")
 	for _, v := range ReadChainMetadata(db) {
 		fmt.Fprintf(os.Stderr, "  %s\n", strings.Join(v, ": "))
@@ -1139,7 +1270,7 @@ func printChainMetadata(db ethdb.KeyValueStore) {
 // ReadChainMetadata returns a set of key/value pairs that contains information
 // about the database chain status. This can be used for diagnostic purposes
 // when investigating the state of the node.
-func ReadChainMetadata(db ethdb.KeyValueStore) [][]string {
+func ReadChainMetadata(db ethdb.Reader) [][]string {
 	pp := func(val *uint64) string {
 		if val == nil {
 			return "<nil>"
@@ -1161,26 +1292,3 @@ func ReadChainMetadata(db ethdb.KeyValueStore) [][]string {
 	}
 	return data
 }
-
-func ReadChainMetadataFromMultiDatabase(db ethdb.Database) [][]string {
-	pp := func(val *uint64) string {
-		if val == nil {
-			return "<nil>"
-		}
-		return fmt.Sprintf("%d (%#x)", *val, *val)
-	}
-	data := [][]string{
-		{"databaseVersion", pp(ReadDatabaseVersion(db))},
-		{"headBlockHash", fmt.Sprintf("%v", ReadHeadBlockHash(db.BlockStore()))},
-		{"headFastBlockHash", fmt.Sprintf("%v", ReadHeadFastBlockHash(db))},
-		{"headHeaderHash", fmt.Sprintf("%v", ReadHeadHeaderHash(db.BlockStore()))},
-		{"lastPivotNumber", pp(ReadLastPivotNumber(db))},
-		{"len(snapshotSyncStatus)", fmt.Sprintf("%d bytes", len(ReadSnapshotSyncStatus(db)))},
-		{"snapshotDisabled", fmt.Sprintf("%v", ReadSnapshotDisabled(db))},
-		{"snapshotJournal", fmt.Sprintf("%d bytes", len(ReadSnapshotJournal(db)))},
-		{"snapshotRecoveryNumber", pp(ReadSnapshotRecoveryNumber(db))},
-		{"snapshotRoot", fmt.Sprintf("%v", ReadSnapshotRoot(db))},
-		{"txIndexTail", pp(ReadTxIndexTail(db))},
-	}
-	return data
-}

core/rawdb/freezer.go

@@ -541,41 +541,6 @@ func gcKvStore(db ethdb.KeyValueStore, ancients []common.Hash, first uint64, fro
 	}
 	batch.Reset()
 
-	// Step into the future and delete and dangling side chains
-	if frozen > 0 {
-		tip := frozen
-		nfdb := &nofreezedb{KeyValueStore: db}
-		for len(dangling) > 0 {
-			drop := make(map[common.Hash]struct{})
-			for _, hash := range dangling {
-				log.Debug("Dangling parent from freezer", "number", tip-1, "hash", hash)
-				drop[hash] = struct{}{}
-			}
-			children := ReadAllHashes(db, tip)
-			for i := 0; i < len(children); i++ {
-				// Dig up the child and ensure it's dangling
-				child := ReadHeader(nfdb, children[i], tip)
-				if child == nil {
-					log.Error("Missing dangling header", "number", tip, "hash", children[i])
-					continue
-				}
-				if _, ok := drop[child.ParentHash]; !ok {
-					children = append(children[:i], children[i+1:]...)
-					i--
-					continue
-				}
-				// Delete all block data associated with the child
-				log.Debug("Deleting dangling block", "number", tip, "hash", children[i], "parent", child.ParentHash)
-				DeleteBlock(batch, children[i], tip)
-			}
-			dangling = children
-			tip++
-		}
-		if err := batch.Write(); err != nil {
-			log.Crit("Failed to delete dangling side blocks", "err", err)
-		}
-	}
-
 	// Log something friendly for the user
 	context := []interface{}{
 		"blocks", frozen - first, "elapsed", common.PrettyDuration(time.Since(start)), "number", frozen - 1,

core/rawdb/freezer_table.go

@@ -127,6 +127,11 @@ func newFreezerTable(path, name string, disableSnappy, readonly bool) (*freezerT
 	return newTable(path, name, metrics.NilMeter{}, metrics.NilMeter{}, metrics.NilGauge{}, freezerTableSize, disableSnappy, readonly)
 }
 
+// newAdditionTable opens the given path as a addition table.
+func newAdditionTable(path, name string, disableSnappy, readonly bool) (*freezerTable, error) {
+	return openAdditionTable(path, name, metrics.NilMeter{}, metrics.NilMeter{}, metrics.NilGauge{}, freezerTableSize, disableSnappy, readonly)
+}
+
 // newTable opens a freezer table, creating the data and index files if they are
 // non-existent. Both files are truncated to the shortest common length to ensure
 // they don't go out of sync.

core/rawdb/prunedfreezer.go

@@ -8,6 +8,8 @@ import (
 	"sync/atomic"
 	"time"
 
+	"golang.org/x/exp/slices"
+
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/ethdb"
 	"github.com/ethereum/go-ethereum/log"
@@ -66,28 +68,49 @@ func newPrunedFreezer(datadir string, db ethdb.KeyValueStore, offset uint64) (*p
 
 // repair init frozen , compatible disk-ancientdb and pruner-block-tool.
 func (f *prunedfreezer) repair(datadir string) error {
-	offset := atomic.LoadUint64(&f.frozen)
 	// compatible freezer
-	min := uint64(math.MaxUint64)
+	minItems := uint64(math.MaxUint64)
 	for name, disableSnappy := range chainFreezerNoSnappy {
-		table, err := newFreezerTable(datadir, name, disableSnappy, false)
+		var (
+			table *freezerTable
+			err   error
+		)
+		if slices.Contains(additionTables, name) {
+			table, err = newAdditionTable(datadir, name, disableSnappy, false)
+		} else {
+			table, err = newFreezerTable(datadir, name, disableSnappy, false)
+		}
 		if err != nil {
 			return err
 		}
+		// addition tables only align head
+		if slices.Contains(additionTables, name) {
+			if EmptyTable(table) {
+				continue
+			}
+		}
 		items := table.items.Load()
-		if min > items {
-			min = items
+		if minItems > items {
+			minItems = items
 		}
 		table.Close()
 	}
-	log.Info("Read ancientdb item counts", "items", min)
-	offset += min
 
-	if frozen := ReadFrozenOfAncientFreezer(f.db); frozen > offset {
-		offset = frozen
+	// If minItems is non-zero, it indicates that the chain freezer was previously enabled, and we should use minItems as the current frozen value.
+	// If minItems is zero, it indicates that the pruneAncient was previously enabled, and we should continue using frozen
+	//	(retrieved from CurrentAncientFreezer) as the current frozen value.
+	offset := minItems
+	if offset == 0 {
+		// no item in ancientDB, init `offset` to the `f.frozen`
+		offset = atomic.LoadUint64(&f.frozen)
 	}
+	log.Info("Read ancientdb item counts", "items", minItems, "offset", offset)
 
-	atomic.StoreUint64(&f.frozen, offset)
+	// FrozenOfAncientFreezer is the progress of the last prune-freezer freeze.
+	frozenInDB := ReadFrozenOfAncientFreezer(f.db)
+	maxOffset := max(offset, frozenInDB)
+
+	atomic.StoreUint64(&f.frozen, maxOffset)
 	if err := f.Sync(); err != nil {
 		return nil
 	}
@@ -299,9 +322,8 @@ func (f *prunedfreezer) freeze() {
 				log.Error("Append ancient err", "number", f.frozen, "hash", hash, "err", err)
 				break
 			}
-			if hash != (common.Hash{}) {
-				ancients = append(ancients, hash)
-			}
+			// may include common.Hash{}, will be delete in gcKvStore
+			ancients = append(ancients, hash)
 		}
 		// Batch of blocks have been frozen, flush them before wiping from leveldb
 		if err := f.Sync(); err != nil {

core/rawdb/table.go

@@ -251,6 +251,10 @@ func (t *table) SetStateStore(state ethdb.Database) {
 	panic("not implement")
 }
 
+func (t *table) GetStateStore() ethdb.Database {
+	return nil
+}
+
 func (t *table) StateStoreReader() ethdb.Reader {
 	return nil
 }

core/state/journal.go

@@ -164,7 +164,7 @@ func (ch createObjectChange) dirtied() *common.Address {
 func (ch resetObjectChange) revert(s *StateDB) {
 	s.setStateObject(ch.prev)
 	if !ch.prevdestruct {
-		s.deleteStateObjectsDestruct(ch.prev.address)
+		delete(s.stateObjectsDestruct, ch.prev.address)
 	}
 	if ch.prevAccount != nil {
 		s.accounts[ch.prev.addrHash] = ch.prevAccount

core/state/state_object.go

@@ -19,7 +19,6 @@ package state
 import (
 	"bytes"
 	"fmt"
-	"golang.org/x/exp/slices"
 	"io"
 	"sync"
 	"time"
@@ -69,11 +68,6 @@ type stateObject struct {
 	origin   *types.StateAccount // Account original data without any change applied, nil means it was not existent
 	data     types.StateAccount  // Account data with all mutations applied in the scope of block
 
-	// dirty account state
-	dirtyBalance  *uint256.Int
-	dirtyNonce    *uint64
-	dirtyCodeHash []byte
-
 	// Write caches.
 	trie Trie // storage trie, which becomes non-nil on first access
 	code Code // contract bytecode, which gets set when code is loaded
@@ -101,7 +95,7 @@ type stateObject struct {
 
 // empty returns whether the account is considered empty.
 func (s *stateObject) empty() bool {
-	return s.Nonce() == 0 && s.Balance().IsZero() && bytes.Equal(s.CodeHash(), types.EmptyCodeHash.Bytes())
+	return s.data.Nonce == 0 && s.data.Balance.IsZero() && bytes.Equal(s.data.CodeHash, types.EmptyCodeHash.Bytes())
 }
 
 // newObject creates a state object.
@@ -119,7 +113,7 @@ func newObject(db *StateDB, address common.Address, acct *types.StateAccount) *s
 		storageMap = db.GetStorage(address)
 	}
 
-	s := &stateObject{
+	return &stateObject{
 		db:                  db,
 		address:             address,
 		addrHash:            crypto.Keccak256Hash(address[:]),
@@ -131,15 +125,6 @@ func newObject(db *StateDB, address common.Address, acct *types.StateAccount) *s
 		dirtyStorage:        make(Storage),
 		created:             created,
 	}
-
-	// dirty data when create a new account
-	if acct == nil {
-		s.dirtyBalance = acct.Balance.Clone()
-		s.dirtyNonce = new(uint64)
-		*s.dirtyNonce = acct.Nonce
-		s.dirtyCodeHash = acct.CodeHash
-	}
-	return s
 }
 
 // EncodeRLP implements rlp.Encoder.
@@ -234,7 +219,7 @@ func (s *stateObject) GetCommittedState(key common.Hash) common.Hash {
 	//   1) resurrect happened, and new slot values were set -- those should
 	//      have been handles via pendingStorage above.
 	//   2) we don't have new values, and can deliver empty response back
-	if _, destructed := s.db.queryStateObjectsDestruct(s.address); destructed {
+	if _, destructed := s.db.stateObjectsDestruct[s.address]; destructed {
 		return common.Hash{}
 	}
 	// If no live objects are available, attempt to use snapshots
@@ -309,18 +294,6 @@ func (s *stateObject) finalise(prefetch bool) {
 			slotsToPrefetch = append(slotsToPrefetch, common.CopyBytes(key[:])) // Copy needed for closure
 		}
 	}
-	if s.dirtyNonce != nil {
-		s.data.Nonce = *s.dirtyNonce
-		s.dirtyNonce = nil
-	}
-	if s.dirtyBalance != nil {
-		s.data.Balance = s.dirtyBalance
-		s.dirtyBalance = nil
-	}
-	if s.dirtyCodeHash != nil {
-		s.data.CodeHash = s.dirtyCodeHash
-		s.dirtyCodeHash = nil
-	}
 	if s.db.prefetcher != nil && prefetch && len(slotsToPrefetch) > 0 && s.data.Root != types.EmptyRootHash {
 		s.db.prefetcher.prefetch(s.addrHash, s.data.Root, s.address, slotsToPrefetch)
 	}
@@ -329,26 +302,6 @@ func (s *stateObject) finalise(prefetch bool) {
 	}
 }
 
-func (s *stateObject) finaliseRWSet() {
-	for key, value := range s.dirtyStorage {
-		// three are some unclean dirtyStorage from previous reverted txs, it will skip finalise
-		// so add a new rule, if val has no change, then skip it
-		if value == s.GetCommittedState(key) {
-			continue
-		}
-		s.db.RecordWrite(types.StorageStateKey(s.address, key), value)
-	}
-	if s.dirtyNonce != nil && *s.dirtyNonce != s.data.Nonce {
-		s.db.RecordWrite(types.AccountStateKey(s.address, types.AccountNonce), *s.dirtyNonce)
-	}
-	if s.dirtyBalance != nil && !s.dirtyBalance.Eq(s.data.Balance) {
-		s.db.RecordWrite(types.AccountStateKey(s.address, types.AccountBalance), s.dirtyBalance.Clone())
-	}
-	if s.dirtyCodeHash != nil && !slices.Equal(s.dirtyCodeHash, s.data.CodeHash) {
-		s.db.RecordWrite(types.AccountStateKey(s.address, types.AccountCodeHash), s.dirtyCodeHash)
-	}
-}
-
 // updateTrie is responsible for persisting cached storage changes into the
 // object's storage trie. In case the storage trie is not yet loaded, this
 // function will load the trie automatically. If any issues arise during the
@@ -548,13 +501,13 @@ func (s *stateObject) SubBalance(amount *uint256.Int) {
 func (s *stateObject) SetBalance(amount *uint256.Int) {
 	s.db.journal.append(balanceChange{
 		account: &s.address,
-		prev:    new(uint256.Int).Set(s.Balance()),
+		prev:    new(uint256.Int).Set(s.data.Balance),
 	})
 	s.setBalance(amount)
 }
 
 func (s *stateObject) setBalance(amount *uint256.Int) {
-	s.dirtyBalance = amount
+	s.data.Balance = amount
 }
 
 func (s *stateObject) deepCopy(db *StateDB) *stateObject {
@@ -575,17 +528,6 @@ func (s *stateObject) deepCopy(db *StateDB) *stateObject {
 	obj.selfDestructed = s.selfDestructed
 	obj.dirtyCode = s.dirtyCode
 	obj.deleted = s.deleted
-
-	// dirty states
-	if s.dirtyNonce != nil {
-		obj.dirtyNonce = new(uint64)
-		*obj.dirtyNonce = *s.dirtyNonce
-	}
-	if s.dirtyBalance != nil {
-		obj.dirtyBalance = s.dirtyBalance.Clone()
-	}
-	obj.dirtyCodeHash = s.dirtyCodeHash
-
 	return obj
 }
 
@@ -643,44 +585,32 @@ func (s *stateObject) SetCode(codeHash common.Hash, code []byte) {
 
 func (s *stateObject) setCode(codeHash common.Hash, code []byte) {
 	s.code = code
-	s.dirtyCodeHash = codeHash[:]
+	s.data.CodeHash = codeHash[:]
 	s.dirtyCode = true
 }
 
 func (s *stateObject) SetNonce(nonce uint64) {
 	s.db.journal.append(nonceChange{
 		account: &s.address,
-		prev:    s.Nonce(),
+		prev:    s.data.Nonce,
 	})
 	s.setNonce(nonce)
 }
 
 func (s *stateObject) setNonce(nonce uint64) {
-	s.dirtyNonce = &nonce
+	s.data.Nonce = nonce
 }
 
 func (s *stateObject) CodeHash() []byte {
-	if len(s.dirtyCodeHash) > 0 {
-		return s.dirtyCodeHash
-	}
-	ret := s.data.CodeHash
-	return ret
+	return s.data.CodeHash
 }
 
 func (s *stateObject) Balance() *uint256.Int {
-	if s.dirtyBalance != nil {
-		return s.dirtyBalance
-	}
-	ret := s.data.Balance
-	return ret
+	return s.data.Balance
 }
 
 func (s *stateObject) Nonce() uint64 {
-	if s.dirtyNonce != nil {
-		return *s.dirtyNonce
-	}
-	ret := s.data.Nonce
-	return ret
+	return s.data.Nonce
 }
 
 func (s *stateObject) Root() common.Hash {

core/state/statedb.go

@@ -95,11 +95,10 @@ type StateDB struct {
 
 	// This map holds 'live' objects, which will get modified while processing
 	// a state transition.
-	stateObjects              map[common.Address]*stateObject
-	stateObjectsPending       map[common.Address]struct{}            // State objects finalized but not yet written to the trie
-	stateObjectsDirty         map[common.Address]struct{}            // State objects modified in the current execution
-	stateObjectsDestruct      map[common.Address]*types.StateAccount // State objects destructed in the block along with its previous value
-	stateObjectsDestructDirty map[common.Address]*types.StateAccount
+	stateObjects         map[common.Address]*stateObject
+	stateObjectsPending  map[common.Address]struct{}            // State objects finalized but not yet written to the trie
+	stateObjectsDirty    map[common.Address]struct{}            // State objects modified in the current execution
+	stateObjectsDestruct map[common.Address]*types.StateAccount // State objects destructed in the block along with its previous value
 
 	storagePool          *StoragePool // sharedPool to store L1 originStorage of stateObjects
 	writeOnSharedStorage bool         // Write to the shared origin storage of a stateObject while reading from the underlying storage layer.
@@ -116,16 +115,10 @@ type StateDB struct {
 	refund uint64
 
 	// The tx context and all occurred logs in the scope of transaction.
-	thash         common.Hash
-	txIndex       int
-	txIncarnation int
-	logs          map[common.Hash][]*types.Log
-	logSize       uint
-
-	// parallel EVM related
-	rwSet    *types.RWSet
-	mvStates *types.MVStates
-	es       *types.ExeStat
+	thash   common.Hash
+	txIndex int
+	logs    map[common.Hash][]*types.Log
+	logSize uint
 
 	// Preimages occurred seen by VM in the scope of block.
 	preimages map[common.Hash][]byte
@@ -180,24 +173,23 @@ func NewWithSharedPool(root common.Hash, db Database, snaps *snapshot.Tree) (*St
 // New creates a new state from a given trie.
 func New(root common.Hash, db Database, snaps *snapshot.Tree) (*StateDB, error) {
 	sdb := &StateDB{
-		db:                        db,
-		originalRoot:              root,
-		snaps:                     snaps,
-		accounts:                  make(map[common.Hash][]byte),
-		storages:                  make(map[common.Hash]map[common.Hash][]byte),
-		accountsOrigin:            make(map[common.Address][]byte),
-		storagesOrigin:            make(map[common.Address]map[common.Hash][]byte),
-		stateObjects:              make(map[common.Address]*stateObject, defaultNumOfSlots),
-		stateObjectsPending:       make(map[common.Address]struct{}, defaultNumOfSlots),
-		stateObjectsDirty:         make(map[common.Address]struct{}, defaultNumOfSlots),
-		stateObjectsDestruct:      make(map[common.Address]*types.StateAccount, defaultNumOfSlots),
-		stateObjectsDestructDirty: make(map[common.Address]*types.StateAccount, defaultNumOfSlots),
-		logs:                      make(map[common.Hash][]*types.Log),
-		preimages:                 make(map[common.Hash][]byte),
-		journal:                   newJournal(),
-		accessList:                newAccessList(),
-		transientStorage:          newTransientStorage(),
-		hasher:                    crypto.NewKeccakState(),
+		db:                   db,
+		originalRoot:         root,
+		snaps:                snaps,
+		accounts:             make(map[common.Hash][]byte),
+		storages:             make(map[common.Hash]map[common.Hash][]byte),
+		accountsOrigin:       make(map[common.Address][]byte),
+		storagesOrigin:       make(map[common.Address]map[common.Hash][]byte),
+		stateObjects:         make(map[common.Address]*stateObject, defaultNumOfSlots),
+		stateObjectsPending:  make(map[common.Address]struct{}, defaultNumOfSlots),
+		stateObjectsDirty:    make(map[common.Address]struct{}, defaultNumOfSlots),
+		stateObjectsDestruct: make(map[common.Address]*types.StateAccount, defaultNumOfSlots),
+		logs:                 make(map[common.Hash][]*types.Log),
+		preimages:            make(map[common.Hash][]byte),
+		journal:              newJournal(),
+		accessList:           newAccessList(),
+		transientStorage:     newTransientStorage(),
+		hasher:               crypto.NewKeccakState(),
 	}
 
 	if sdb.snaps != nil {
@@ -433,10 +425,7 @@ func (s *StateDB) Empty(addr common.Address) bool {
 }
 
 // GetBalance retrieves the balance from the given address or 0 if object not found
-func (s *StateDB) GetBalance(addr common.Address) (ret *uint256.Int) {
-	defer func() {
-		s.RecordRead(types.AccountStateKey(addr, types.AccountBalance), ret)
-	}()
+func (s *StateDB) GetBalance(addr common.Address) *uint256.Int {
 	stateObject := s.getStateObject(addr)
 	if stateObject != nil {
 		return stateObject.Balance()
@@ -445,10 +434,7 @@ func (s *StateDB) GetBalance(addr common.Address) (ret *uint256.Int) {
 }
 
 // GetNonce retrieves the nonce from the given address or 0 if object not found
-func (s *StateDB) GetNonce(addr common.Address) (ret uint64) {
-	defer func() {
-		s.RecordRead(types.AccountStateKey(addr, types.AccountNonce), ret)
-	}()
+func (s *StateDB) GetNonce(addr common.Address) uint64 {
 	stateObject := s.getStateObject(addr)
 	if stateObject != nil {
 		return stateObject.Nonce()
@@ -496,10 +482,7 @@ func (s *StateDB) GetCodeSize(addr common.Address) int {
 	return 0
 }
 
-func (s *StateDB) GetCodeHash(addr common.Address) (ret common.Hash) {
-	defer func() {
-		s.RecordRead(types.AccountStateKey(addr, types.AccountCodeHash), ret.Bytes())
-	}()
+func (s *StateDB) GetCodeHash(addr common.Address) common.Hash {
 	stateObject := s.getStateObject(addr)
 	if stateObject != nil {
 		return common.BytesToHash(stateObject.CodeHash())
@@ -508,10 +491,7 @@ func (s *StateDB) GetCodeHash(addr common.Address) (ret common.Hash) {
 }
 
 // GetState retrieves a value from the given account's storage trie.
-func (s *StateDB) GetState(addr common.Address, hash common.Hash) (ret common.Hash) {
-	defer func() {
-		s.RecordRead(types.StorageStateKey(addr, hash), ret)
-	}()
+func (s *StateDB) GetState(addr common.Address, hash common.Hash) common.Hash {
 	stateObject := s.getStateObject(addr)
 	if stateObject != nil {
 		return stateObject.GetState(hash)
@@ -520,10 +500,7 @@ func (s *StateDB) GetState(addr common.Address, hash common.Hash) (ret common.Ha
 }
 
 // GetCommittedState retrieves a value from the given account's committed storage trie.
-func (s *StateDB) GetCommittedState(addr common.Address, hash common.Hash) (ret common.Hash) {
-	defer func() {
-		s.RecordRead(types.StorageStateKey(addr, hash), ret)
-	}()
+func (s *StateDB) GetCommittedState(addr common.Address, hash common.Hash) common.Hash {
 	stateObject := s.getStateObject(addr)
 	if stateObject != nil {
 		return stateObject.GetCommittedState(hash)
@@ -552,7 +529,6 @@ func (s *StateDB) HasSelfDestructed(addr common.Address) bool {
 func (s *StateDB) AddBalance(addr common.Address, amount *uint256.Int) {
 	stateObject := s.getOrNewStateObject(addr)
 	if stateObject != nil {
-		s.RecordRead(types.AccountStateKey(addr, types.AccountBalance), stateObject.Balance())
 		stateObject.AddBalance(amount)
 	}
 }
@@ -561,7 +537,6 @@ func (s *StateDB) AddBalance(addr common.Address, amount *uint256.Int) {
 func (s *StateDB) SubBalance(addr common.Address, amount *uint256.Int) {
 	stateObject := s.getOrNewStateObject(addr)
 	if stateObject != nil {
-		s.RecordRead(types.AccountStateKey(addr, types.AccountBalance), stateObject.Balance())
 		stateObject.SubBalance(amount)
 	}
 }
@@ -606,8 +581,8 @@ func (s *StateDB) SetStorage(addr common.Address, storage map[common.Hash]common
 	//
 	// TODO(rjl493456442) this function should only be supported by 'unwritable'
 	// state and all mutations made should all be discarded afterwards.
-	if _, ok := s.queryStateObjectsDestruct(addr); !ok {
-		s.tagStateObjectsDestruct(addr, nil)
+	if _, ok := s.stateObjectsDestruct[addr]; !ok {
+		s.stateObjectsDestruct[addr] = nil
 	}
 	stateObject := s.getOrNewStateObject(addr)
 	for k, v := range storage {
@@ -631,7 +606,7 @@ func (s *StateDB) SelfDestruct(addr common.Address) {
 		prevbalance: new(uint256.Int).Set(stateObject.Balance()),
 	})
 	stateObject.markSelfdestructed()
-	stateObject.setBalance(new(uint256.Int))
+	stateObject.data.Balance = new(uint256.Int)
 }
 
 func (s *StateDB) Selfdestruct6780(addr common.Address) {
@@ -737,7 +712,6 @@ func (s *StateDB) getStateObject(addr common.Address) *stateObject {
 // flag set. This is needed by the state journal to revert to the correct s-
 // destructed object instead of wiping all knowledge about the state object.
 func (s *StateDB) getDeletedStateObject(addr common.Address) *stateObject {
-	s.RecordRead(types.AccountStateKey(addr, types.AccountSelf), struct{}{})
 	// Prefer live objects if any is available
 	if obj := s.stateObjects[addr]; obj != nil {
 		return obj
@@ -824,9 +798,9 @@ func (s *StateDB) createObject(addr common.Address) (newobj, prev *stateObject)
 		// account and storage data should be cleared as well. Note, it must
 		// be done here, otherwise the destruction event of "original account"
 		// will be lost.
-		_, prevdestruct := s.queryStateObjectsDestruct(prev.address)
+		_, prevdestruct := s.stateObjectsDestruct[prev.address]
 		if !prevdestruct {
-			s.tagStateObjectsDestruct(prev.address, prev.origin)
+			s.stateObjectsDestruct[prev.address] = prev.origin
 		}
 		// There may be some cached account/storage data already since IntermediateRoot
 		// will be called for each transaction before byzantium fork which will always
@@ -867,7 +841,7 @@ func (s *StateDB) createObject(addr common.Address) (newobj, prev *stateObject)
 func (s *StateDB) CreateAccount(addr common.Address) {
 	newObj, prev := s.createObject(addr)
 	if prev != nil {
-		newObj.setBalance(prev.Balance())
+		newObj.setBalance(prev.data.Balance)
 	}
 }
 
@@ -895,16 +869,15 @@ func (s *StateDB) copyInternal(doPrefetch bool) *StateDB {
 		originalRoot: s.originalRoot,
 		// fullProcessed:        s.fullProcessed,
 		// pipeCommit:           s.pipeCommit,
-		accounts:                  make(map[common.Hash][]byte),
-		storages:                  make(map[common.Hash]map[common.Hash][]byte),
-		accountsOrigin:            make(map[common.Address][]byte),
-		storagesOrigin:            make(map[common.Address]map[common.Hash][]byte),
-		stateObjects:              make(map[common.Address]*stateObject, len(s.journal.dirties)),
-		stateObjectsPending:       make(map[common.Address]struct{}, len(s.stateObjectsPending)),
-		stateObjectsDirty:         make(map[common.Address]struct{}, len(s.journal.dirties)),
-		stateObjectsDestruct:      make(map[common.Address]*types.StateAccount, len(s.stateObjectsDestruct)),
-		stateObjectsDestructDirty: make(map[common.Address]*types.StateAccount, len(s.stateObjectsDestructDirty)),
-		storagePool:               s.storagePool,
+		accounts:             make(map[common.Hash][]byte),
+		storages:             make(map[common.Hash]map[common.Hash][]byte),
+		accountsOrigin:       make(map[common.Address][]byte),
+		storagesOrigin:       make(map[common.Address]map[common.Hash][]byte),
+		stateObjects:         make(map[common.Address]*stateObject, len(s.journal.dirties)),
+		stateObjectsPending:  make(map[common.Address]struct{}, len(s.stateObjectsPending)),
+		stateObjectsDirty:    make(map[common.Address]struct{}, len(s.journal.dirties)),
+		stateObjectsDestruct: make(map[common.Address]*types.StateAccount, len(s.stateObjectsDestruct)),
+		storagePool:          s.storagePool,
 		// writeOnSharedStorage: s.writeOnSharedStorage,
 		refund:    s.refund,
 		logs:      make(map[common.Hash][]*types.Log, len(s.logs)),
@@ -956,15 +929,12 @@ func (s *StateDB) copyInternal(doPrefetch bool) *StateDB {
 	for addr, value := range s.stateObjectsDestruct {
 		state.stateObjectsDestruct[addr] = value
 	}
-	for addr, value := range s.stateObjectsDestructDirty {
-		state.stateObjectsDestructDirty[addr] = value
-	}
 	// Deep copy the state changes made in the scope of block
 	// along with their original values.
 	state.accounts = copySet(s.accounts)
 	state.storages = copy2DSet(s.storages)
-	state.accountsOrigin = copySet(state.accountsOrigin)
-	state.storagesOrigin = copy2DSet(state.storagesOrigin)
+	state.accountsOrigin = copySet(s.accountsOrigin)
+	state.storagesOrigin = copy2DSet(s.storagesOrigin)
 
 	// Deep copy the logs occurred in the scope of block
 	for hash, logs := range s.logs {
@@ -997,12 +967,6 @@ func (s *StateDB) copyInternal(doPrefetch bool) *StateDB {
 		// know that they need to explicitly terminate an active copy).
 		state.prefetcher = state.prefetcher.copy()
 	}
-
-	// parallel EVM related
-	if s.mvStates != nil {
-		state.mvStates = s.mvStates
-	}
-
 	return state
 }
 
@@ -1051,11 +1015,6 @@ func (s *StateDB) WaitPipeVerification() error {
 // into the tries just yet. Only IntermediateRoot or Commit will do that.
 func (s *StateDB) Finalise(deleteEmptyObjects bool) {
 	addressesToPrefetch := make([][]byte, 0, len(s.journal.dirties))
-	// finalise stateObjectsDestruct
-	for addr, acc := range s.stateObjectsDestructDirty {
-		s.stateObjectsDestruct[addr] = acc
-	}
-	s.stateObjectsDestructDirty = make(map[common.Address]*types.StateAccount)
 	for addr := range s.journal.dirties {
 		obj, exist := s.stateObjects[addr]
 		if !exist {
@@ -1290,10 +1249,9 @@ func (s *StateDB) StateIntermediateRoot() common.Hash {
 // SetTxContext sets the current transaction hash and index which are
 // used when the EVM emits new state logs. It should be invoked before
 // transaction execution.
-func (s *StateDB) SetTxContext(thash common.Hash, txIndex int, incarnation int) {
+func (s *StateDB) SetTxContext(thash common.Hash, ti int) {
 	s.thash = thash
-	s.txIndex = txIndex
-	s.txIncarnation = incarnation
+	s.txIndex = ti
 	s.accessList = nil // can't delete this line now, because StateDB.Prepare is not called before processsing a system transaction
 }
 
@@ -1942,131 +1900,6 @@ func (s *StateDB) GetSnap() snapshot.Snapshot {
 	return s.snap
 }
 
-func (s *StateDB) BeforeTxTransition() {
-	log.Debug("BeforeTxTransition", "mvStates", s.mvStates == nil, "rwSet", s.rwSet == nil)
-	if s.mvStates == nil {
-		return
-	}
-	s.rwSet = types.NewRWSet(types.StateVersion{
-		TxIndex:       s.txIndex,
-		TxIncarnation: s.txIncarnation,
-	})
-}
-
-func (s *StateDB) BeginTxStat(index int) {
-	if s.mvStates == nil {
-		return
-	}
-	s.es = types.NewExeStat(index).Begin()
-}
-
-func (s *StateDB) StopTxStat(usedGas uint64) {
-	if s.mvStates == nil {
-		return
-	}
-	// record stat first
-	if s.es != nil {
-		s.es.Done().WithGas(usedGas).WithRead(len(s.rwSet.ReadSet()))
-	}
-}
-
-func (s *StateDB) RecordRead(key types.RWKey, val interface{}) {
-	if s.mvStates == nil || s.rwSet == nil {
-		return
-	}
-	// TODO: read from MVStates, record with ver
-	s.rwSet.RecordRead(key, types.StateVersion{
-		TxIndex: -1,
-	}, val)
-}
-
-func (s *StateDB) RecordWrite(key types.RWKey, val interface{}) {
-	if s.mvStates == nil || s.rwSet == nil {
-		return
-	}
-	s.rwSet.RecordWrite(key, val)
-}
-
-func (s *StateDB) ResetMVStates(txCount int) {
-	log.Debug("ResetMVStates", "mvStates", s.mvStates == nil, "rwSet", s.rwSet == nil)
-	s.mvStates = types.NewMVStates(txCount)
-	s.rwSet = nil
-}
-
-func (s *StateDB) FinaliseRWSet() error {
-	log.Debug("FinaliseRWSet", "mvStates", s.mvStates == nil, "rwSet", s.rwSet == nil)
-	if s.mvStates == nil || s.rwSet == nil {
-		return nil
-	}
-	// finalise stateObjectsDestruct
-	for addr, acc := range s.stateObjectsDestructDirty {
-		s.stateObjectsDestruct[addr] = acc
-		s.RecordWrite(types.AccountStateKey(addr, types.AccountSuicide), struct{}{})
-	}
-	for addr := range s.journal.dirties {
-		obj, exist := s.stateObjects[addr]
-		if !exist {
-			continue
-		}
-		if obj.selfDestructed || obj.empty() {
-			// We need to maintain account deletions explicitly (will remain
-			// set indefinitely). Note only the first occurred self-destruct
-			// event is tracked.
-			if _, ok := s.stateObjectsDestruct[obj.address]; !ok {
-				log.Debug("FinaliseRWSet find Destruct", "tx", s.txIndex, "addr", addr, "selfDestructed", obj.selfDestructed)
-				s.RecordWrite(types.AccountStateKey(addr, types.AccountSuicide), struct{}{})
-			}
-		} else {
-			// finalise account & storages
-			obj.finaliseRWSet()
-		}
-	}
-	ver := types.StateVersion{
-		TxIndex:       s.txIndex,
-		TxIncarnation: s.txIncarnation,
-	}
-	if ver != s.rwSet.Version() {
-		return errors.New("you finalize a wrong ver of RWSet")
-	}
-
-	log.Debug("FinaliseRWSet", "rwset", s.rwSet)
-	return s.mvStates.FulfillRWSet(s.rwSet, s.es)
-}
-
-func (s *StateDB) queryStateObjectsDestruct(addr common.Address) (*types.StateAccount, bool) {
-	if acc, ok := s.stateObjectsDestructDirty[addr]; ok {
-		return acc, ok
-	}
-	acc, ok := s.stateObjectsDestruct[addr]
-	return acc, ok
-}
-
-func (s *StateDB) tagStateObjectsDestruct(addr common.Address, acc *types.StateAccount) {
-	s.stateObjectsDestructDirty[addr] = acc
-}
-
-func (s *StateDB) deleteStateObjectsDestruct(addr common.Address) {
-	delete(s.stateObjectsDestructDirty, addr)
-}
-
-func (s *StateDB) MVStates2TxDAG() (*types.TxDAG, []*types.ExeStat) {
-	if s.mvStates == nil {
-		return nil, nil
-	}
-
-	return s.mvStates.ResolveTxDAG(), s.mvStates.Stats()
-}
-
-func (s *StateDB) RecordSystemTxRWSet(index int) {
-	if s.mvStates == nil {
-		return
-	}
-	s.mvStates.FulfillRWSet(types.NewRWSet(types.StateVersion{
-		TxIndex:       index,
-		TxIncarnation: 0,
-	}).WithSerialFlag(), types.NewExeStat(index).WithSerialFlag())
-}
-
 // copySet returns a deep-copied set.
 func copySet[k comparable](set map[k][]byte) map[k][]byte {
 	copied := make(map[k][]byte, len(set))

core/state/sync.go

@@ -25,7 +25,7 @@ import (
 )
 
 // NewStateSync creates a new state trie download scheduler.
-func NewStateSync(root common.Hash, database ethdb.KeyValueReader, onLeaf func(keys [][]byte, leaf []byte) error, scheme string) *trie.Sync {
+func NewStateSync(root common.Hash, database ethdb.Database, onLeaf func(keys [][]byte, leaf []byte) error, scheme string) *trie.Sync {
 	// Register the storage slot callback if the external callback is specified.
 	var onSlot func(keys [][]byte, path []byte, leaf []byte, parent common.Hash, parentPath []byte) error
 	if onLeaf != nil {

core/state/sync_test.go

@@ -268,7 +268,7 @@ func testIterativeStateSync(t *testing.T, count int, commit bool, bypath bool, s
 			}
 		}
 		batch := dstDb.NewBatch()
-		if err := sched.Commit(batch); err != nil {
+		if err := sched.Commit(batch, nil); err != nil {
 			t.Fatalf("failed to commit data: %v", err)
 		}
 		batch.Write()
@@ -369,7 +369,7 @@ func testIterativeDelayedStateSync(t *testing.T, scheme string) {
 			nodeProcessed = len(nodeResults)
 		}
 		batch := dstDb.NewBatch()
-		if err := sched.Commit(batch); err != nil {
+		if err := sched.Commit(batch, nil); err != nil {
 			t.Fatalf("failed to commit data: %v", err)
 		}
 		batch.Write()
@@ -469,7 +469,7 @@ func testIterativeRandomStateSync(t *testing.T, count int, scheme string) {
 			}
 		}
 		batch := dstDb.NewBatch()
-		if err := sched.Commit(batch); err != nil {
+		if err := sched.Commit(batch, nil); err != nil {
 			t.Fatalf("failed to commit data: %v", err)
 		}
 		batch.Write()
@@ -575,7 +575,7 @@ func testIterativeRandomDelayedStateSync(t *testing.T, scheme string) {
 			}
 		}
 		batch := dstDb.NewBatch()
-		if err := sched.Commit(batch); err != nil {
+		if err := sched.Commit(batch, nil); err != nil {
 			t.Fatalf("failed to commit data: %v", err)
 		}
 		batch.Write()
@@ -688,7 +688,7 @@ func testIncompleteStateSync(t *testing.T, scheme string) {
 			}
 		}
 		batch := dstDb.NewBatch()
-		if err := sched.Commit(batch); err != nil {
+		if err := sched.Commit(batch, nil); err != nil {
 			t.Fatalf("failed to commit data: %v", err)
 		}
 		batch.Write()

core/state_prefetcher.go

@@ -76,7 +76,7 @@ func (p *statePrefetcher) Prefetch(block *types.Block, statedb *state.StateDB, c
 					if err != nil {
 						return // Also invalid block, bail out
 					}
-					newStatedb.SetTxContext(tx.Hash(), txIndex, 0)
+					newStatedb.SetTxContext(tx.Hash(), txIndex)
 					precacheTransaction(msg, p.config, gaspool, newStatedb, header, evm)
 
 				case <-interruptCh:
@@ -125,7 +125,7 @@ func (p *statePrefetcher) PrefetchMining(txs TransactionsByPriceAndNonce, header
 						return // Also invalid block, bail out
 					}
 					idx++
-					newStatedb.SetTxContext(tx.Hash(), idx, 0)
+					newStatedb.SetTxContext(tx.Hash(), idx)
 					precacheTransaction(msg, p.config, gaspool, newStatedb, header, evm)
 					gaspool = new(GasPool).AddGas(gasLimit)
 				case <-stopCh:

core/state_processor.go

@@ -19,6 +19,8 @@ package core
 import (
 	"errors"
 	"fmt"
+	"math/big"
+
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/consensus"
 	"github.com/ethereum/go-ethereum/consensus/misc"
@@ -27,11 +29,7 @@ import (
 	"github.com/ethereum/go-ethereum/core/types"
 	"github.com/ethereum/go-ethereum/core/vm"
 	"github.com/ethereum/go-ethereum/crypto"
-	"github.com/ethereum/go-ethereum/log"
-	"github.com/ethereum/go-ethereum/metrics"
 	"github.com/ethereum/go-ethereum/params"
-	"math/big"
-	"time"
 )
 
 // StateProcessor is a basic Processor, which takes care of transitioning
@@ -53,12 +51,6 @@ func NewStateProcessor(config *params.ChainConfig, bc *BlockChain, engine consen
 	}
 }
 
-var (
-	dagExecutionTimer   = metrics.NewRegisteredTimer("dag/executiontime", nil)
-	dagAccountReadTimer = metrics.NewRegisteredTimer("dag/accountreadtime", nil)
-	dagStorageReadTimer = metrics.NewRegisteredTimer("dag/storagereadtime", nil)
-)
-
 // Process processes the state changes according to the Ethereum rules by running
 // the transaction messages using the statedb and applying any rewards to both
 // the processor (coinbase) and any included uncles.
@@ -67,7 +59,6 @@ var (
 // returns the amount of gas that was used in the process. If any of the
 // transactions failed to execute due to insufficient gas it will return an error.
 func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg vm.Config) (*state.StateDB, types.Receipts, []*types.Log, uint64, error) {
-
 	var (
 		usedGas     = new(uint64)
 		header      = block.Header()
@@ -108,20 +99,16 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg
 	// initialise bloom processors
 	bloomProcessors := NewAsyncReceiptBloomGenerator(txNum)
 	statedb.MarkFullProcessed()
-	statedb.ResetMVStates(len(block.Transactions()))
-	log.Debug("ResetMVStates", "block", block.NumberU64(), "txs", len(block.Transactions()))
 
 	// usually do have two tx, one for validator set contract, another for system reward contract.
 	systemTxs := make([]*types.Transaction, 0, 2)
-	start := time.Now()
+
 	for i, tx := range block.Transactions() {
-		statedb.BeginTxStat(i)
 		if isPoSA {
 			if isSystemTx, err := posa.IsSystemTransaction(tx, block.Header()); err != nil {
 				bloomProcessors.Close()
 				return statedb, nil, nil, 0, err
 			} else if isSystemTx {
-				statedb.RecordSystemTxRWSet(i)
 				systemTxs = append(systemTxs, tx)
 				continue
 			}
@@ -138,7 +125,7 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg
 			bloomProcessors.Close()
 			return statedb, nil, nil, 0, err
 		}
-		statedb.SetTxContext(tx.Hash(), i, 0)
+		statedb.SetTxContext(tx.Hash(), i)
 
 		receipt, err := applyTransaction(msg, p.config, gp, statedb, blockNumber, blockHash, tx, usedGas, vmenv, bloomProcessors)
 		if err != nil {
@@ -147,10 +134,7 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg
 		}
 		commonTxs = append(commonTxs, tx)
 		receipts = append(receipts, receipt)
-		statedb.StopTxStat(receipt.GasUsed)
 	}
-	eTime := time.Since(start)
-	// this bloomProcessors may take ~20ms
 	bloomProcessors.Close()
 
 	// Fail if Shanghai not enabled and len(withdrawals) is non-zero.
@@ -159,20 +143,7 @@ func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg
 		return nil, nil, nil, 0, errors.New("withdrawals before shanghai")
 	}
 
-	// TODO: temporary add time metrics
-	dag, exrStats := statedb.MVStates2TxDAG()
-	//log.Info("MVStates2TxDAG", "block", block.NumberU64(), "tx", len(block.Transactions()), "dag", dag)
-	fmt.Printf("MVStates2TxDAG, block: %v|%v, tx: %v, time: %v\n", block.NumberU64(), block.Hash(), len(block.Transactions()), time.Now().Format(time.DateTime))
-	fmt.Print(types.EvaluateTxDAGPerformance(dag, exrStats))
-	fmt.Printf("block: %v, execution: %.2fms, accountRead: %.2fms, storageRead: %.2fms\n",
-		block.NumberU64(), float64(eTime.Microseconds())/1000, float64((statedb.SnapshotAccountReads+statedb.AccountReads).Microseconds())/1000,
-		float64((statedb.SnapshotStorageReads+statedb.StorageReads).Microseconds())/1000)
-
-	dagExecutionTimer.Update(eTime)
-	dagAccountReadTimer.Update(statedb.SnapshotAccountReads + statedb.AccountReads)
-	dagStorageReadTimer.Update(statedb.SnapshotStorageReads + statedb.StorageReads)
 	// Finalize the block, applying any consensus engine specific extras (e.g. block rewards)
-	// TODO: system txs must execute at last
 	err := p.engine.Finalize(p.bc, header, statedb, &commonTxs, block.Uncles(), withdrawals, &receipts, &systemTxs, usedGas)
 	if err != nil {
 		return statedb, receipts, allLogs, *usedGas, err
@@ -260,6 +231,13 @@ func ApplyTransaction(config *params.ChainConfig, bc ChainContext, author *commo
 // ProcessBeaconBlockRoot applies the EIP-4788 system call to the beacon block root
 // contract. This method is exported to be used in tests.
 func ProcessBeaconBlockRoot(beaconRoot common.Hash, vmenv *vm.EVM, statedb *state.StateDB) {
+	// Return immediately if beaconRoot equals the zero hash when using the Parlia engine.
+	if beaconRoot == (common.Hash{}) {
+		if chainConfig := vmenv.ChainConfig(); chainConfig != nil && chainConfig.Parlia != nil {
+			return
+		}
+	}
+
 	// If EIP-4788 is enabled, we need to invoke the beaconroot storage contract with
 	// the new root
 	msg := &Message{

core/state_transition.go

@@ -367,8 +367,6 @@ func (st *StateTransition) preCheck() error {
 // However if any consensus issue encountered, return the error directly with
 // nil evm execution result.
 func (st *StateTransition) TransitionDb() (*ExecutionResult, error) {
-	// start record rw set in here
-	st.state.BeforeTxTransition()
 	// First check this message satisfies all consensus rules before
 	// applying the message. The rules include these clauses
 	//
@@ -448,10 +446,6 @@ func (st *StateTransition) TransitionDb() (*ExecutionResult, error) {
 		ret, st.gasRemaining, vmerr = st.evm.Call(sender, st.to(), msg.Data, st.gasRemaining, value)
 	}
 
-	// stop record rw set in here
-	if err := st.state.FinaliseRWSet(); err != nil {
-		return nil, err
-	}
 	var gasRefund uint64
 	if !rules.IsLondon {
 		// Before EIP-3529: refunds were capped to gasUsed / 2

core/systemcontracts/bohr/types.go

@@ -0,0 +1,27 @@
+package bohr
+
+import _ "embed"
+
+// contract codes for Mainnet upgrade
+var (
+	//go:embed mainnet/ValidatorContract
+	MainnetValidatorContract string
+	//go:embed mainnet/StakeHubContract
+	MainnetStakeHubContract string
+)
+
+// contract codes for Chapel upgrade
+var (
+	//go:embed chapel/ValidatorContract
+	ChapelValidatorContract string
+	//go:embed chapel/StakeHubContract
+	ChapelStakeHubContract string
+)
+
+// contract codes for Rialto upgrade
+var (
+	//go:embed rialto/ValidatorContract
+	RialtoValidatorContract string
+	//go:embed rialto/StakeHubContract
+	RialtoStakeHubContract string
+)

core/systemcontracts/haber_fix/types.go

@@ -0,0 +1,19 @@
+package haber_fix
+
+import _ "embed"
+
+// contract codes for Chapel upgrade
+var (
+	//go:embed chapel/ValidatorContract
+	ChapelValidatorContract string
+	//go:embed chapel/SlashContract
+	ChapelSlashContract string
+)
+
+// contract codes for Mainnet upgrade
+var (
+	//go:embed mainnet/ValidatorContract
+	MainnetValidatorContract string
+	//go:embed mainnet/SlashContract
+	MainnetSlashContract string
+)

core/systemcontracts/upgrade.go

@@ -4,17 +4,17 @@ import (
 	"encoding/hex"
 	"fmt"
 	"math/big"
+	"reflect"
+	"strings"
 
 	"github.com/ethereum/go-ethereum/common"
-	"github.com/ethereum/go-ethereum/core/state"
-	"github.com/ethereum/go-ethereum/log"
-	"github.com/ethereum/go-ethereum/params"
-
+	"github.com/ethereum/go-ethereum/core/systemcontracts/bohr"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/bruno"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/euler"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/feynman"
 	feynmanFix "github.com/ethereum/go-ethereum/core/systemcontracts/feynman_fix"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/gibbs"
+	haberFix "github.com/ethereum/go-ethereum/core/systemcontracts/haber_fix"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/kepler"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/luban"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/mirror"
@@ -23,6 +23,9 @@ import (
 	"github.com/ethereum/go-ethereum/core/systemcontracts/planck"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/plato"
 	"github.com/ethereum/go-ethereum/core/systemcontracts/ramanujan"
+	"github.com/ethereum/go-ethereum/core/vm"
+	"github.com/ethereum/go-ethereum/log"
+	"github.com/ethereum/go-ethereum/params"
 )
 
 type UpgradeConfig struct {
@@ -38,7 +41,7 @@ type Upgrade struct {
 	Configs     []*UpgradeConfig
 }
 
-type upgradeHook func(blockNumber *big.Int, contractAddr common.Address, statedb *state.StateDB) error
+type upgradeHook func(blockNumber *big.Int, contractAddr common.Address, statedb vm.StateDB) error
 
 const (
 	mainNet    = "Mainnet"
@@ -75,6 +78,10 @@ var (
 	feynmanUpgrade = make(map[string]*Upgrade)
 
 	feynmanFixUpgrade = make(map[string]*Upgrade)
+
+	haberFixUpgrade = make(map[string]*Upgrade)
+
+	bohrUpgrade = make(map[string]*Upgrade)
 )
 
 func init() {
@@ -701,12 +708,93 @@ func init() {
 			},
 		},
 	}
+
+	haberFixUpgrade[mainNet] = &Upgrade{
+		UpgradeName: "haberFix",
+		Configs: []*UpgradeConfig{
+			{
+				ContractAddr: common.HexToAddress(ValidatorContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/b743ce3f1f1e94c349b175cd6593bc263463b33b",
+				Code:         haberFix.MainnetValidatorContract,
+			},
+			{
+				ContractAddr: common.HexToAddress(SlashContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/b743ce3f1f1e94c349b175cd6593bc263463b33b",
+				Code:         haberFix.MainnetSlashContract,
+			},
+		},
+	}
+
+	haberFixUpgrade[chapelNet] = &Upgrade{
+		UpgradeName: "haberFix",
+		Configs: []*UpgradeConfig{
+			{
+				ContractAddr: common.HexToAddress(ValidatorContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/b743ce3f1f1e94c349b175cd6593bc263463b33b",
+				Code:         haberFix.ChapelValidatorContract,
+			},
+			{
+				ContractAddr: common.HexToAddress(SlashContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/b743ce3f1f1e94c349b175cd6593bc263463b33b",
+				Code:         haberFix.ChapelSlashContract,
+			},
+		},
+	}
+
+	bohrUpgrade[mainNet] = &Upgrade{
+		UpgradeName: "bohr",
+		Configs: []*UpgradeConfig{
+			{
+				ContractAddr: common.HexToAddress(ValidatorContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/398c9364aad5261c1ecd90ac3ab2df89b65c45e3",
+				Code:         bohr.MainnetValidatorContract,
+			},
+			{
+				ContractAddr: common.HexToAddress(StakeHubContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/398c9364aad5261c1ecd90ac3ab2df89b65c45e3",
+				Code:         bohr.MainnetStakeHubContract,
+			},
+		},
+	}
+
+	bohrUpgrade[chapelNet] = &Upgrade{
+		UpgradeName: "bohr",
+		Configs: []*UpgradeConfig{
+			{
+				ContractAddr: common.HexToAddress(ValidatorContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/398c9364aad5261c1ecd90ac3ab2df89b65c45e3",
+				Code:         bohr.ChapelValidatorContract,
+			},
+			{
+				ContractAddr: common.HexToAddress(StakeHubContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/398c9364aad5261c1ecd90ac3ab2df89b65c45e3",
+				Code:         bohr.ChapelStakeHubContract,
+			},
+		},
+	}
+
+	bohrUpgrade[rialtoNet] = &Upgrade{
+		UpgradeName: "bohr",
+		Configs: []*UpgradeConfig{
+			{
+				ContractAddr: common.HexToAddress(ValidatorContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/398c9364aad5261c1ecd90ac3ab2df89b65c45e3",
+				Code:         bohr.RialtoValidatorContract,
+			},
+			{
+				ContractAddr: common.HexToAddress(StakeHubContract),
+				CommitUrl:    "https://github.com/bnb-chain/bsc-genesis-contract/commit/398c9364aad5261c1ecd90ac3ab2df89b65c45e3",
+				Code:         bohr.RialtoStakeHubContract,
+			},
+		},
+	}
 }
 
-func UpgradeBuildInSystemContract(config *params.ChainConfig, blockNumber *big.Int, lastBlockTime uint64, blockTime uint64, statedb *state.StateDB) {
-	if config == nil || blockNumber == nil || statedb == nil {
+func UpgradeBuildInSystemContract(config *params.ChainConfig, blockNumber *big.Int, lastBlockTime uint64, blockTime uint64, statedb vm.StateDB) {
+	if config == nil || blockNumber == nil || statedb == nil || reflect.ValueOf(statedb).IsNil() {
 		return
 	}
+
 	var network string
 	switch GenesisHash {
 	/* Add mainnet genesis hash */
@@ -777,12 +865,20 @@ func UpgradeBuildInSystemContract(config *params.ChainConfig, blockNumber *big.I
 		applySystemContractUpgrade(feynmanFixUpgrade[network], blockNumber, statedb, logger)
 	}
 
+	if config.IsOnHaberFix(blockNumber, lastBlockTime, blockTime) {
+		applySystemContractUpgrade(haberFixUpgrade[network], blockNumber, statedb, logger)
+	}
+
+	if config.IsOnBohr(blockNumber, lastBlockTime, blockTime) {
+		applySystemContractUpgrade(bohrUpgrade[network], blockNumber, statedb, logger)
+	}
+
 	/*
 		apply other upgrades
 	*/
 }
 
-func applySystemContractUpgrade(upgrade *Upgrade, blockNumber *big.Int, statedb *state.StateDB, logger log.Logger) {
+func applySystemContractUpgrade(upgrade *Upgrade, blockNumber *big.Int, statedb vm.StateDB, logger log.Logger) {
 	if upgrade == nil {
 		logger.Info("Empty upgrade config", "height", blockNumber.String())
 		return
@@ -799,7 +895,7 @@ func applySystemContractUpgrade(upgrade *Upgrade, blockNumber *big.Int, statedb
 			}
 		}
 
-		newContractCode, err := hex.DecodeString(cfg.Code)
+		newContractCode, err := hex.DecodeString(strings.TrimSpace(cfg.Code))
 		if err != nil {
 			panic(fmt.Errorf("failed to decode new contract code: %s", err.Error()))
 		}

core/systemcontracts/upgrade_test.go

@@ -2,9 +2,13 @@ package systemcontracts
 
 import (
 	"crypto/sha256"
+	"math/big"
 	"testing"
 
 	"github.com/ethereum/go-ethereum/common"
+	"github.com/ethereum/go-ethereum/core/state"
+	"github.com/ethereum/go-ethereum/core/vm"
+	"github.com/ethereum/go-ethereum/params"
 	"github.com/stretchr/testify/require"
 )
 
@@ -39,3 +43,31 @@ func TestAllCodesHash(t *testing.T) {
 	allCodeHash := sha256.Sum256(allCodes)
 	require.Equal(t, allCodeHash[:], common.Hex2Bytes("833cc0fc87c46ad8a223e44ccfdc16a51a7e7383525136441bd0c730f06023df"))
 }
+
+func TestUpgradeBuildInSystemContractNilInterface(t *testing.T) {
+	var (
+		config               = params.BSCChainConfig
+		blockNumber          = big.NewInt(37959559)
+		lastBlockTime uint64 = 1713419337
+		blockTime     uint64 = 1713419340
+		statedb       vm.StateDB
+	)
+
+	GenesisHash = params.BSCGenesisHash
+
+	UpgradeBuildInSystemContract(config, blockNumber, lastBlockTime, blockTime, statedb)
+}
+
+func TestUpgradeBuildInSystemContractNilValue(t *testing.T) {
+	var (
+		config                   = params.BSCChainConfig
+		blockNumber              = big.NewInt(37959559)
+		lastBlockTime uint64     = 1713419337
+		blockTime     uint64     = 1713419340
+		statedb       vm.StateDB = (*state.StateDB)(nil)
+	)
+
+	GenesisHash = params.BSCGenesisHash
+
+	UpgradeBuildInSystemContract(config, blockNumber, lastBlockTime, blockTime, statedb)
+}

core/types/bid.go

@@ -193,5 +193,7 @@ type MevParams struct {
 	ValidatorCommission   uint64 // 100 means 1%
 	BidSimulationLeftOver time.Duration
 	GasCeil               uint64
+	GasPrice              *big.Int // Minimum avg gas price for bid block
 	BuilderFeeCeil        *big.Int
+	Version               string
 }

core/types/block.go

@@ -204,8 +204,6 @@ type Body struct {
 	Transactions []*Transaction
 	Uncles       []*Header
 	Withdrawals  []*Withdrawal `rlp:"optional"`
-	// TODO: add TxDAG in block body
-	//TxDAG        *TxDAG        `rlp:"optional"`
 }
 
 // Block represents an Ethereum block.
@@ -675,10 +673,7 @@ type DiffAccountsInBlock struct {
 	Transactions []DiffAccountsInTx
 }
 
-var (
-	extraVanity = 32 // Fixed number of extra-data prefix bytes reserved for signer vanity
-	extraSeal   = 65 // Fixed number of extra-data suffix bytes reserved for signer seal
-)
+var extraSeal = 65 // Fixed number of extra-data suffix bytes reserved for signer seal
 
 // SealHash returns the hash of a block prior to it being sealed.
 func SealHash(header *Header, chainId *big.Int) (hash common.Hash) {
@@ -689,48 +684,51 @@ func SealHash(header *Header, chainId *big.Int) (hash common.Hash) {
 }
 
 func EncodeSigHeader(w io.Writer, header *Header, chainId *big.Int) {
-	err := rlp.Encode(w, []interface{}{
-		chainId,
-		header.ParentHash,
-		header.UncleHash,
-		header.Coinbase,
-		header.Root,
-		header.TxHash,
-		header.ReceiptHash,
-		header.Bloom,
-		header.Difficulty,
-		header.Number,
-		header.GasLimit,
-		header.GasUsed,
-		header.Time,
-		header.Extra[:len(header.Extra)-extraSeal], // this will panic if extra is too short, should check before calling encodeSigHeader
-		header.MixDigest,
-		header.Nonce,
-	})
-	if err != nil {
-		panic("can't encode: " + err.Error())
-	}
-}
-
-func EncodeSigHeaderWithoutVoteAttestation(w io.Writer, header *Header, chainId *big.Int) {
-	err := rlp.Encode(w, []interface{}{
-		chainId,
-		header.ParentHash,
-		header.UncleHash,
-		header.Coinbase,
-		header.Root,
-		header.TxHash,
-		header.ReceiptHash,
-		header.Bloom,
-		header.Difficulty,
-		header.Number,
-		header.GasLimit,
-		header.GasUsed,
-		header.Time,
-		header.Extra[:extraVanity], // this will panic if extra is too short, should check before calling encodeSigHeaderWithoutVoteAttestation
-		header.MixDigest,
-		header.Nonce,
-	})
+	var err error
+	if header.ParentBeaconRoot != nil && *header.ParentBeaconRoot == (common.Hash{}) {
+		err = rlp.Encode(w, []interface{}{
+			chainId,
+			header.ParentHash,
+			header.UncleHash,
+			header.Coinbase,
+			header.Root,
+			header.TxHash,
+			header.ReceiptHash,
+			header.Bloom,
+			header.Difficulty,
+			header.Number,
+			header.GasLimit,
+			header.GasUsed,
+			header.Time,
+			header.Extra[:len(header.Extra)-extraSeal], // this will panic if extra is too short, should check before calling encodeSigHeader
+			header.MixDigest,
+			header.Nonce,
+			header.BaseFee,
+			header.WithdrawalsHash,
+			header.BlobGasUsed,
+			header.ExcessBlobGas,
+			header.ParentBeaconRoot,
+		})
+	} else {
+		err = rlp.Encode(w, []interface{}{
+			chainId,
+			header.ParentHash,
+			header.UncleHash,
+			header.Coinbase,
+			header.Root,
+			header.TxHash,
+			header.ReceiptHash,
+			header.Bloom,
+			header.Difficulty,
+			header.Number,
+			header.GasLimit,
+			header.GasUsed,
+			header.Time,
+			header.Extra[:len(header.Extra)-extraSeal], // this will panic if extra is too short, should check before calling encodeSigHeader
+			header.MixDigest,
+			header.Nonce,
+		})
+	}
 	if err != nil {
 		panic("can't encode: " + err.Error())
 	}

core/types/dag.go

@@ -1,698 +0,0 @@
-package types
-
-import (
-	"encoding/hex"
-	"errors"
-	"fmt"
-	"github.com/ethereum/go-ethereum/common"
-	"github.com/ethereum/go-ethereum/log"
-	"github.com/ethereum/go-ethereum/metrics"
-	"github.com/holiman/uint256"
-	"golang.org/x/exp/slices"
-	"sort"
-	"strings"
-	"sync"
-	"time"
-)
-
-// TxDep store the current tx dependency relation with other txs
-type TxDep struct {
-	// It describes the Relation with below txs
-	// 0: this tx depends on below txs
-	// 1: this transaction does not depend on below txs, all other previous txs depend on
-	Relation  uint8
-	TxIndexes []int
-}
-
-func (d *TxDep) AppendDep(i int) {
-	d.TxIndexes = append(d.TxIndexes, i)
-}
-
-func (d *TxDep) Exist(i int) bool {
-	for _, index := range d.TxIndexes {
-		if index == i {
-			return true
-		}
-	}
-
-	return false
-}
-
-// TxDAG indicate how to use the dependency of txs
-type TxDAG struct {
-	// The TxDAG type
-	// 0: delay the distribution of GasFee, it will ignore all gas fee distribution when tx execute
-	// 1: timely distribution of transaction fees, it will keep partial serial execution when tx cannot delay the distribution
-	Type uint8
-	// Tx Dependency List, the list index is equal to TxIndex
-	TxDeps []TxDep
-	// It indicates the scheduling priority of the transactions
-	SchedulePriority []int
-}
-
-func NewTxDAG(txLen int) *TxDAG {
-	return &TxDAG{
-		Type:   0,
-		TxDeps: make([]TxDep, txLen),
-	}
-}
-
-func (d *TxDAG) String() string {
-	builder := strings.Builder{}
-	exePaths := d.travelExecutionPaths()
-	for _, path := range exePaths {
-		builder.WriteString(fmt.Sprintf("%v\n", path))
-	}
-	return builder.String()
-}
-
-func (d *TxDAG) travelExecutionPaths() [][]int {
-	// regenerate TxDAG
-	nd := NewTxDAG(len(d.TxDeps))
-	for i, txDep := range d.TxDeps {
-		nd.TxDeps[i].Relation = 0
-		if txDep.Relation == 0 {
-			nd.TxDeps[i] = txDep
-			continue
-		}
-
-		// recover to relation 0
-		for j := 0; j < i; j++ {
-			if !txDep.Exist(j) {
-				nd.TxDeps[i].AppendDep(j)
-			}
-		}
-	}
-	exePaths := make([][]int, 0)
-
-	// travel tx deps with BFS
-	for i := 0; i < len(nd.TxDeps); i++ {
-		exePaths = append(exePaths, travelTargetPath(nd.TxDeps, i))
-	}
-	return exePaths
-}
-
-var (
-	longestTimeTimer = metrics.NewRegisteredTimer("dag/longesttime", nil)
-	longestGasTimer  = metrics.NewRegisteredTimer("dag/longestgas", nil)
-	serialTimeTimer  = metrics.NewRegisteredTimer("dag/serialtime", nil)
-	totalTxMeter     = metrics.NewRegisteredMeter("dag/txcnt", nil)
-	totalNoDepMeter  = metrics.NewRegisteredMeter("dag/nodepcntcnt", nil)
-	total2DepMeter   = metrics.NewRegisteredMeter("dag/2depcntcnt", nil)
-	total4DepMeter   = metrics.NewRegisteredMeter("dag/4depcntcnt", nil)
-	total8DepMeter   = metrics.NewRegisteredMeter("dag/8depcntcnt", nil)
-	total16DepMeter  = metrics.NewRegisteredMeter("dag/16depcntcnt", nil)
-	total32DepMeter  = metrics.NewRegisteredMeter("dag/32depcntcnt", nil)
-)
-
-func EvaluateTxDAGPerformance(dag *TxDAG, stats []*ExeStat) string {
-	if len(stats) != len(dag.TxDeps) || len(dag.TxDeps) == 0 {
-		return ""
-	}
-	sb := strings.Builder{}
-	sb.WriteString("TxDAG:\n")
-	for i, dep := range dag.TxDeps {
-		if stats[i].mustSerialFlag {
-			continue
-		}
-		sb.WriteString(fmt.Sprintf("%v: %v\n", i, dep.TxIndexes))
-	}
-	sb.WriteString("Parallel Execution Path:\n")
-	paths := dag.travelExecutionPaths()
-	// Attention: this is based on best schedule, it will reduce a lot by executing previous txs in parallel
-	// It assumes that there is no parallel thread limit
-	var (
-		maxGasIndex     int
-		maxGas          uint64
-		maxTimeIndex    int
-		maxTime         time.Duration
-		txTimes         = make([]time.Duration, len(dag.TxDeps))
-		txGases         = make([]uint64, len(dag.TxDeps))
-		txReads         = make([]int, len(dag.TxDeps))
-		noDepdencyCount int
-	)
-
-	totalTxMeter.Mark(int64(len(dag.TxDeps)))
-	for i, path := range paths {
-		if stats[i].mustSerialFlag {
-			continue
-		}
-		if len(path) <= 1 {
-			noDepdencyCount++
-			totalNoDepMeter.Mark(1)
-		}
-		if len(path) <= 3 {
-			total2DepMeter.Mark(1)
-		}
-		if len(path) <= 5 {
-			total4DepMeter.Mark(1)
-		}
-		if len(path) <= 9 {
-			total8DepMeter.Mark(1)
-		}
-		if len(path) <= 17 {
-			total16DepMeter.Mark(1)
-		}
-		if len(path) <= 33 {
-			total32DepMeter.Mark(1)
-		}
-
-		// find the biggest cost time from dependency txs
-		for j := 0; j < len(path)-1; j++ {
-			prev := path[j]
-			if txTimes[prev] > txTimes[i] {
-				txTimes[i] = txTimes[prev]
-			}
-			if txGases[prev] > txGases[i] {
-				txGases[i] = txGases[prev]
-			}
-			if txReads[prev] > txReads[i] {
-				txReads[i] = txReads[prev]
-			}
-		}
-		txTimes[i] += stats[i].costTime
-		txGases[i] += stats[i].usedGas
-		txReads[i] += stats[i].readCount
-
-		//sb.WriteString(fmt.Sprintf("Tx%v, %.2fms|%vgas|%vreads\npath: %v\n", i, float64(txTimes[i].Microseconds())/1000, txGases[i], txReads[i], path))
-		sb.WriteString(fmt.Sprintf("%v: %v\n", i, path))
-		// try to find max gas
-		if txGases[i] > maxGas {
-			maxGas = txGases[i]
-			maxGasIndex = i
-		}
-		if txTimes[i] > maxTime {
-			maxTime = txTimes[i]
-			maxTimeIndex = i
-		}
-	}
-
-	sb.WriteString(fmt.Sprintf("LargestGasPath: %.2fms|%vgas|%vreads\npath: %v\n", float64(txTimes[maxGasIndex].Microseconds())/1000, txGases[maxGasIndex], txReads[maxGasIndex], paths[maxGasIndex]))
-	sb.WriteString(fmt.Sprintf("LongestTimePath: %.2fms|%vgas|%vreads\npath: %v\n", float64(txTimes[maxTimeIndex].Microseconds())/1000, txGases[maxTimeIndex], txReads[maxTimeIndex], paths[maxTimeIndex]))
-	longestTimeTimer.Update(txTimes[maxTimeIndex])
-	longestGasTimer.Update(txTimes[maxGasIndex])
-	// serial path
-	var (
-		sTime time.Duration
-		sGas  uint64
-		sRead int
-		sPath []int
-	)
-	for i, stat := range stats {
-		if stat.mustSerialFlag {
-			continue
-		}
-		sPath = append(sPath, i)
-		sTime += stat.costTime
-		sGas += stat.usedGas
-		sRead += stat.readCount
-	}
-	if sTime == 0 {
-		return ""
-	}
-	sb.WriteString(fmt.Sprintf("SerialPath: %.2fms|%vgas|%vreads\npath: %v\n", float64(sTime.Microseconds())/1000, sGas, sRead, sPath))
-	maxParaTime := txTimes[maxTimeIndex]
-	sb.WriteString(fmt.Sprintf("Estimated saving: %.2fms, %.2f%%, %.2fX, noDepCnt: %v|%.2f%%\n",
-		float64((sTime-maxParaTime).Microseconds())/1000, float64(sTime-maxParaTime)/float64(sTime)*100,
-		float64(sTime)/float64(maxParaTime), noDepdencyCount, float64(noDepdencyCount)/float64(len(dag.TxDeps))*100))
-	serialTimeTimer.Update(sTime)
-	return sb.String()
-}
-
-func travelTargetPath(deps []TxDep, from int) []int {
-	q := make([]int, 0, len(deps))
-	path := make([]int, 0, len(deps))
-
-	q = append(q, from)
-	path = append(path, from)
-	for len(q) > 0 {
-		t := make([]int, 0, len(deps))
-		for _, i := range q {
-			for _, dep := range deps[i].TxIndexes {
-				if !slices.Contains(path, dep) {
-					path = append(path, dep)
-					t = append(t, dep)
-				}
-			}
-		}
-		q = t
-	}
-	sort.Ints(path)
-	return path
-}
-
-type ValidatorExtraItem struct {
-	ValidatorAddress common.Address
-	VoteAddress      BLSPublicKey
-}
-
-type HeaderCustomExtra struct {
-	ValidatorSet ValidatorExtraItem
-	TxDAG        TxDAG
-}
-
-// StateVersion record specific TxIndex & TxIncarnation
-// if TxIndex equals to -1, it means the state read from DB.
-type StateVersion struct {
-	TxIndex       int
-	TxIncarnation int
-}
-
-// ReadRecord keep read value & its version
-type ReadRecord struct {
-	StateVersion
-	Val interface{}
-}
-
-// WriteRecord keep latest state value & change count
-type WriteRecord struct {
-	Val interface{}
-}
-
-// RWSet record all read & write set in txs
-// Attention: this is not a concurrent safety structure
-type RWSet struct {
-	ver      StateVersion
-	readSet  map[RWKey]*ReadRecord
-	writeSet map[RWKey]*WriteRecord
-
-	// some flags
-	mustSerial bool
-}
-
-func NewRWSet(ver StateVersion) *RWSet {
-	return &RWSet{
-		ver:      ver,
-		readSet:  make(map[RWKey]*ReadRecord),
-		writeSet: make(map[RWKey]*WriteRecord),
-	}
-}
-
-func (s *RWSet) RecordRead(key RWKey, ver StateVersion, val interface{}) {
-	// only record the first read version
-	if _, exist := s.readSet[key]; exist {
-		return
-	}
-	s.readSet[key] = &ReadRecord{
-		StateVersion: ver,
-		Val:          val,
-	}
-}
-
-func (s *RWSet) RecordWrite(key RWKey, val interface{}) {
-	wr, exist := s.writeSet[key]
-	if !exist {
-		s.writeSet[key] = &WriteRecord{
-			Val: val,
-		}
-		return
-	}
-	wr.Val = val
-}
-
-func (s *RWSet) Version() StateVersion {
-	return s.ver
-}
-
-func (s *RWSet) ReadSet() map[RWKey]*ReadRecord {
-	return s.readSet
-}
-
-func (s *RWSet) WriteSet() map[RWKey]*WriteRecord {
-	return s.writeSet
-}
-
-func (s *RWSet) WithSerialFlag() *RWSet {
-	s.mustSerial = true
-	return s
-}
-
-func (s *RWSet) String() string {
-	builder := strings.Builder{}
-	builder.WriteString(fmt.Sprintf("tx: %v, inc: %v\nreadSet: [", s.ver.TxIndex, s.ver.TxIncarnation))
-	i := 0
-	for key, _ := range s.readSet {
-		if i > 0 {
-			builder.WriteString(fmt.Sprintf(", %v", key.String()))
-			continue
-		}
-		builder.WriteString(fmt.Sprintf("%v", key.String()))
-		i++
-	}
-	builder.WriteString("]\nwriteSet: [")
-	i = 0
-	for key, _ := range s.writeSet {
-		if i > 0 {
-			builder.WriteString(fmt.Sprintf(", %v", key.String()))
-			continue
-		}
-		builder.WriteString(fmt.Sprintf("%v", key.String()))
-		i++
-	}
-	builder.WriteString("]\n")
-	return builder.String()
-}
-
-const (
-	AccountStatePrefix = 'a'
-	StorageStatePrefix = 's'
-)
-
-type RWKey [1 + common.AddressLength + common.HashLength]byte
-
-type AccountState byte
-
-const (
-	AccountSelf AccountState = iota
-	AccountNonce
-	AccountBalance
-	AccountCodeHash
-	AccountSuicide
-)
-
-func AccountStateKey(account common.Address, state AccountState) RWKey {
-	var key RWKey
-	key[0] = AccountStatePrefix
-	copy(key[1:], account.Bytes())
-	key[1+common.AddressLength] = byte(state)
-	return key
-}
-
-func StorageStateKey(account common.Address, state common.Hash) RWKey {
-	var key RWKey
-	key[0] = StorageStatePrefix
-	copy(key[1:], account.Bytes())
-	copy(key[1+common.AddressLength:], state.Bytes())
-	return key
-}
-
-func (key *RWKey) IsAccountState() (bool, AccountState) {
-	return AccountStatePrefix == key[0], AccountState(key[1+common.AddressLength])
-}
-
-func (key *RWKey) IsAccountSelf() bool {
-	ok, s := key.IsAccountState()
-	if !ok {
-		return false
-	}
-	return s == AccountSelf
-}
-
-func (key *RWKey) IsAccountSuicide() bool {
-	ok, s := key.IsAccountState()
-	if !ok {
-		return false
-	}
-	return s == AccountSuicide
-}
-
-func (key *RWKey) ToAccountSelf() RWKey {
-	return AccountStateKey(key.Addr(), AccountSelf)
-}
-
-func (key *RWKey) IsStorageState() bool {
-	return StorageStatePrefix == key[0]
-}
-
-func (key *RWKey) String() string {
-	return hex.EncodeToString(key[:])
-}
-
-func (key *RWKey) Addr() common.Address {
-	return common.BytesToAddress(key[1 : 1+common.AddressLength])
-}
-
-type PendingWrite struct {
-	Ver StateVersion
-	Val interface{}
-}
-
-func NewPendingWrite(ver StateVersion, wr *WriteRecord) *PendingWrite {
-	return &PendingWrite{
-		Ver: ver,
-		Val: wr.Val,
-	}
-}
-
-func (w *PendingWrite) TxIndex() int {
-	return w.Ver.TxIndex
-}
-
-func (w *PendingWrite) TxIncarnation() int {
-	return w.Ver.TxIncarnation
-}
-
-type PendingWrites struct {
-	list []*PendingWrite
-}
-
-func NewPendingWrites() *PendingWrites {
-	return &PendingWrites{
-		list: make([]*PendingWrite, 0),
-	}
-}
-
-func (w *PendingWrites) Append(pw *PendingWrite) {
-	if i, found := w.SearchTxIndex(pw.TxIndex()); found {
-		w.list[i] = pw
-		return
-	}
-
-	w.list = append(w.list, pw)
-	for i := len(w.list) - 1; i > 0; i-- {
-		if w.list[i].TxIndex() > w.list[i-1].TxIndex() {
-			break
-		}
-		w.list[i-1], w.list[i] = w.list[i], w.list[i-1]
-	}
-}
-
-func (w *PendingWrites) SearchTxIndex(txIndex int) (int, bool) {
-	n := len(w.list)
-	i, j := 0, n
-	for i < j {
-		h := int(uint(i+j) >> 1)
-		// i ≤ h < j
-		if w.list[h].TxIndex() < txIndex {
-			i = h + 1
-		} else {
-			j = h
-		}
-	}
-	return i, i < n && w.list[i].TxIndex() == txIndex
-}
-
-func (w *PendingWrites) FindLastWrite(txIndex int) *PendingWrite {
-	var i, _ = w.SearchTxIndex(txIndex)
-	for j := i - 1; j >= 0; j-- {
-		if w.list[j].TxIndex() < txIndex {
-			return w.list[j]
-		}
-	}
-
-	return nil
-}
-
-type MVStates struct {
-	rwSets          []*RWSet
-	stats           []*ExeStat
-	pendingWriteSet map[RWKey]*PendingWrites
-	lock            sync.RWMutex
-}
-
-func NewMVStates(txCount int) *MVStates {
-	return &MVStates{
-		rwSets:          make([]*RWSet, txCount),
-		stats:           make([]*ExeStat, txCount),
-		pendingWriteSet: make(map[RWKey]*PendingWrites, txCount*8),
-	}
-}
-
-func (s *MVStates) RWSets() []*RWSet {
-	s.lock.RLock()
-	defer s.lock.RUnlock()
-	return s.rwSets
-}
-
-func (s *MVStates) Stats() []*ExeStat {
-	s.lock.RLock()
-	defer s.lock.RUnlock()
-	return s.stats
-}
-
-func (s *MVStates) RWSet(index int) *RWSet {
-	s.lock.RLock()
-	defer s.lock.RUnlock()
-	return s.rwSets[index]
-}
-
-func (s *MVStates) FulfillRWSet(rwSet *RWSet, stat *ExeStat) error {
-	s.lock.Lock()
-	defer s.lock.Unlock()
-	index := rwSet.ver.TxIndex
-	if index >= len(s.rwSets) {
-		return errors.New("refill out of bound")
-	}
-	if s := s.rwSets[index]; s != nil {
-		return errors.New("refill a exist RWSet")
-	}
-	if stat != nil {
-		if stat.txIndex != index {
-			return errors.New("wrong execution stat")
-		}
-		s.stats[index] = stat
-	}
-
-	for k, v := range rwSet.writeSet {
-		// ignore no changed write record
-		checkRWSetInconsistent(index, k, rwSet.readSet, rwSet.writeSet)
-		// this will be handled by state object
-		//if rwSet.readSet[k] != nil && isEqualRWVal(k, rwSet.readSet[k].Val, v.Val) {
-		//	delete(rwSet.writeSet, k)
-		//	continue
-		//}
-		if _, exist := s.pendingWriteSet[k]; !exist {
-			s.pendingWriteSet[k] = NewPendingWrites()
-		}
-		s.pendingWriteSet[k].Append(NewPendingWrite(rwSet.ver, v))
-	}
-	s.rwSets[index] = rwSet
-	return nil
-}
-
-func checkRWSetInconsistent(index int, k RWKey, readSet map[RWKey]*ReadRecord, writeSet map[RWKey]*WriteRecord) bool {
-	var (
-		readOk  bool
-		writeOk bool
-		r       *WriteRecord
-	)
-
-	if k.IsAccountSuicide() {
-		_, readOk = readSet[k.ToAccountSelf()]
-	} else {
-		_, readOk = readSet[k]
-	}
-
-	r, writeOk = writeSet[k]
-	if readOk != writeOk {
-		// check if it's correct? read nil, write non-nil
-		log.Info("checkRWSetInconsistent find inconsistent", "tx", index, "k", k.String(), "read", readOk, "write", writeOk, "val", r.Val)
-		return true
-	}
-
-	return false
-}
-
-func isEqualRWVal(key RWKey, src interface{}, compared interface{}) bool {
-	if ok, state := key.IsAccountState(); ok {
-		switch state {
-		case AccountBalance:
-			if src != nil && compared != nil {
-				return equalUint256(src.(*uint256.Int), compared.(*uint256.Int))
-			}
-			return src == compared
-		case AccountNonce:
-			return src.(uint64) == compared.(uint64)
-		case AccountCodeHash:
-			if src != nil && compared != nil {
-				return slices.Equal(src.([]byte), compared.([]byte))
-			}
-			return src == compared
-		}
-		return false
-	}
-
-	if src != nil && compared != nil {
-		return src.(common.Hash) == compared.(common.Hash)
-	}
-	return src == compared
-}
-
-func equalUint256(s, c *uint256.Int) bool {
-	if s != nil && c != nil {
-		return s.Eq(c)
-	}
-
-	return s == c
-}
-
-func (s *MVStates) ResolveTxDAG() *TxDAG {
-	rwSets := s.RWSets()
-	txDAG := NewTxDAG(len(rwSets))
-	for i := len(rwSets) - 1; i >= 0; i-- {
-		txDAG.TxDeps[i].TxIndexes = []int{}
-		if rwSets[i].mustSerial {
-			txDAG.TxDeps[i].Relation = 1
-			continue
-		}
-		readSet := rwSets[i].ReadSet()
-		// TODO: check if there are RW with system address
-		// check if there has written op before i
-		for j := 0; j < i; j++ {
-			if checkDependency(rwSets[j].writeSet, readSet) {
-				txDAG.TxDeps[i].AppendDep(j)
-			}
-		}
-	}
-
-	return txDAG
-}
-
-func checkDependency(writeSet map[RWKey]*WriteRecord, readSet map[RWKey]*ReadRecord) bool {
-	// check tx dependency, only check key, skip version
-	for k, _ := range writeSet {
-		// check suicide, add read address flag, it only for check suicide quickly, and cannot for other scenarios.
-		if k.IsAccountSuicide() {
-			if _, ok := readSet[k.ToAccountSelf()]; ok {
-				return true
-			}
-			continue
-		}
-		if _, ok := readSet[k]; ok {
-			return true
-		}
-	}
-
-	return false
-}
-
-type ExeStat struct {
-	txIndex   int
-	usedGas   uint64
-	readCount int
-	startTime time.Time
-	costTime  time.Duration
-	// TODO: consider system tx, gas fee issues, may need to use different flag
-	mustSerialFlag bool
-}
-
-func NewExeStat(txIndex int) *ExeStat {
-	return &ExeStat{
-		txIndex: txIndex,
-	}
-}
-
-func (s *ExeStat) Begin() *ExeStat {
-	s.startTime = time.Now()
-	return s
-}
-
-func (s *ExeStat) Done() *ExeStat {
-	s.costTime = time.Since(s.startTime)
-	return s
-}
-
-func (s *ExeStat) WithSerialFlag() *ExeStat {
-	s.mustSerialFlag = true
-	return s
-}
-
-func (s *ExeStat) WithGas(gas uint64) *ExeStat {
-	s.usedGas = gas
-	return s
-}
-
-func (s *ExeStat) WithRead(rc int) *ExeStat {
-	s.readCount = rc
-	return s
-}

core/types/dag_test.go

@@ -1,239 +0,0 @@
-package types
-
-import (
-	"github.com/ethereum/go-ethereum/common"
-	"github.com/holiman/uint256"
-	"github.com/stretchr/testify/require"
-	"testing"
-)
-
-var (
-	mockAddr = common.HexToAddress("0x482bA86399ab6Dcbe54071f8d22258688B4509b1")
-	mockHash = common.HexToHash("0xdc13f8d7bdb8ec4de02cd4a50a1aa2ab73ec8814e0cdb550341623be3dd8ab7a")
-)
-
-func TestTxDAG(t *testing.T) {
-	dag := mockSimpleDAG()
-	t.Log(dag.String())
-	dag = mockSystemTxDAG()
-	t.Log(dag.String())
-}
-
-func TestEvaluateTxDAG(t *testing.T) {
-	dag := mockSystemTxDAG()
-	stats := make([]*ExeStat, len(dag.TxDeps))
-	for i, dep := range dag.TxDeps {
-		stats[i] = NewExeStat(i).WithGas(uint64(i)).WithRead(i)
-		stats[i].costTime = int64(i)
-		if dep.Relation == 1 {
-			stats[i].WithSerialFlag()
-		}
-	}
-	t.Log(EvaluateTxDAGPerformance(dag, stats))
-}
-
-func TestSimpleMVStates2TxDAG(t *testing.T) {
-	ms := NewMVStates(10)
-
-	ms.rwSets[0] = mockRWSet(0, []string{"0x00"}, []string{"0x00"})
-	ms.rwSets[1] = mockRWSet(1, []string{"0x01"}, []string{"0x01"})
-	ms.rwSets[2] = mockRWSet(2, []string{"0x02"}, []string{"0x02"})
-	ms.rwSets[3] = mockRWSet(3, []string{"0x00", "0x03"}, []string{"0x03"})
-	ms.rwSets[4] = mockRWSet(4, []string{"0x00", "0x04"}, []string{"0x04"})
-	ms.rwSets[5] = mockRWSet(5, []string{"0x01", "0x02", "0x05"}, []string{"0x05"})
-	ms.rwSets[6] = mockRWSet(6, []string{"0x02", "0x05", "0x06"}, []string{"0x06"})
-	ms.rwSets[7] = mockRWSet(7, []string{"0x06", "0x07"}, []string{"0x07"})
-	ms.rwSets[8] = mockRWSet(8, []string{"0x08"}, []string{"0x08"})
-	ms.rwSets[9] = mockRWSet(9, []string{"0x08", "0x09"}, []string{"0x09"})
-
-	dag := ms.ResolveTxDAG()
-	require.Equal(t, mockSimpleDAG(), dag)
-	t.Log(dag.String())
-}
-
-func TestSystemTxMVStates2TxDAG(t *testing.T) {
-	ms := NewMVStates(12)
-
-	ms.rwSets[0] = mockRWSet(0, []string{"0x00"}, []string{"0x00"})
-	ms.rwSets[1] = mockRWSet(1, []string{"0x01"}, []string{"0x01"})
-	ms.rwSets[2] = mockRWSet(2, []string{"0x02"}, []string{"0x02"})
-	ms.rwSets[3] = mockRWSet(3, []string{"0x00", "0x03"}, []string{"0x03"})
-	ms.rwSets[4] = mockRWSet(4, []string{"0x00", "0x04"}, []string{"0x04"})
-	ms.rwSets[5] = mockRWSet(5, []string{"0x01", "0x02", "0x05"}, []string{"0x05"})
-	ms.rwSets[6] = mockRWSet(6, []string{"0x02", "0x05", "0x06"}, []string{"0x06"})
-	ms.rwSets[7] = mockRWSet(7, []string{"0x06", "0x07"}, []string{"0x07"})
-	ms.rwSets[8] = mockRWSet(8, []string{"0x08"}, []string{"0x08"})
-	ms.rwSets[9] = mockRWSet(9, []string{"0x08", "0x09"}, []string{"0x09"})
-	ms.rwSets[10] = mockRWSet(10, []string{"0x10"}, []string{"0x10"}).WithSerialFlag()
-	ms.rwSets[11] = mockRWSet(11, []string{"0x11"}, []string{"0x11"}).WithSerialFlag()
-
-	dag := ms.ResolveTxDAG()
-	require.Equal(t, mockSystemTxDAG(), dag)
-	t.Log(dag.String())
-}
-
-func TestIsEqualRWVal(t *testing.T) {
-	tests := []struct {
-		key      RWKey
-		src      interface{}
-		compared interface{}
-		isEqual  bool
-	}{
-		{
-			key:      AccountStateKey(mockAddr, AccountNonce),
-			src:      uint64(0),
-			compared: uint64(0),
-			isEqual:  true,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountNonce),
-			src:      uint64(0),
-			compared: uint64(1),
-			isEqual:  false,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountBalance),
-			src:      new(uint256.Int).SetUint64(1),
-			compared: new(uint256.Int).SetUint64(1),
-			isEqual:  true,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountBalance),
-			src:      nil,
-			compared: new(uint256.Int).SetUint64(1),
-			isEqual:  false,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountBalance),
-			src:      (*uint256.Int)(nil),
-			compared: new(uint256.Int).SetUint64(1),
-			isEqual:  false,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountBalance),
-			src:      (*uint256.Int)(nil),
-			compared: (*uint256.Int)(nil),
-			isEqual:  true,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountCodeHash),
-			src:      []byte{1},
-			compared: []byte{1},
-			isEqual:  true,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountCodeHash),
-			src:      nil,
-			compared: []byte{1},
-			isEqual:  false,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountCodeHash),
-			src:      ([]byte)(nil),
-			compared: []byte{1},
-			isEqual:  false,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountCodeHash),
-			src:      ([]byte)(nil),
-			compared: ([]byte)(nil),
-			isEqual:  true,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountSuicide),
-			src:      struct{}{},
-			compared: struct{}{},
-			isEqual:  false,
-		},
-		{
-			key:      AccountStateKey(mockAddr, AccountSuicide),
-			src:      nil,
-			compared: struct{}{},
-			isEqual:  false,
-		},
-		{
-			key:      StorageStateKey(mockAddr, mockHash),
-			src:      mockHash,
-			compared: mockHash,
-			isEqual:  true,
-		},
-		{
-			key:      StorageStateKey(mockAddr, mockHash),
-			src:      nil,
-			compared: mockHash,
-			isEqual:  false,
-		},
-	}
-
-	for i, item := range tests {
-		require.Equal(t, item.isEqual, isEqualRWVal(item.key, item.src, item.compared), i)
-	}
-}
-
-func mockSimpleDAG() *TxDAG {
-	dag := NewTxDAG(10)
-	dag.TxDeps[0].TxIndexes = []int{}
-	dag.TxDeps[1].TxIndexes = []int{}
-	dag.TxDeps[2].TxIndexes = []int{}
-	dag.TxDeps[3].TxIndexes = []int{0}
-	dag.TxDeps[4].TxIndexes = []int{0}
-	dag.TxDeps[5].TxIndexes = []int{1, 2}
-	dag.TxDeps[6].TxIndexes = []int{2, 5}
-	dag.TxDeps[7].TxIndexes = []int{6}
-	dag.TxDeps[8].TxIndexes = []int{}
-	dag.TxDeps[9].TxIndexes = []int{8}
-	return dag
-}
-
-func mockSystemTxDAG() *TxDAG {
-	dag := NewTxDAG(12)
-	dag.TxDeps[0].TxIndexes = []int{}
-	dag.TxDeps[1].TxIndexes = []int{}
-	dag.TxDeps[2].TxIndexes = []int{}
-	dag.TxDeps[3].TxIndexes = []int{0}
-	dag.TxDeps[4].TxIndexes = []int{0}
-	dag.TxDeps[5].TxIndexes = []int{1, 2}
-	dag.TxDeps[6].TxIndexes = []int{2, 5}
-	dag.TxDeps[7].TxIndexes = []int{6}
-	dag.TxDeps[8].TxIndexes = []int{}
-	dag.TxDeps[9].TxIndexes = []int{8}
-	dag.TxDeps[10] = TxDep{
-		Relation:  1,
-		TxIndexes: []int{},
-	}
-	dag.TxDeps[11] = TxDep{
-		Relation:  1,
-		TxIndexes: []int{},
-	}
-	return dag
-}
-
-func mockRWSet(index int, read []string, write []string) *RWSet {
-	ver := StateVersion{
-		TxIndex: index,
-	}
-	set := NewRWSet(ver)
-	for _, k := range read {
-		key := RWKey{}
-		if len(k) > len(key) {
-			k = k[:len(key)]
-		}
-		copy(key[:], k)
-		set.readSet[key] = &ReadRecord{
-			StateVersion: ver,
-			Val:          struct{}{},
-		}
-	}
-	for _, k := range write {
-		key := RWKey{}
-		if len(k) > len(key) {
-			k = k[:len(key)]
-		}
-		copy(key[:], k)
-		set.writeSet[key] = &WriteRecord{
-			Val: struct{}{},
-		}
-	}
-
-	return set
-}

core/vm/interface.go

@@ -79,10 +79,6 @@ type StateDB interface {
 
 	AddLog(*types.Log)
 	AddPreimage(common.Hash, []byte)
-
-	// parallel DAG related
-	BeforeTxTransition()
-	FinaliseRWSet() error
 }
 
 // CallContext provides a basic interface for the EVM calling conventions. The EVM

core/vote/vote_manager.go

@@ -3,10 +3,13 @@ package vote
 import (
 	"bytes"
 	"fmt"
+	"math/big"
 	"sync"
+	"time"
 
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/consensus"
+	"github.com/ethereum/go-ethereum/consensus/parlia"
 	"github.com/ethereum/go-ethereum/core"
 	"github.com/ethereum/go-ethereum/core/types"
 	"github.com/ethereum/go-ethereum/eth/downloader"
@@ -17,7 +20,13 @@ import (
 
 const blocksNumberSinceMining = 5 // the number of blocks need to wait before voting, counting from the validator begin to mine
 
+var diffInTurn = big.NewInt(2) // Block difficulty for in-turn signatures
 var votesManagerCounter = metrics.NewRegisteredCounter("votesManager/local", nil)
+var notJustified = metrics.NewRegisteredCounter("votesManager/notJustified", nil)
+var inTurnJustified = metrics.NewRegisteredCounter("votesManager/inTurnJustified", nil)
+var notInTurnJustified = metrics.NewRegisteredCounter("votesManager/notInTurnJustified", nil)
+var continuousJustified = metrics.NewRegisteredCounter("votesManager/continuousJustified", nil)
+var notContinuousJustified = metrics.NewRegisteredCounter("votesManager/notContinuousJustified", nil)
 
 // Backend wraps all methods required for voting.
 type Backend interface {
@@ -31,8 +40,8 @@ type VoteManager struct {
 
 	chain *core.BlockChain
 
-	chainHeadCh  chan core.ChainHeadEvent
-	chainHeadSub event.Subscription
+	highestVerifiedBlockCh  chan core.HighestVerifiedBlockEvent
+	highestVerifiedBlockSub event.Subscription
 
 	// used for backup validators to sync votes from corresponding mining validator
 	syncVoteCh  chan core.NewVoteEvent
@@ -47,12 +56,12 @@ type VoteManager struct {
 
 func NewVoteManager(eth Backend, chain *core.BlockChain, pool *VotePool, journalPath, blsPasswordPath, blsWalletPath string, engine consensus.PoSA) (*VoteManager, error) {
 	voteManager := &VoteManager{
-		eth:         eth,
-		chain:       chain,
-		chainHeadCh: make(chan core.ChainHeadEvent, chainHeadChanSize),
-		syncVoteCh:  make(chan core.NewVoteEvent, voteBufferForPut),
-		pool:        pool,
-		engine:      engine,
+		eth:                    eth,
+		chain:                  chain,
+		highestVerifiedBlockCh: make(chan core.HighestVerifiedBlockEvent, highestVerifiedBlockChanSize),
+		syncVoteCh:             make(chan core.NewVoteEvent, voteBufferForPut),
+		pool:                   pool,
+		engine:                 engine,
 	}
 
 	// Create voteSigner.
@@ -72,7 +81,7 @@ func NewVoteManager(eth Backend, chain *core.BlockChain, pool *VotePool, journal
 	voteManager.journal = voteJournal
 
 	// Subscribe to chain head event.
-	voteManager.chainHeadSub = voteManager.chain.SubscribeChainHeadEvent(voteManager.chainHeadCh)
+	voteManager.highestVerifiedBlockSub = voteManager.chain.SubscribeHighestVerifiedHeaderEvent(voteManager.highestVerifiedBlockCh)
 	voteManager.syncVoteSub = voteManager.pool.SubscribeNewVoteEvent(voteManager.syncVoteCh)
 
 	go voteManager.loop()
@@ -82,7 +91,7 @@ func NewVoteManager(eth Backend, chain *core.BlockChain, pool *VotePool, journal
 
 func (voteManager *VoteManager) loop() {
 	log.Debug("vote manager routine loop started")
-	defer voteManager.chainHeadSub.Unsubscribe()
+	defer voteManager.highestVerifiedBlockSub.Unsubscribe()
 	defer voteManager.syncVoteSub.Unsubscribe()
 
 	events := voteManager.eth.EventMux().Subscribe(downloader.StartEvent{}, downloader.DoneEvent{}, downloader.FailedEvent{})
@@ -117,7 +126,7 @@ func (voteManager *VoteManager) loop() {
 				log.Debug("downloader is in DoneEvent mode, set the startVote flag to true")
 				startVote = true
 			}
-		case cHead := <-voteManager.chainHeadCh:
+		case cHead := <-voteManager.highestVerifiedBlockCh:
 			if !startVote {
 				log.Debug("startVote flag is false, continue")
 				continue
@@ -133,18 +142,27 @@ func (voteManager *VoteManager) loop() {
 				continue
 			}
 
-			if cHead.Block == nil {
-				log.Debug("cHead.Block is nil, continue")
+			if cHead.Header == nil {
+				log.Debug("cHead.Header is nil, continue")
 				continue
 			}
 
-			curHead := cHead.Block.Header()
+			curHead := cHead.Header
+			if p, ok := voteManager.engine.(*parlia.Parlia); ok {
+				nextBlockMinedTime := time.Unix(int64((curHead.Time + p.Period())), 0)
+				timeForBroadcast := 50 * time.Millisecond // enough to broadcast a vote
+				if time.Now().Add(timeForBroadcast).After(nextBlockMinedTime) {
+					log.Warn("too late to vote", "Head.Time(Second)", curHead.Time, "Now(Millisecond)", time.Now().UnixMilli())
+					continue
+				}
+			}
+
 			// Check if cur validator is within the validatorSet at curHead
 			if !voteManager.engine.IsActiveValidatorAt(voteManager.chain, curHead,
 				func(bLSPublicKey *types.BLSPublicKey) bool {
 					return bytes.Equal(voteManager.signer.PubKey[:], bLSPublicKey[:])
 				}) {
-				log.Debug("cur validator is not within the validatorSet at curHead")
+				log.Debug("local validator with voteKey is not within the validatorSet at curHead")
 				continue
 			}
 
@@ -191,6 +209,36 @@ func (voteManager *VoteManager) loop() {
 				voteManager.pool.PutVote(voteMessage)
 				votesManagerCounter.Inc(1)
 			}
+
+			// check the latest justified block, which indicating the stability of the network
+			curJustifiedNumber, _, err := voteManager.engine.GetJustifiedNumberAndHash(voteManager.chain, []*types.Header{curHead})
+			if err == nil && curJustifiedNumber != 0 {
+				if curJustifiedNumber+1 != curHead.Number.Uint64() {
+					log.Debug("not justified", "blockNumber", curHead.Number.Uint64()-1)
+					notJustified.Inc(1)
+				} else {
+					parent := voteManager.chain.GetHeaderByHash(curHead.ParentHash)
+					if parent != nil {
+						if parent.Difficulty.Cmp(diffInTurn) == 0 {
+							inTurnJustified.Inc(1)
+						} else {
+							log.Debug("not in turn block justified", "blockNumber", parent.Number.Int64(), "blockHash", parent.Hash())
+							notInTurnJustified.Inc(1)
+						}
+
+						lastJustifiedNumber, _, err := voteManager.engine.GetJustifiedNumberAndHash(voteManager.chain, []*types.Header{parent})
+						if err == nil {
+							if lastJustifiedNumber == 0 || lastJustifiedNumber+1 == curJustifiedNumber {
+								continuousJustified.Inc(1)
+							} else {
+								log.Debug("not continuous block justified", "lastJustified", lastJustifiedNumber, "curJustified", curJustifiedNumber)
+								notContinuousJustified.Inc(1)
+							}
+						}
+					}
+				}
+			}
+
 		case event := <-voteManager.syncVoteCh:
 			voteMessage := event.Vote
 			if voteManager.eth.IsMining() || !bytes.Equal(voteManager.signer.PubKey[:], voteMessage.VoteAddress[:]) {
@@ -206,7 +254,7 @@ func (voteManager *VoteManager) loop() {
 		case <-voteManager.syncVoteSub.Err():
 			log.Debug("voteManager subscribed votes failed")
 			return
-		case <-voteManager.chainHeadSub.Err():
+		case <-voteManager.highestVerifiedBlockSub.Err():
 			log.Debug("voteManager subscribed chainHead failed")
 			return
 		}

core/vote/vote_pool.go

@@ -24,7 +24,7 @@ const (
 	lowerLimitOfVoteBlockNumber = 256
 	upperLimitOfVoteBlockNumber = 11 // refer to fetcher.maxUncleDist
 
-	chainHeadChanSize = 10 // chainHeadChanSize is the size of channel listening to ChainHeadEvent.
+	highestVerifiedBlockChanSize = 10 // highestVerifiedBlockChanSize is the size of channel listening to HighestVerifiedBlockEvent.
 )
 
 var (
@@ -57,8 +57,8 @@ type VotePool struct {
 	curVotesPq    *votesPriorityQueue
 	futureVotesPq *votesPriorityQueue
 
-	chainHeadCh  chan core.ChainHeadEvent
-	chainHeadSub event.Subscription
+	highestVerifiedBlockCh  chan core.HighestVerifiedBlockEvent
+	highestVerifiedBlockSub event.Subscription
 
 	votesCh chan *types.VoteEnvelope
 
@@ -69,19 +69,19 @@ type votesPriorityQueue []*types.VoteData
 
 func NewVotePool(chain *core.BlockChain, engine consensus.PoSA) *VotePool {
 	votePool := &VotePool{
-		chain:         chain,
-		receivedVotes: mapset.NewSet[common.Hash](),
-		curVotes:      make(map[common.Hash]*VoteBox),
-		futureVotes:   make(map[common.Hash]*VoteBox),
-		curVotesPq:    &votesPriorityQueue{},
-		futureVotesPq: &votesPriorityQueue{},
-		chainHeadCh:   make(chan core.ChainHeadEvent, chainHeadChanSize),
-		votesCh:       make(chan *types.VoteEnvelope, voteBufferForPut),
-		engine:        engine,
+		chain:                  chain,
+		receivedVotes:          mapset.NewSet[common.Hash](),
+		curVotes:               make(map[common.Hash]*VoteBox),
+		futureVotes:            make(map[common.Hash]*VoteBox),
+		curVotesPq:             &votesPriorityQueue{},
+		futureVotesPq:          &votesPriorityQueue{},
+		highestVerifiedBlockCh: make(chan core.HighestVerifiedBlockEvent, highestVerifiedBlockChanSize),
+		votesCh:                make(chan *types.VoteEnvelope, voteBufferForPut),
+		engine:                 engine,
 	}
 
 	// Subscribe events from blockchain and start the main event loop.
-	votePool.chainHeadSub = votePool.chain.SubscribeChainHeadEvent(votePool.chainHeadCh)
+	votePool.highestVerifiedBlockSub = votePool.chain.SubscribeHighestVerifiedHeaderEvent(votePool.highestVerifiedBlockCh)
 
 	go votePool.loop()
 	return votePool
@@ -89,18 +89,18 @@ func NewVotePool(chain *core.BlockChain, engine consensus.PoSA) *VotePool {
 
 // loop is the vote pool's main even loop, waiting for and reacting to outside blockchain events and votes channel event.
 func (pool *VotePool) loop() {
-	defer pool.chainHeadSub.Unsubscribe()
+	defer pool.highestVerifiedBlockSub.Unsubscribe()
 
 	for {
 		select {
 		// Handle ChainHeadEvent.
-		case ev := <-pool.chainHeadCh:
-			if ev.Block != nil {
-				latestBlockNumber := ev.Block.NumberU64()
+		case ev := <-pool.highestVerifiedBlockCh:
+			if ev.Header != nil {
+				latestBlockNumber := ev.Header.Number.Uint64()
 				pool.prune(latestBlockNumber)
-				pool.transferVotesFromFutureToCur(ev.Block.Header())
+				pool.transferVotesFromFutureToCur(ev.Header)
 			}
-		case <-pool.chainHeadSub.Err():
+		case <-pool.highestVerifiedBlockSub.Err():
 			return
 
 		// Handle votes channel and put the vote into vote pool.
@@ -135,7 +135,7 @@ func (pool *VotePool) putIntoVotePool(vote *types.VoteEnvelope) bool {
 	var votesPq *votesPriorityQueue
 	isFutureVote := false
 
-	voteBlock := pool.chain.GetHeaderByHash(targetHash)
+	voteBlock := pool.chain.GetVerifiedBlockByHash(targetHash)
 	if voteBlock == nil {
 		votes = pool.futureVotes
 		votesPq = pool.futureVotesPq
@@ -226,7 +226,7 @@ func (pool *VotePool) transferVotesFromFutureToCur(latestBlockHeader *types.Head
 	futurePqBuffer := make([]*types.VoteData, 0)
 	for futurePq.Len() > 0 && futurePq.Peek().TargetNumber <= latestBlockNumber {
 		blockHash := futurePq.Peek().TargetHash
-		header := pool.chain.GetHeaderByHash(blockHash)
+		header := pool.chain.GetVerifiedBlockByHash(blockHash)
 		if header == nil {
 			// Put into pq buffer used for later put again into futurePq
 			futurePqBuffer = append(futurePqBuffer, heap.Pop(futurePq).(*types.VoteData))

eth/api_backend.go

@@ -26,6 +26,7 @@ import (
 	"github.com/ethereum/go-ethereum/accounts"
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/ethereum/go-ethereum/consensus"
+	"github.com/ethereum/go-ethereum/consensus/parlia"
 	"github.com/ethereum/go-ethereum/core"
 	"github.com/ethereum/go-ethereum/core/bloombits"
 	"github.com/ethereum/go-ethereum/core/rawdb"
@@ -440,6 +441,16 @@ func (b *EthAPIBackend) Engine() consensus.Engine {
 	return b.eth.engine
 }
 
+func (b *EthAPIBackend) CurrentTurnLength() (turnLength uint8, err error) {
+	if p, ok := b.eth.engine.(*parlia.Parlia); ok {
+		service := p.APIs(b.Chain())[0].Service
+		currentHead := rpc.LatestBlockNumber
+		return service.(*parlia.API).GetTurnLength(&currentHead)
+	}
+
+	return 1, nil
+}
+
 func (b *EthAPIBackend) CurrentHeader() *types.Header {
 	return b.eth.blockchain.CurrentHeader()
 }
@@ -484,6 +495,10 @@ func (b *EthAPIBackend) RemoveBuilder(builder common.Address) error {
 	return b.Miner().RemoveBuilder(builder)
 }
 
+func (b *EthAPIBackend) HasBuilder(builder common.Address) bool {
+	return b.Miner().HasBuilder(builder)
+}
+
 func (b *EthAPIBackend) SendBid(ctx context.Context, bid *types.BidArgs) (common.Hash, error) {
 	return b.Miner().SendBid(ctx, bid)
 }

eth/backend.go

@@ -161,12 +161,18 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) {
 	// Optimize memory distribution by reallocating surplus allowance from the
 	// dirty cache to the clean cache.
 	if config.StateScheme == rawdb.PathScheme && config.TrieDirtyCache > pathdb.MaxDirtyBufferSize/1024/1024 {
-		log.Info("Capped dirty cache size", "provided", common.StorageSize(config.TrieDirtyCache)*1024*1024, "adjusted", common.StorageSize(pathdb.MaxDirtyBufferSize))
-		log.Info("Clean cache size", "provided", common.StorageSize(config.TrieCleanCache)*1024*1024)
+		log.Info("Capped dirty cache size", "provided", common.StorageSize(config.TrieDirtyCache)*1024*1024,
+			"adjusted", common.StorageSize(pathdb.MaxDirtyBufferSize))
+		log.Info("Clean cache size", "provided", common.StorageSize(config.TrieCleanCache)*1024*1024,
+			"adjusted", common.StorageSize(config.TrieCleanCache+config.TrieDirtyCache-pathdb.MaxDirtyBufferSize/1024/1024)*1024*1024)
+		config.TrieCleanCache += config.TrieDirtyCache - pathdb.MaxDirtyBufferSize/1024/1024
 		config.TrieDirtyCache = pathdb.MaxDirtyBufferSize / 1024 / 1024
 	}
-	log.Info("Allocated trie memory caches", "clean", common.StorageSize(config.TrieCleanCache)*1024*1024, "dirty", common.StorageSize(config.TrieDirtyCache)*1024*1024)
-
+	log.Info("Allocated memory caches",
+		"state_scheme", config.StateScheme,
+		"trie_clean_cache", common.StorageSize(config.TrieCleanCache)*1024*1024,
+		"trie_dirty_cache", common.StorageSize(config.TrieDirtyCache)*1024*1024,
+		"snapshot_cache", common.StorageSize(config.SnapshotCache)*1024*1024)
 	// Try to recover offline state pruning only in hash-based.
 	if config.StateScheme == rawdb.HashScheme {
 		if err := pruner.RecoverPruning(stack.ResolvePath(""), chainDb, config.TriesInMemory); err != nil {
@@ -179,13 +185,15 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) {
 	}
 	// Override the chain config with provided settings.
 	var overrides core.ChainOverrides
-	if config.OverrideCancun != nil {
-		chainConfig.CancunTime = config.OverrideCancun
-		overrides.OverrideCancun = config.OverrideCancun
-	}
-	if config.OverrideHaber != nil {
-		chainConfig.HaberTime = config.OverrideHaber
-		overrides.OverrideHaber = config.OverrideHaber
+	if config.OverridePassedForkTime != nil {
+		chainConfig.ShanghaiTime = config.OverridePassedForkTime
+		chainConfig.KeplerTime = config.OverridePassedForkTime
+		chainConfig.FeynmanTime = config.OverridePassedForkTime
+		chainConfig.FeynmanFixTime = config.OverridePassedForkTime
+		chainConfig.CancunTime = config.OverridePassedForkTime
+		chainConfig.HaberTime = config.OverridePassedForkTime
+		chainConfig.HaberFixTime = config.OverridePassedForkTime
+		overrides.OverridePassedForkTime = config.OverridePassedForkTime
 	}
 	if config.OverrideBohr != nil {
 		chainConfig.BohrTime = config.OverrideBohr

eth/downloader/downloader.go

@@ -558,8 +558,8 @@ func (d *Downloader) syncWithPeer(p *peerConnection, hash common.Hash, td, ttd *
 		} else {
 			d.ancientLimit = 0
 		}
-		frozen, _ := d.stateDB.Ancients() // Ignore the error here since light client can also hit here.
-		itemAmountInAncient, _ := d.stateDB.ItemAmountInAncient()
+		frozen, _ := d.stateDB.BlockStore().Ancients() // Ignore the error here since light client can also hit here.
+		itemAmountInAncient, _ := d.stateDB.BlockStore().ItemAmountInAncient()
 		// If a part of blockchain data has already been written into active store,
 		// disable the ancient style insertion explicitly.
 		if origin >= frozen && itemAmountInAncient != 0 {
@@ -1671,9 +1671,9 @@ func (d *Downloader) reportSnapSyncProgress(force bool) {
 	}
 	// Don't report anything until we have a meaningful progress
 	var (
-		headerBytes, _  = d.stateDB.AncientSize(rawdb.ChainFreezerHeaderTable)
-		bodyBytes, _    = d.stateDB.AncientSize(rawdb.ChainFreezerBodiesTable)
-		receiptBytes, _ = d.stateDB.AncientSize(rawdb.ChainFreezerReceiptTable)
+		headerBytes, _  = d.stateDB.BlockStore().AncientSize(rawdb.ChainFreezerHeaderTable)
+		bodyBytes, _    = d.stateDB.BlockStore().AncientSize(rawdb.ChainFreezerBodiesTable)
+		receiptBytes, _ = d.stateDB.BlockStore().AncientSize(rawdb.ChainFreezerReceiptTable)
 	)
 	syncedBytes := common.StorageSize(headerBytes + bodyBytes + receiptBytes)
 	if syncedBytes == 0 {

eth/ethconfig/config.go

@@ -188,11 +188,8 @@ type Config struct {
 	// send-transaction variants. The unit is ether.
 	RPCTxFeeCap float64
 
-	// OverrideCancun (TODO: remove after the fork)
-	OverrideCancun *uint64 `toml:",omitempty"`
-
-	// OverrideHaber (TODO: remove after the fork)
-	OverrideHaber *uint64 `toml:",omitempty"`
+	// OverridePassedForkTime
+	OverridePassedForkTime *uint64 `toml:",omitempty"`
 
 	// OverrideBohr (TODO: remove after the fork)
 	OverrideBohr *uint64 `toml:",omitempty"`

eth/ethconfig/gen_config.go

@@ -70,8 +70,7 @@ func (c Config) MarshalTOML() (interface{}, error) {
 		RPCGasCap               uint64
 		RPCEVMTimeout           time.Duration
 		RPCTxFeeCap             float64
-		OverrideCancun          *uint64 `toml:",omitempty"`
-		OverrideHaber           *uint64 `toml:",omitempty"`
+		OverridePassedForkTime      *uint64 `toml:",omitempty"`
 		OverrideBohr            *uint64 `toml:",omitempty"`
 		OverrideVerkle          *uint64 `toml:",omitempty"`
 		BlobExtraReserve        uint64
@@ -130,8 +129,7 @@ func (c Config) MarshalTOML() (interface{}, error) {
 	enc.RPCGasCap = c.RPCGasCap
 	enc.RPCEVMTimeout = c.RPCEVMTimeout
 	enc.RPCTxFeeCap = c.RPCTxFeeCap
-	enc.OverrideCancun = c.OverrideCancun
-	enc.OverrideHaber = c.OverrideHaber
+	enc.OverridePassedForkTime = c.OverridePassedForkTime
 	enc.OverrideBohr = c.OverrideBohr
 	enc.OverrideVerkle = c.OverrideVerkle
 	enc.BlobExtraReserve = c.BlobExtraReserve
@@ -194,8 +192,7 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error {
 		RPCGasCap               *uint64
 		RPCEVMTimeout           *time.Duration
 		RPCTxFeeCap             *float64
-		OverrideCancun          *uint64 `toml:",omitempty"`
-		OverrideHaber           *uint64 `toml:",omitempty"`
+		OverridePassedForkTime      *uint64 `toml:",omitempty"`
 		OverrideBohr            *uint64 `toml:",omitempty"`
 		OverrideVerkle          *uint64 `toml:",omitempty"`
 		BlobExtraReserve        *uint64
@@ -363,11 +360,8 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error {
 	if dec.RPCTxFeeCap != nil {
 		c.RPCTxFeeCap = *dec.RPCTxFeeCap
 	}
-	if dec.OverrideCancun != nil {
-		c.OverrideCancun = dec.OverrideCancun
-	}
-	if dec.OverrideHaber != nil {
-		c.OverrideHaber = dec.OverrideHaber
+	if dec.OverridePassedForkTime != nil {
+		c.OverridePassedForkTime = dec.OverridePassedForkTime
 	}
 	if dec.OverrideBohr != nil {
 		c.OverrideBohr = dec.OverrideBohr

eth/fetcher/block_fetcher.go

@@ -731,9 +731,6 @@ func (f *BlockFetcher) loop() {
 						matched = true
 						if f.getBlock(hash) == nil {
 							block := types.NewBlockWithHeader(announce.header).WithBody(task.transactions[i], task.uncles[i])
-							if block.Header().EmptyWithdrawalsHash() {
-								block = block.WithWithdrawals(make([]*types.Withdrawal, 0))
-							}
 							block = block.WithSidecars(task.sidecars[i])
 							block.ReceivedAt = task.time
 							blocks = append(blocks, block)
@@ -919,6 +916,10 @@ func (f *BlockFetcher) importBlocks(op *blockOrHeaderInject) {
 			return
 		}
 
+		if block.Header().EmptyWithdrawalsHash() {
+			block = block.WithWithdrawals(make([]*types.Withdrawal, 0))
+		}
+
 		defer func() { f.done <- hash }()
 		// Quickly validate the header and propagate the block if it passes
 		switch err := f.verifyHeader(block.Header()); err {

eth/fetcher/block_fetcher_test.go

@@ -158,7 +158,7 @@ func (f *fetcherTester) chainFinalizedHeight() uint64 {
 	return f.blocks[f.hashes[len(f.hashes)-3]].NumberU64()
 }
 
-// insertChain injects a new headers into the simulated chain.
+// insertHeaders injects a new headers into the simulated chain.
 func (f *fetcherTester) insertHeaders(headers []*types.Header) (int, error) {
 	f.lock.Lock()
 	defer f.lock.Unlock()

eth/filters/filter_system_test.go

@@ -436,7 +436,7 @@ func TestInvalidLogFilterCreation(t *testing.T) {
 	}
 }
 
-// TestLogFilterUninstall tests invalid getLogs requests
+// TestInvalidGetLogsRequest tests invalid getLogs requests
 func TestInvalidGetLogsRequest(t *testing.T) {
 	t.Parallel()
 

eth/handler.go

@@ -369,6 +369,7 @@ func newHandler(config *handlerConfig) (*handler, error) {
 	}
 	h.txFetcher = fetcher.NewTxFetcher(h.txpool.Has, addTxs, fetchTx, h.removePeer)
 	h.chainSync = newChainSyncer(h)
+	h.printPeerStatus()
 	return h, nil
 }
 
@@ -1017,3 +1018,67 @@ func (h *handler) enableSyncedFeatures() {
 	// 	h.chain.TrieDB().SetBufferSize(pathdb.DefaultBufferSize)
 	// }
 }
+
+type PeerDummy struct {
+	ID        string
+	Head      common.Hash
+	TD        *big.Int
+	TimeStamp time.Time
+}
+
+func (h *handler) printPeerStatus() {
+	go func() {
+		statusMap := make(map[string]PeerDummy)
+		for {
+			// run in timer very 1 minue of time
+			time.Sleep(1 * time.Minute)
+			// Create a set to track current peers
+			currentPeers := make(map[string]struct{})
+			// print peer status
+			h.peers.lock.RLock()
+			for _, peer := range h.peers.peers {
+				currentPeers[peer.Peer.ID()] = struct{}{}
+				head, td := peer.Peer.Head()
+				if p, ok := statusMap[peer.Peer.ID()]; ok {
+					if p.Head == head && p.TD.Cmp(td) == 0 {
+						continue
+					}
+					p.Head = head
+					p.TD = td
+					p.TimeStamp = time.Now()
+					statusMap[peer.Peer.ID()] = p
+				} else {
+					statusMap[peer.Peer.ID()] = PeerDummy{
+						ID:        peer.Peer.ID(),
+						Head:      head,
+						TD:        td,
+						TimeStamp: time.Now(),
+					}
+				}
+			}
+			h.peers.lock.RUnlock()
+
+			// Remove peers from statusMap that are no longer in h.peers.peers
+			for id := range statusMap {
+				if _, exists := currentPeers[id]; !exists {
+					delete(statusMap, id)
+				}
+			}
+
+			var count int
+			for _, peer := range statusMap {
+				if peer.TimeStamp.Before(time.Now().Add(-60 * time.Minute)) {
+					count++
+					if count == 1 {
+						log.Warn("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
+					}
+					log.Warn("peer", peer.ID, "head", peer.Head, "TD", peer.TD, "TimeStamp", peer.TimeStamp)
+				}
+			}
+			if count > 0 {
+				log.Warn("Total peers: ", len(statusMap), "inactive peers: ", count)
+				log.Warn("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX")
+			}
+		}
+	}()
+}

eth/handler_eth_test.go

@@ -633,6 +633,9 @@ func testBroadcastBlock(t *testing.T, peers, bcasts int) {
 		go source.handler.runEthPeer(sourcePeer, func(peer *eth.Peer) error {
 			return eth.Handle((*ethHandler)(source.handler), peer)
 		})
+		// Wait a bit for the above handlers to start
+		time.Sleep(100 * time.Millisecond)
+
 		if err := sinkPeer.Handshake(1, td, genesis.Hash(), genesis.Hash(), forkid.NewIDWithChain(source.chain), forkid.NewFilter(source.chain), nil); err != nil {
 			t.Fatalf("failed to run protocol handshake")
 		}

eth/protocols/eth/dispatcher.go

@@ -136,7 +136,7 @@ func (p *Peer) dispatchRequest(req *Request) error {
 	}
 }
 
-// dispatchRequest fulfils a pending request and delivers it to the requested
+// dispatchResponse fulfils a pending request and delivers it to the requested
 // sink.
 func (p *Peer) dispatchResponse(res *Response, metadata func() interface{}) error {
 	resOp := &response{

eth/protocols/snap/sync.go

@@ -409,8 +409,8 @@ type SyncPeer interface {
 //   - The peer delivers a stale response after a previous timeout
 //   - The peer delivers a refusal to serve the requested state
 type Syncer struct {
-	db     ethdb.KeyValueStore // Database to store the trie nodes into (and dedup)
-	scheme string              // Node scheme used in node database
+	db     ethdb.Database // Database to store the trie nodes into (and dedup)
+	scheme string         // Node scheme used in node database
 
 	root    common.Hash    // Current state trie root being synced
 	tasks   []*accountTask // Current account task set being synced
@@ -478,7 +478,7 @@ type Syncer struct {
 
 // NewSyncer creates a new snapshot syncer to download the Ethereum state over the
 // snap protocol.
-func NewSyncer(db ethdb.KeyValueStore, scheme string) *Syncer {
+func NewSyncer(db ethdb.Database, scheme string) *Syncer {
 	return &Syncer{
 		db:     db,
 		scheme: scheme,
@@ -719,11 +719,11 @@ func (s *Syncer) Sync(root common.Hash, cancel chan struct{}) error {
 
 // cleanPath is used to remove the dangling nodes in the stackTrie.
 func (s *Syncer) cleanPath(batch ethdb.Batch, owner common.Hash, path []byte) {
-	if owner == (common.Hash{}) && rawdb.ExistsAccountTrieNode(s.db, path) {
+	if owner == (common.Hash{}) && rawdb.ExistsAccountTrieNode(s.db.StateStoreReader(), path) {
 		rawdb.DeleteAccountTrieNode(batch, path)
 		deletionGauge.Inc(1)
 	}
-	if owner != (common.Hash{}) && rawdb.ExistsStorageTrieNode(s.db, owner, path) {
+	if owner != (common.Hash{}) && rawdb.ExistsStorageTrieNode(s.db.StateStoreReader(), owner, path) {
 		rawdb.DeleteStorageTrieNode(batch, owner, path)
 		deletionGauge.Inc(1)
 	}
@@ -735,6 +735,7 @@ func (s *Syncer) cleanPath(batch ethdb.Batch, owner common.Hash, path []byte) {
 func (s *Syncer) loadSyncStatus() {
 	var progress SyncProgress
 
+	stateDiskDB := s.db.GetStateStore()
 	if status := rawdb.ReadSnapshotSyncStatus(s.db); status != nil {
 		if err := json.Unmarshal(status, &progress); err != nil {
 			log.Error("Failed to decode snap sync status", "err", err)
@@ -747,7 +748,7 @@ func (s *Syncer) loadSyncStatus() {
 				task := task // closure for task.genBatch in the stacktrie writer callback
 
 				task.genBatch = ethdb.HookedBatch{
-					Batch: s.db.NewBatch(),
+					Batch: stateDiskDB.NewBatch(),
 					OnPut: func(key []byte, value []byte) {
 						s.accountBytes += common.StorageSize(len(key) + len(value))
 					},
@@ -773,7 +774,7 @@ func (s *Syncer) loadSyncStatus() {
 						subtask := subtask // closure for subtask.genBatch in the stacktrie writer callback
 
 						subtask.genBatch = ethdb.HookedBatch{
-							Batch: s.db.NewBatch(),
+							Batch: stateDiskDB.NewBatch(),
 							OnPut: func(key []byte, value []byte) {
 								s.storageBytes += common.StorageSize(len(key) + len(value))
 							},
@@ -841,7 +842,7 @@ func (s *Syncer) loadSyncStatus() {
 			last = common.MaxHash
 		}
 		batch := ethdb.HookedBatch{
-			Batch: s.db.NewBatch(),
+			Batch: stateDiskDB.NewBatch(),
 			OnPut: func(key []byte, value []byte) {
 				s.accountBytes += common.StorageSize(len(key) + len(value))
 			},
@@ -1894,7 +1895,7 @@ func (s *Syncer) processAccountResponse(res *accountResponse) {
 		}
 		// Check if the account is a contract with an unknown storage trie
 		if account.Root != types.EmptyRootHash {
-			if !rawdb.HasTrieNode(s.db, res.hashes[i], nil, account.Root, s.scheme) {
+			if !rawdb.HasTrieNode(s.db.StateStoreReader(), res.hashes[i], nil, account.Root, s.scheme) {
 				// If there was a previous large state retrieval in progress,
 				// don't restart it from scratch. This happens if a sync cycle
 				// is interrupted and resumed later. However, *do* update the
@@ -1986,12 +1987,25 @@ func (s *Syncer) processStorageResponse(res *storageResponse) {
 	if res.subTask != nil {
 		res.subTask.req = nil
 	}
+
+	var usingMultDatabase bool
 	batch := ethdb.HookedBatch{
-		Batch: s.db.NewBatch(),
+		Batch: s.db.GetStateStore().NewBatch(),
 		OnPut: func(key []byte, value []byte) {
 			s.storageBytes += common.StorageSize(len(key) + len(value))
 		},
 	}
+	var snapBatch ethdb.HookedBatch
+	if s.db.StateStore() != nil {
+		usingMultDatabase = true
+		snapBatch = ethdb.HookedBatch{
+			Batch: s.db.NewBatch(),
+			OnPut: func(key []byte, value []byte) {
+				s.storageBytes += common.StorageSize(len(key) + len(value))
+			},
+		}
+	}
+
 	var (
 		slots           int
 		oldStorageBytes = s.storageBytes
@@ -2061,7 +2075,7 @@ func (s *Syncer) processStorageResponse(res *storageResponse) {
 					}
 					// Our first task is the one that was just filled by this response.
 					batch := ethdb.HookedBatch{
-						Batch: s.db.NewBatch(),
+						Batch: s.db.GetStateStore().NewBatch(),
 						OnPut: func(key []byte, value []byte) {
 							s.storageBytes += common.StorageSize(len(key) + len(value))
 						},
@@ -2088,7 +2102,7 @@ func (s *Syncer) processStorageResponse(res *storageResponse) {
 					})
 					for r.Next() {
 						batch := ethdb.HookedBatch{
-							Batch: s.db.NewBatch(),
+							Batch: s.db.GetStateStore().NewBatch(),
 							OnPut: func(key []byte, value []byte) {
 								s.storageBytes += common.StorageSize(len(key) + len(value))
 							},
@@ -2184,8 +2198,11 @@ func (s *Syncer) processStorageResponse(res *storageResponse) {
 		// outdated during the sync, but it can be fixed later during the
 		// snapshot generation.
 		for j := 0; j < len(res.hashes[i]); j++ {
-			rawdb.WriteStorageSnapshot(batch, account, res.hashes[i][j], res.slots[i][j])
-
+			if usingMultDatabase {
+				rawdb.WriteStorageSnapshot(snapBatch, account, res.hashes[i][j], res.slots[i][j])
+			} else {
+				rawdb.WriteStorageSnapshot(batch, account, res.hashes[i][j], res.slots[i][j])
+			}
 			// If we're storing large contracts, generate the trie nodes
 			// on the fly to not trash the gluing points
 			if i == len(res.hashes)-1 && res.subTask != nil {
@@ -2205,7 +2222,7 @@ func (s *Syncer) processStorageResponse(res *storageResponse) {
 			// If the chunk's root is an overflown but full delivery,
 			// clear the heal request.
 			accountHash := res.accounts[len(res.accounts)-1]
-			if root == res.subTask.root && rawdb.HasStorageTrieNode(s.db, accountHash, nil, root) {
+			if root == res.subTask.root && rawdb.HasStorageTrieNode(s.db.StateStoreReader(), accountHash, nil, root) {
 				for i, account := range res.mainTask.res.hashes {
 					if account == accountHash {
 						res.mainTask.needHeal[i] = false
@@ -2225,6 +2242,11 @@ func (s *Syncer) processStorageResponse(res *storageResponse) {
 	if err := batch.Write(); err != nil {
 		log.Crit("Failed to persist storage slots", "err", err)
 	}
+	if usingMultDatabase {
+		if err := snapBatch.Write(); err != nil {
+			log.Crit("Failed to persist storage slots", "err", err)
+		}
+	}
 	s.storageSynced += uint64(slots)
 
 	log.Debug("Persisted set of storage slots", "accounts", len(res.hashes), "slots", slots, "bytes", s.storageBytes-oldStorageBytes)
@@ -2323,12 +2345,25 @@ func (s *Syncer) commitHealer(force bool) {
 		return
 	}
 	batch := s.db.NewBatch()
-	if err := s.healer.scheduler.Commit(batch); err != nil {
+	var stateBatch ethdb.Batch
+	var err error
+	if s.db.StateStore() != nil {
+		stateBatch = s.db.StateStore().NewBatch()
+		err = s.healer.scheduler.Commit(batch, stateBatch)
+	} else {
+		err = s.healer.scheduler.Commit(batch, nil)
+	}
+	if err != nil {
 		log.Error("Failed to commit healing data", "err", err)
 	}
 	if err := batch.Write(); err != nil {
 		log.Crit("Failed to persist healing data", "err", err)
 	}
+	if s.db.StateStore() != nil {
+		if err := stateBatch.Write(); err != nil {
+			log.Crit("Failed to persist healing data", "err", err)
+		}
+	}
 	log.Debug("Persisted set of healing data", "type", "trienodes", "bytes", common.StorageSize(batch.ValueSize()))
 }
 

eth/state_accessor.go

@@ -281,7 +281,7 @@ func (eth *Ethereum) stateAtTransaction(ctx context.Context, block *types.Block,
 		}
 		// Not yet the searched for transaction, execute on top of the current state
 		vmenv := vm.NewEVM(context, txContext, statedb, eth.blockchain.Config(), vm.Config{})
-		statedb.SetTxContext(tx.Hash(), idx, 0)
+		statedb.SetTxContext(tx.Hash(), idx)
 		if _, err := core.ApplyMessage(vmenv, msg, new(core.GasPool).AddGas(tx.Gas())); err != nil {
 			return nil, vm.BlockContext{}, nil, nil, fmt.Errorf("transaction %#x failed: %v", tx.Hash(), err)
 		}

eth/sync_test.go

@@ -112,7 +112,14 @@ func testChainSyncWithBlobs(t *testing.T, mode downloader.SyncMode, preCancunBlk
 	cancunTime := (preCancunBlks + 1) * 10
 	config.CancunTime = &cancunTime
 
-	// Create a full handler and ensure snap sync ends up disabled
+	// Create an empty handler
+	empty := newTestParliaHandlerAfterCancun(t, &config, mode, 0, 0)
+	defer empty.close()
+	if downloader.SnapSync == mode && !empty.handler.snapSync.Load() {
+		t.Fatalf("snap sync disabled on pristine blockchain")
+	}
+
+	// Create a full handler
 	full := newTestParliaHandlerAfterCancun(t, &config, mode, preCancunBlks, postCancunBlks)
 	defer full.close()
 	if downloader.SnapSync == mode && full.handler.snapSync.Load() {
@@ -122,13 +129,6 @@ func testChainSyncWithBlobs(t *testing.T, mode downloader.SyncMode, preCancunBlk
 	// check blocks and blobs
 	checkChainWithBlobs(t, full.chain, preCancunBlks, postCancunBlks)
 
-	// Create an empty handler and ensure it's in snap sync mode
-	empty := newTestParliaHandlerAfterCancun(t, &config, mode, 0, 0)
-	defer empty.close()
-	if downloader.SnapSync == mode && !empty.handler.snapSync.Load() {
-		t.Fatalf("snap sync disabled on pristine blockchain")
-	}
-
 	// Sync up the two handlers via both `eth` and `snap`
 	ethVer := uint(eth.ETH68)
 	snapVer := uint(snap.SNAP1)
@@ -165,14 +165,17 @@ func testChainSyncWithBlobs(t *testing.T, mode downloader.SyncMode, preCancunBlk
 	go full.handler.runSnapExtension(fullPeerSnap, func(peer *snap.Peer) error {
 		return snap.Handle((*snapHandler)(full.handler), peer)
 	})
-	// Wait a bit for the above handlers to start
-	time.Sleep(250 * time.Millisecond)
 
-	// Check that snap sync was disabled
+	for empty.handler.peers.snapLen() < 1 {
+		// Wait a bit for the above handlers to start
+		time.Sleep(100 * time.Millisecond)
+	}
+
 	op := peerToSyncOp(mode, empty.handler.peers.peerWithHighestTD())
 	if err := empty.handler.doSync(op); err != nil {
 		t.Fatal("sync failed:", err)
 	}
+	// Check that snap sync was disabled
 	if !empty.handler.synced.Load() {
 		t.Fatalf("full sync not done after successful synchronisation")
 	}

eth/tracers/api.go

@@ -587,7 +587,7 @@ func (api *API) IntermediateRoots(ctx context.Context, hash common.Hash, config
 			}
 		}
 
-		statedb.SetTxContext(tx.Hash(), i, 0)
+		statedb.SetTxContext(tx.Hash(), i)
 		if _, err := core.ApplyMessage(vmenv, msg, new(core.GasPool).AddGas(msg.GasLimit)); err != nil {
 			log.Warn("Tracing intermediate roots did not complete", "txindex", i, "txhash", tx.Hash(), "err", err)
 			// We intentionally don't return the error here: if we do, then the RPC server will not
@@ -786,7 +786,7 @@ txloop:
 
 		// Generate the next state snapshot fast without tracing
 		msg, _ := core.TransactionToMessage(tx, signer, block.BaseFee())
-		statedb.SetTxContext(tx.Hash(), i, 0)
+		statedb.SetTxContext(tx.Hash(), i)
 		vmenv := vm.NewEVM(blockCtx, core.NewEVMTxContext(msg), statedb, api.backend.ChainConfig(), vm.Config{})
 		if _, err := core.ApplyMessage(vmenv, msg, new(core.GasPool).AddGas(msg.GasLimit)); err != nil {
 			failed = err
@@ -919,7 +919,7 @@ func (api *API) standardTraceBlockToFile(ctx context.Context, block *types.Block
 		}
 		// Execute the transaction and flush any traces to disk
 		vmenv := vm.NewEVM(vmctx, txContext, statedb, chainConfig, vmConf)
-		statedb.SetTxContext(tx.Hash(), i, 0)
+		statedb.SetTxContext(tx.Hash(), i)
 		_, err = core.ApplyMessage(vmenv, msg, new(core.GasPool).AddGas(msg.GasLimit))
 		if writer != nil {
 			writer.Flush()
@@ -1127,7 +1127,7 @@ func (api *API) traceTx(ctx context.Context, message *core.Message, txctx *Conte
 	}
 
 	// Call Prepare to clear out the statedb access list
-	statedb.SetTxContext(txctx.TxHash, txctx.TxIndex, 0)
+	statedb.SetTxContext(txctx.TxHash, txctx.TxIndex)
 	if _, err = core.ApplyMessage(vmenv, message, new(core.GasPool).AddGas(message.GasLimit)); err != nil {
 		return nil, fmt.Errorf("tracing failed: %w", err)
 	}

ethclient/ethclient.go

@@ -133,7 +133,7 @@ func (ec *Client) BlockReceipts(ctx context.Context, blockNrOrHash rpc.BlockNumb
 // BlobSidecars return the Sidecars of a given block number or hash.
 func (ec *Client) BlobSidecars(ctx context.Context, blockNrOrHash rpc.BlockNumberOrHash) ([]*types.BlobTxSidecar, error) {
 	var r []*types.BlobTxSidecar
-	err := ec.c.CallContext(ctx, &r, "eth_getBlobSidecars", blockNrOrHash.String(), true)
+	err := ec.c.CallContext(ctx, &r, "eth_getBlobSidecars", blockNrOrHash.String())
 	if err == nil && r == nil {
 		return nil, ethereum.NotFound
 	}
@@ -143,7 +143,7 @@ func (ec *Client) BlobSidecars(ctx context.Context, blockNrOrHash rpc.BlockNumbe
 // BlobSidecarByTxHash return a sidecar of a given blob transaction
 func (ec *Client) BlobSidecarByTxHash(ctx context.Context, hash common.Hash) (*types.BlobTxSidecar, error) {
 	var r *types.BlobTxSidecar
-	err := ec.c.CallContext(ctx, &r, "eth_getBlockSidecarByTxHash", hash, true)
+	err := ec.c.CallContext(ctx, &r, "eth_getBlobSidecarByTxHash", hash)
 	if err == nil && r == nil {
 		return nil, ethereum.NotFound
 	}
@@ -361,7 +361,7 @@ func (ec *Client) TransactionInBlock(ctx context.Context, blockHash common.Hash,
 	return json.tx, err
 }
 
-// TransactionInBlock returns a single transaction at index in the given block.
+// TransactionsInBlock returns a single transaction at index in the given block.
 func (ec *Client) TransactionsInBlock(ctx context.Context, number *big.Int) ([]*types.Transaction, error) {
 	var rpcTxs []*rpcTransaction
 	err := ec.c.CallContext(ctx, &rpcTxs, "eth_getTransactionsByBlockNumber", toBlockNumArg(number))
@@ -376,7 +376,7 @@ func (ec *Client) TransactionsInBlock(ctx context.Context, number *big.Int) ([]*
 	return txs, err
 }
 
-// TransactionInBlock returns a single transaction at index in the given block.
+// TransactionRecipientsInBlock returns a single transaction at index in the given block.
 func (ec *Client) TransactionRecipientsInBlock(ctx context.Context, number *big.Int) ([]*types.Receipt, error) {
 	var rs []*types.Receipt
 	err := ec.c.CallContext(ctx, &rs, "eth_getTransactionReceiptsByBlockNumber", toBlockNumArg(number))
@@ -752,6 +752,13 @@ func (ec *Client) MevRunning(ctx context.Context) (bool, error) {
 	return result, err
 }
 
+// HasBuilder returns whether the builder is registered
+func (ec *Client) HasBuilder(ctx context.Context, address common.Address) (bool, error) {
+	var result bool
+	err := ec.c.CallContext(ctx, &result, "mev_hasBuilder", address)
+	return result, err
+}
+
 // SendBid sends a bid
 func (ec *Client) SendBid(ctx context.Context, args types.BidArgs) (common.Hash, error) {
 	var hash common.Hash

ethdb/database.go

@@ -180,12 +180,6 @@ type StateStoreReader interface {
 	StateStoreReader() Reader
 }
 
-type BlockStore interface {
-	BlockStore() Database
-	SetBlockStore(block Database)
-	HasSeparateBlockStore() bool
-}
-
 type BlockStoreReader interface {
 	BlockStoreReader() Reader
 }
@@ -194,6 +188,14 @@ type BlockStoreWriter interface {
 	BlockStoreWriter() Writer
 }
 
+// MultiDatabaseReader contains the methods required to read data from both key-value as well as
+// blockStore or stateStore.
+type MultiDatabaseReader interface {
+	KeyValueReader
+	StateStoreReader
+	BlockStoreReader
+}
+
 // Reader contains the methods required to read data from both key-value as well as
 // immutable ancient data.
 type Reader interface {
@@ -234,6 +236,13 @@ type DiffStore interface {
 type StateStore interface {
 	StateStore() Database
 	SetStateStore(state Database)
+	GetStateStore() Database
+}
+
+type BlockStore interface {
+	BlockStore() Database
+	SetBlockStore(block Database)
+	HasSeparateBlockStore() bool
 }
 
 // Database contains all the methods required by the high level database to not

ethdb/memorydb/memorydb.go

@@ -39,6 +39,9 @@ var (
 	// errSnapshotReleased is returned if callers want to retrieve data from a
 	// released snapshot.
 	errSnapshotReleased = errors.New("snapshot released")
+
+	// errNotSupported is returned if the database doesn't support the required operation.
+	errNotSupported = errors.New("this operation is not supported")
 )
 
 // Database is an ephemeral key-value store. Apart from basic data storage
@@ -47,6 +50,84 @@ var (
 type Database struct {
 	db   map[string][]byte
 	lock sync.RWMutex
+
+	stateStore ethdb.Database
+	blockStore ethdb.Database
+}
+
+func (db *Database) ModifyAncients(f func(ethdb.AncientWriteOp) error) (int64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) TruncateHead(n uint64) (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) TruncateTail(n uint64) (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) Sync() error {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) TruncateTableTail(kind string, tail uint64) (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) ResetTable(kind string, startAt uint64, onlyEmpty bool) error {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) HasAncient(kind string, number uint64) (bool, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) Ancient(kind string, number uint64) ([]byte, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) Ancients() (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) Tail() (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) AncientSize(kind string) (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) ItemAmountInAncient() (uint64, error) {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) AncientOffSet() uint64 {
+	//TODO implement me
+	panic("implement me")
+}
+
+func (db *Database) ReadAncients(fn func(ethdb.AncientReaderOp) error) (err error) {
+	//TODO implement me
+	panic("implement me")
 }
 
 // New returns a wrapped map with all the required database interface methods
@@ -204,6 +285,37 @@ func (db *Database) Len() int {
 	return len(db.db)
 }
 
+func (db *Database) StateStoreReader() ethdb.Reader {
+	if db.stateStore == nil {
+		return db
+	}
+	return db.stateStore
+}
+
+func (db *Database) BlockStoreReader() ethdb.Reader {
+	if db.blockStore == nil {
+		return db
+	}
+	return db.blockStore
+}
+
+func (db *Database) BlockStoreWriter() ethdb.Writer {
+	if db.blockStore == nil {
+		return db
+	}
+	return db.blockStore
+}
+
+// convertLegacyFn takes a raw freezer entry in an older format and
+// returns it in the new format.
+type convertLegacyFn = func([]byte) ([]byte, error)
+
+// MigrateTable processes the entries in a given table in sequence
+// converting them to a new format if they're of an old format.
+func (db *Database) MigrateTable(kind string, convert convertLegacyFn) error {
+	return errNotSupported
+}
+
 // keyvalue is a key-value tuple tagged with a deletion field to allow creating
 // memory-database write batches.
 type keyvalue struct {

ethdb/remotedb/remotedb.go

@@ -122,6 +122,10 @@ func (db *Database) SetStateStore(state ethdb.Database) {
 	panic("not supported")
 }
 
+func (db *Database) GetStateStore() ethdb.Database {
+	panic("not supported")
+}
+
 func (db *Database) StateStoreReader() ethdb.Reader {
 	return db
 }

go.mod

@@ -2,8 +2,6 @@ module github.com/ethereum/go-ethereum
 
 go 1.21
 
-toolchain go1.21.5
-
 require (
 	github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.2.0
 	github.com/Microsoft/go-winio v0.6.1
@@ -17,8 +15,8 @@ require (
 	github.com/btcsuite/btcd/btcec/v2 v2.3.2
 	github.com/cespare/cp v1.1.1
 	github.com/cloudflare/cloudflare-go v0.79.0
-	github.com/cometbft/cometbft v0.37.0
 	github.com/cockroachdb/pebble v1.1.0
+	github.com/cometbft/cometbft v0.37.0
 	github.com/consensys/gnark-crypto v0.12.1
 	github.com/crate-crypto/go-ipa v0.0.0-20231025140028-3c0104f4b233
 	github.com/crate-crypto/go-kzg-4844 v0.7.0
@@ -26,7 +24,7 @@ require (
 	github.com/deckarep/golang-set/v2 v2.5.0
 	github.com/dop251/goja v0.0.0-20230806174421-c933cf95e127
 	github.com/ethereum/c-kzg-4844 v0.4.0
-	github.com/fatih/color v1.13.0
+	github.com/fatih/color v1.16.0
 	github.com/fatih/structs v1.1.0
 	github.com/fjl/gencodec v0.0.0-20230517082657-f9840df7b83e
 	github.com/fjl/memsize v0.0.2
@@ -39,7 +37,7 @@ require (
 	github.com/golang/snappy v0.0.5-0.20220116011046-fa5810519dcb
 	github.com/google/gofuzz v1.2.0
 	github.com/google/pprof v0.0.0-20240207164012-fb44976bdcd5
-	github.com/google/uuid v1.4.0
+	github.com/google/uuid v1.5.0
 	github.com/gorilla/websocket v1.5.1
 	github.com/graph-gophers/graphql-go v1.3.0
 	github.com/hashicorp/go-bexpr v0.1.10
@@ -84,7 +82,7 @@ require (
 	golang.org/x/crypto v0.21.0
 	golang.org/x/exp v0.0.0-20240213143201-ec583247a57a
 	golang.org/x/sync v0.6.0
-	golang.org/x/sys v0.18.0
+	golang.org/x/sys v0.20.0
 	golang.org/x/text v0.14.0
 	golang.org/x/time v0.5.0
 	golang.org/x/tools v0.18.0
@@ -98,6 +96,7 @@ require (
 	github.com/Azure/azure-sdk-for-go/sdk/internal v1.3.0 // indirect
 	github.com/BurntSushi/toml v1.3.2 // indirect
 	github.com/DataDog/zstd v1.5.5 // indirect
+	github.com/allegro/bigcache v1.2.1 // indirect
 	github.com/aristanetworks/goarista v0.0.0-20200805130819-fd197cf57d96 // indirect
 	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.13 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.43 // indirect
@@ -160,7 +159,7 @@ require (
 	github.com/grpc-ecosystem/grpc-gateway/v2 v2.0.1 // indirect
 	github.com/gtank/merlin v0.1.1 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
-	github.com/hashicorp/go-retryablehttp v0.7.4 // indirect
+	github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
 	github.com/hashicorp/golang-lru/v2 v2.0.5 // indirect
 	github.com/herumi/bls-eth-go-binary v0.0.0-20210917013441-d37c07cfda4e // indirect
 	github.com/influxdata/line-protocol v0.0.0-20210311194329-9aa0e372d097 // indirect
@@ -233,12 +232,10 @@ require (
 	github.com/prometheus/common v0.47.0 // indirect
 	github.com/prometheus/procfs v0.12.0 // indirect
 	github.com/prometheus/prom2json v1.3.0 // indirect
-	github.com/prysmaticlabs/eth2-types v0.0.0-20210303084904-c9735a06829d // indirect
 	github.com/prysmaticlabs/fastssz v0.0.0-20221107182844-78142813af44 // indirect
 	github.com/prysmaticlabs/go-bitfield v0.0.0-20210809151128-385d8c5e3fb7 // indirect
 	github.com/prysmaticlabs/gohashtree v0.0.4-beta // indirect
 	github.com/prysmaticlabs/prombbolt v0.0.0-20210126082820-9b7adba6db7c // indirect
-	github.com/prysmaticlabs/prysm v0.0.0-20220124113610-e26cde5e091b // indirect
 	github.com/quic-go/qpack v0.4.0 // indirect
 	github.com/quic-go/quic-go v0.42.0 // indirect
 	github.com/quic-go/webtransport-go v0.6.0 // indirect
@@ -249,9 +246,10 @@ require (
 	github.com/russross/blackfriday/v2 v2.1.0 // indirect
 	github.com/sasha-s/go-deadlock v0.3.1 // indirect
 	github.com/schollz/progressbar/v3 v3.3.4 // indirect
-	github.com/sirupsen/logrus v1.9.0 // indirect
+	github.com/sirupsen/logrus v1.9.3 // indirect
 	github.com/spaolacci/murmur3 v1.1.0 // indirect
 	github.com/spf13/afero v1.10.0 // indirect
+	github.com/tecbot/gorocksdb v0.0.0-20191217155057-f0fad39f321c // indirect
 	github.com/thomaso-mirodin/intmath v0.0.0-20160323211736-5dc6d854e46e // indirect
 	github.com/tidwall/gjson v1.10.2 // indirect
 	github.com/tidwall/match v1.1.1 // indirect
@@ -266,7 +264,7 @@ require (
 	github.com/wealdtech/go-eth2-util v1.6.3 // indirect
 	github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
 	github.com/yusufpapurcu/wmi v1.2.3 // indirect
-	go.etcd.io/bbolt v1.3.7 // indirect
+	go.etcd.io/bbolt v1.3.9 // indirect
 	go.opencensus.io v0.24.0 // indirect
 	go.uber.org/dig v1.17.1 // indirect
 	go.uber.org/fx v1.20.1 // indirect
@@ -279,8 +277,10 @@ require (
 	golang.org/x/term v0.18.0 // indirect
 	google.golang.org/api v0.44.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
-	google.golang.org/grpc v1.56.3 // indirect
+	google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20231012201019-e917dd12ba7a // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20231016165738-49dd2c1f3d0b // indirect
+	google.golang.org/grpc v1.59.0 // indirect
 	google.golang.org/protobuf v1.33.0 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
@@ -295,8 +295,7 @@ require (
 )
 
 replace (
-	github.com/btcsuite/btcd => github.com/btcsuite/btcd v0.23.0
-	github.com/cometbft/cometbft => github.com/bnb-chain/greenfield-tendermint v0.0.0-20230417032003-4cda1f296fb2
+	github.com/cometbft/cometbft => github.com/bnb-chain/greenfield-cometbft v1.3.1
 	github.com/grpc-ecosystem/grpc-gateway/v2 => github.com/prysmaticlabs/grpc-gateway/v2 v2.3.1-0.20210702154020-550e1cd83ec1
 	github.com/syndtr/goleveldb v1.0.1 => github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
 	github.com/tendermint/tendermint => github.com/bnb-chain/tendermint v0.31.16