Compare commits

...

58 Commits

Author SHA1 Message Date
Roshan
4ab9babd11 update for test 2024-06-24 17:32:00 +08:00
Roshan
ea5f3ceaba fix: lint issue 2024-06-24 16:20:36 +08:00
Roshan
338cd0508f upgrade: add HaberFix hardfork 2024-06-24 16:10:12 +08:00
zzzckck
1548452def release: prepare for release v1.4.10 (#2526) 2024-06-21 15:31:43 +08:00
Nathan
f2e7f1dd24 fix: ensure empty withdrawals after cancun before broadcast (#2525) 2024-06-21 15:00:56 +08:00
Eric
27a3ec5d72 fix getBlobSidecars by ethclient (#2515) 2024-06-20 11:29:33 +08:00
zzzckck
6094d7157e UT: random failure of TestSnapSyncWithBlobs (#2519)
this case failed randomly in github CI, the ratio of around 20%
but can not be reproduced locally by:
```
  go test -p 1 -v -run TestSnapSyncWithBlobs -count 100
```
there could be a potential risk between `runEthPeer` and `runSnapExtension`,as
they shared the same handler.peers.
Add extra wait time for `runEthPeer` to resolve it.

And same for another UT: testBroadcastBlock
2024-06-20 11:14:52 +08:00
zzzckck
be0fbfb79e fix: remove zero gasprice check for BSC (#2518)
note: blobGasFeePrice can not be zero right now.
2024-06-19 19:51:30 +08:00
Nathan
00f094c37e core/forkchoice: improve stability when inturn block not generated (#2451) 2024-06-18 15:53:05 +08:00
Chris Li
7b08a70a23 perf: optimize chain commit performance for multi-database (#2509) 2024-06-18 15:20:23 +08:00
will-2012
99d31aeb28 perf: speedup pbss trienode read (#2508) 2024-06-18 11:47:22 +08:00
irrun
f467c6018b feat: add mev helper params and func (#2512) 2024-06-13 11:13:39 +08:00
zzzckck
aab4b8812a release: prepare for release v1.4.9 (#2510) 2024-06-11 10:47:44 +08:00
irrun
af7e9b95bd fix: waiting for the last simulation before pick best bid (#2507) 2024-06-07 16:41:20 +08:00
zoupingshi
1047f0e59a chore: fix function name (#2501)
Signed-off-by: zoupingshi <hellocatty@tom.com>
2024-06-05 15:12:48 +08:00
VM
9bb4fed1bf fix: add an empty freeze db (#2495) 2024-06-04 15:47:16 +08:00
zzzckck
35e71a769b doc: update url linker (#2499) 2024-05-30 14:08:48 +08:00
zzzckck
6b02ac7ac5 doc: update url linker (#2498) 2024-05-30 12:19:07 +08:00
Nathan
8b9558bb4d params/config: add Bohr hardfork (#2497) 2024-05-30 11:51:34 +08:00
Chris Li
63e7eac394 fix: keep 9W blocks in ancient db when prune block (#2481) 2024-05-29 15:11:52 +08:00
Chris Li
05543e558d fix: fix inspect database error (#2484) 2024-05-27 15:10:59 +08:00
Satyajit Das
b0146261c7 jsutils: faucet successful requests within blocks (#2470) 2024-05-23 15:08:36 +08:00
lx
d7b9866d3b Merge pull request #2487 from bnb-chain/master
merge PRs from master to develop branch
2024-05-22 13:56:19 +08:00
zzzckck
f5ba30ed47 release: prepare for release v1.4.8 (#2486) 2024-05-21 18:30:28 +08:00
Nathan
f190c49252 core/vm: add secp256r1 into PrecompiledContractsHaber (#2483) 2024-05-21 17:46:42 +08:00
Mars
08769ead2b dev: ensure consistency in BPS bundle result (#2479)
* dev: ensure consistency in BPS bundle result

* fix: remove env operation once the sim is discarded & rename
2024-05-21 12:24:41 +08:00
Matus Kysel
4d0f1e7117 RIP-7212: Precompile for secp256r1 Curve Support (#2400) 2024-05-21 11:44:21 +08:00
irrun
c77bb1110d fix: limit the gas price of the mev bid (#2473) 2024-05-20 14:33:47 +08:00
Mars
c856d21719 fix: move mev op to MinerAPI & add command to console (#2475) 2024-05-20 14:00:28 +08:00
Nathan
f45305b1ad cmd/utils: add a flag to change breathe block interval for testing (#2472) 2024-05-17 16:18:29 +08:00
Eric
d16532d678 internal/ethapi: add optional parameter for blobSidecars (#2468) 2024-05-16 19:07:14 +08:00
Eric
5edd032cdb internal/ethapi: add optional parameter for blobSidecars (#2467) 2024-05-16 19:06:49 +08:00
galaio
6b8cbbe172 sync: fix some sync issues caused by prune-block. (#2466) 2024-05-16 12:07:13 +08:00
setunapo
5ea2ada0ee utils: add check_blobtx.js (#2463) 2024-05-15 18:17:57 +08:00
Fynn
b230a02006 cmd: fix memory leak when big dataset (#2455) 2024-05-15 15:28:57 +08:00
Nathan
86e3a02490 cmd/utils: add a flag to change breathe block interval for testing (#2462) 2024-05-15 15:27:05 +08:00
Nathan
0c0958ff87 eth/handler: check lists in body before broadcast blocks (#2461) 2024-05-15 14:54:25 +08:00
buddho
c577ce3720 Merge pull request #2460 from bnb-chain/develop
merge some PRs for v1.4.7(2nd)
2024-05-14 19:59:57 +08:00
Eric
d436f9e2e8 eth/fetcher/block_fetcher.go: add metrics for import failed block (#2459) 2024-05-14 19:29:43 +08:00
Eric
97c3b9b267 internal/api.go: add choice about not show full blob (#2458) 2024-05-14 17:17:58 +08:00
Nathan
0560685460 core: clearup feynman testflag and rialto code (#2457) 2024-05-14 16:37:17 +08:00
rjl493456442
bf16a39876 ethdb/pebble: print warning log if pebble performance degrades (#29478) 2024-05-14 15:26:21 +08:00
Devon Bear
2c8720016d go.mod: bump pebble db to official release (#29038)
bump pebble
2024-05-14 15:26:21 +08:00
Nathan
f2ec3cc6a5 eth/handler: check blobs before broadcast blocks (#2450) 2024-05-13 17:21:45 +08:00
Martin HS
0a2e1282d2 core/rawdb: add sanity-limit to header accessor (#29534) 2024-05-13 17:21:45 +08:00
Nathan
adb5e8fe86 eth/filters: enforce topic-limit early on filter criterias (#29535) (#2448)
This PR adds a limit of 1000 to the "inner" topics in a filter-criteria

Co-authored-by: Martin HS <martin@swende.se>
2024-05-13 17:21:45 +08:00
Nathan
23f6194fad eth/handler: check blobs before broadcast blocks (#2450) 2024-05-13 16:20:50 +08:00
Martin HS
691d195526 core/rawdb: add sanity-limit to header accessor (#29534) 2024-05-11 10:24:39 +08:00
Nathan
b57c779759 eth/filters: enforce topic-limit early on filter criterias (#29535) (#2448)
This PR adds a limit of 1000 to the "inner" topics in a filter-criteria

Co-authored-by: Martin HS <martin@swende.se>
2024-05-11 10:24:19 +08:00
zzzckck
4ab1c865b2 Merge pull request #2441 from bnb-chain/develop
Draft release v1.4.7
2024-05-10 13:10:56 +08:00
zzzckck
a7d5b02919 release: prepare for release v1.4.7 (#2442)
* release: prepare for release v1.4.7

* code: avoid golang-lint error for TxLookupLimit
2024-05-09 19:47:52 +08:00
zzzckck
1ce9bb044d config: setup Mainnet Tycho(Cancun) hardfork date (#2439)
expected hard fork date:
Mainnet: 2024-06-20 06:05:00 AM UTC
2024-05-09 16:24:04 +08:00
dependabot[bot]
7948950f7a build(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 (#2411) 2024-05-09 16:06:28 +08:00
knowmost
0c101e618a chore: fix some typos (#2433) 2024-05-09 16:05:46 +08:00
zzzckck
571ea2c4b9 nancy: add files .nancy-ignore (#2440) 2024-05-09 15:58:06 +08:00
galaio
7bc5a3353d txpool: limit max gas when mining is enabled; (#2435) 2024-05-09 15:54:31 +08:00
Chris Li
901ea2e0d2 fix: performance issue when load journal (#2438) 2024-05-08 15:36:01 +08:00
buddho
1d81f3316f metrics: add blockInsertMgaspsGauge to trace mgasps (#2396) 2024-04-30 17:58:04 +08:00
109 changed files with 7300 additions and 722 deletions

1
.nancy-ignore Normal file
View File

@@ -0,0 +1 @@
CVE-2024-34478 # "CWE-754: Improper Check for Unusual or Exceptional Conditions." This vulnerability is BTC only, BSC does not have the issue.

View File

@@ -1,4 +1,67 @@
# Changelog # Changelog
## v1.4.10
### FEATURE
NA
### IMPROVEMENT
* [\#2512](https://github.com/bnb-chain/bsc/pull/2512) feat: add mev helper params and func
* [\#2508](https://github.com/bnb-chain/bsc/pull/2508) perf: speedup pbss trienode read
* [\#2509](https://github.com/bnb-chain/bsc/pull/2509) perf: optimize chain commit performance for multi-database
* [\#2451](https://github.com/bnb-chain/bsc/pull/2451) core/forkchoice: improve stability when inturn block not generate
### BUGFIX
* [\#2518](https://github.com/bnb-chain/bsc/pull/2518) fix: remove zero gasprice check for BSC
* [\#2519](https://github.com/bnb-chain/bsc/pull/2519) UT: random failure of TestSnapSyncWithBlobs
* [\#2515](https://github.com/bnb-chain/bsc/pull/2515) fix getBlobSidecars by ethclient
* [\#2525](https://github.com/bnb-chain/bsc/pull/2525) fix: ensure empty withdrawals after cancun before broadcast
## v1.4.9
### FEATURE
* [\#2463](https://github.com/bnb-chain/bsc/pull/2463) utils: add check_blobtx.js
* [\#2470](https://github.com/bnb-chain/bsc/pull/2470) jsutils: faucet successful requests within blocks
* [\#2467](https://github.com/bnb-chain/bsc/pull/2467) internal/ethapi: add optional parameter for blobSidecars
### IMPROVEMENT
* [\#2462](https://github.com/bnb-chain/bsc/pull/2462) cmd/utils: add a flag to change breathe block interval for testing
* [\#2497](https://github.com/bnb-chain/bsc/pull/2497) params/config: add Bohr hardfork
* [\#2479](https://github.com/bnb-chain/bsc/pull/2479) dev: ensure consistency in BPS bundle result
### BUGFIX
* [\#2461](https://github.com/bnb-chain/bsc/pull/2461) eth/handler: check lists in body before broadcast blocks
* [\#2455](https://github.com/bnb-chain/bsc/pull/2455) cmd: fix memory leak when big dataset
* [\#2466](https://github.com/bnb-chain/bsc/pull/2466) sync: fix some sync issues caused by prune-block.
* [\#2475](https://github.com/bnb-chain/bsc/pull/2475) fix: move mev op to MinerAPI & add command to console
* [\#2473](https://github.com/bnb-chain/bsc/pull/2473) fix: limit the gas price of the mev bid
* [\#2484](https://github.com/bnb-chain/bsc/pull/2484) fix: fix inspect database error
* [\#2481](https://github.com/bnb-chain/bsc/pull/2481) fix: keep 9W blocks in ancient db when prune block
* [\#2495](https://github.com/bnb-chain/bsc/pull/2495) fix: add an empty freeze db
* [\#2507](https://github.com/bnb-chain/bsc/pull/2507) fix: waiting for the last simulation before pick best bid
## v1.4.8
### FEATURE
* [\#2483](https://github.com/bnb-chain/bsc/pull/2483) core/vm: add secp256r1 into PrecompiledContractsHaber
* [\#2400](https://github.com/bnb-chain/bsc/pull/2400) RIP-7212: Precompile for secp256r1 Curve Support
### IMPROVEMENT
NA
### BUGFIX
NA
## v1.4.7
### FEATURE
* [\#2439](https://github.com/bnb-chain/bsc/pull/2439) config: setup Mainnet Tycho(Cancun) hardfork date
### IMPROVEMENT
* [\#2396](https://github.com/bnb-chain/bsc/pull/2396) metrics: add blockInsertMgaspsGauge to trace mgasps
* [\#2411](https://github.com/bnb-chain/bsc/pull/2411) build(deps): bump golang.org/x/net from 0.19.0 to 0.23.0
* [\#2435](https://github.com/bnb-chain/bsc/pull/2435) txpool: limit max gas when mining is enabled
* [\#2438](https://github.com/bnb-chain/bsc/pull/2438) fix: performance issue when load journal
* [\#2440](https://github.com/bnb-chain/bsc/pull/2440) nancy: add files .nancy-ignore
### BUGFIX
NA
## v1.4.6 ## v1.4.6
### FEATURE ### FEATURE
* [\#2227](https://github.com/bnb-chain/bsc/pull/2227) core: separated databases for block data * [\#2227](https://github.com/bnb-chain/bsc/pull/2227) core: separated databases for block data

View File

@@ -36,7 +36,7 @@ To combine DPoS and PoA for consensus, BNB Smart Chain implement a novel consens
2. Validators take turns to produce blocks in a PoA manner, similar to Ethereum's Clique consensus engine. 2. Validators take turns to produce blocks in a PoA manner, similar to Ethereum's Clique consensus engine.
3. Validator set are elected in and out based on a staking based governance on BNB Beacon Chain. 3. Validator set are elected in and out based on a staking based governance on BNB Beacon Chain.
4. The validator set change is relayed via a cross-chain communication mechanism. 4. The validator set change is relayed via a cross-chain communication mechanism.
5. Parlia consensus engine will interact with a set of [system contracts](https://docs.bnbchain.org/docs/learn/system-contract) to achieve liveness slash, revenue distributing and validator set renewing func. 5. Parlia consensus engine will interact with a set of [system contracts](https://docs.bnbchain.org/bnb-smart-chain/staking/overview/#system-contracts) to achieve liveness slash, revenue distributing and validator set renewing func.
### Light Client of BNB Beacon Chain ### Light Client of BNB Beacon Chain
@@ -183,7 +183,7 @@ This tool is optional and if you leave it out you can always attach to an alread
#### 7. More #### 7. More
More details about [running a node](https://docs.bnbchain.org/docs/validator/fullnode) and [becoming a validator](https://docs.bnbchain.org/docs/validator/create-val) More details about [running a node](https://docs.bnbchain.org/bnb-smart-chain/developers/node_operators/full_node/) and [becoming a validator](https://docs.bnbchain.org/bnb-smart-chain/validator/create-val/)
*Note: Although some internal protective measures prevent transactions from *Note: Although some internal protective measures prevent transactions from
crossing over between the main network and test network, you should always crossing over between the main network and test network, you should always

View File

@@ -63,6 +63,8 @@ var (
Flags: flags.Merge([]cli.Flag{ Flags: flags.Merge([]cli.Flag{
utils.CachePreimagesFlag, utils.CachePreimagesFlag,
utils.OverrideCancun, utils.OverrideCancun,
utils.OverrideHaber,
utils.OverrideBohr,
utils.OverrideVerkle, utils.OverrideVerkle,
}, utils.DatabaseFlags), }, utils.DatabaseFlags),
Description: ` Description: `
@@ -256,6 +258,14 @@ func initGenesis(ctx *cli.Context) error {
v := ctx.Uint64(utils.OverrideCancun.Name) v := ctx.Uint64(utils.OverrideCancun.Name)
overrides.OverrideCancun = &v overrides.OverrideCancun = &v
} }
if ctx.IsSet(utils.OverrideHaber.Name) {
v := ctx.Uint64(utils.OverrideHaber.Name)
overrides.OverrideHaber = &v
}
if ctx.IsSet(utils.OverrideBohr.Name) {
v := ctx.Uint64(utils.OverrideBohr.Name)
overrides.OverrideBohr = &v
}
if ctx.IsSet(utils.OverrideVerkle.Name) { if ctx.IsSet(utils.OverrideVerkle.Name) {
v := ctx.Uint64(utils.OverrideVerkle.Name) v := ctx.Uint64(utils.OverrideVerkle.Name)
overrides.OverrideVerkle = &v overrides.OverrideVerkle = &v

View File

@@ -189,18 +189,18 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
v := ctx.Uint64(utils.OverrideCancun.Name) v := ctx.Uint64(utils.OverrideCancun.Name)
cfg.Eth.OverrideCancun = &v cfg.Eth.OverrideCancun = &v
} }
if ctx.IsSet(utils.OverrideHaber.Name) {
v := ctx.Uint64(utils.OverrideHaber.Name)
cfg.Eth.OverrideHaber = &v
}
if ctx.IsSet(utils.OverrideBohr.Name) {
v := ctx.Uint64(utils.OverrideBohr.Name)
cfg.Eth.OverrideBohr = &v
}
if ctx.IsSet(utils.OverrideVerkle.Name) { if ctx.IsSet(utils.OverrideVerkle.Name) {
v := ctx.Uint64(utils.OverrideVerkle.Name) v := ctx.Uint64(utils.OverrideVerkle.Name)
cfg.Eth.OverrideVerkle = &v cfg.Eth.OverrideVerkle = &v
} }
if ctx.IsSet(utils.OverrideFeynman.Name) {
v := ctx.Uint64(utils.OverrideFeynman.Name)
cfg.Eth.OverrideFeynman = &v
}
if ctx.IsSet(utils.OverrideFeynmanFix.Name) {
v := ctx.Uint64(utils.OverrideFeynmanFix.Name)
cfg.Eth.OverrideFeynmanFix = &v
}
if ctx.IsSet(utils.OverrideFullImmutabilityThreshold.Name) { if ctx.IsSet(utils.OverrideFullImmutabilityThreshold.Name) {
params.FullImmutabilityThreshold = ctx.Uint64(utils.OverrideFullImmutabilityThreshold.Name) params.FullImmutabilityThreshold = ctx.Uint64(utils.OverrideFullImmutabilityThreshold.Name)
downloader.FullMaxForkAncestry = ctx.Uint64(utils.OverrideFullImmutabilityThreshold.Name) downloader.FullMaxForkAncestry = ctx.Uint64(utils.OverrideFullImmutabilityThreshold.Name)
@@ -211,6 +211,9 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
if ctx.IsSet(utils.OverrideDefaultExtraReserveForBlobRequests.Name) { if ctx.IsSet(utils.OverrideDefaultExtraReserveForBlobRequests.Name) {
params.DefaultExtraReserveForBlobRequests = ctx.Uint64(utils.OverrideDefaultExtraReserveForBlobRequests.Name) params.DefaultExtraReserveForBlobRequests = ctx.Uint64(utils.OverrideDefaultExtraReserveForBlobRequests.Name)
} }
if ctx.IsSet(utils.OverrideBreatheBlockInterval.Name) {
params.BreatheBlockInterval = ctx.Uint64(utils.OverrideBreatheBlockInterval.Name)
}
backend, eth := utils.RegisterEthService(stack, &cfg.Eth) backend, eth := utils.RegisterEthService(stack, &cfg.Eth)

View File

@@ -106,12 +106,12 @@ Remove blockchain and state databases`,
dbInspectTrieCmd = &cli.Command{ dbInspectTrieCmd = &cli.Command{
Action: inspectTrie, Action: inspectTrie,
Name: "inspect-trie", Name: "inspect-trie",
ArgsUsage: "<blocknum> <jobnum>", ArgsUsage: "<blocknum> <jobnum> <topn>",
Flags: []cli.Flag{ Flags: []cli.Flag{
utils.DataDirFlag, utils.DataDirFlag,
utils.SyncModeFlag, utils.SyncModeFlag,
}, },
Usage: "Inspect the MPT tree of the account and contract.", Usage: "Inspect the MPT tree of the account and contract. 'blocknum' can be latest/snapshot/number. 'topn' means output the top N storage tries info ranked by the total number of TrieNodes",
Description: `This commands iterates the entrie WorldState.`, Description: `This commands iterates the entrie WorldState.`,
} }
dbCheckStateContentCmd = &cli.Command{ dbCheckStateContentCmd = &cli.Command{
@@ -386,6 +386,7 @@ func inspectTrie(ctx *cli.Context) error {
blockNumber uint64 blockNumber uint64
trieRootHash common.Hash trieRootHash common.Hash
jobnum uint64 jobnum uint64
topN uint64
) )
stack, _ := makeConfigNode(ctx) stack, _ := makeConfigNode(ctx)
@@ -405,24 +406,37 @@ func inspectTrie(ctx *cli.Context) error {
var err error var err error
blockNumber, err = strconv.ParseUint(ctx.Args().Get(0), 10, 64) blockNumber, err = strconv.ParseUint(ctx.Args().Get(0), 10, 64)
if err != nil { if err != nil {
return fmt.Errorf("failed to Parse blocknum, Args[0]: %v, err: %v", ctx.Args().Get(0), err) return fmt.Errorf("failed to parse blocknum, Args[0]: %v, err: %v", ctx.Args().Get(0), err)
} }
} }
if ctx.NArg() == 1 { if ctx.NArg() == 1 {
jobnum = 1000 jobnum = 1000
topN = 10
} else if ctx.NArg() == 2 {
var err error
jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64)
if err != nil {
return fmt.Errorf("failed to parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
}
topN = 10
} else { } else {
var err error var err error
jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64) jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64)
if err != nil { if err != nil {
return fmt.Errorf("failed to Parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err) return fmt.Errorf("failed to parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
}
topN, err = strconv.ParseUint(ctx.Args().Get(2), 10, 64)
if err != nil {
return fmt.Errorf("failed to parse topn, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
} }
} }
if blockNumber != math.MaxUint64 { if blockNumber != math.MaxUint64 {
headerBlockHash = rawdb.ReadCanonicalHash(db, blockNumber) headerBlockHash = rawdb.ReadCanonicalHash(db, blockNumber)
if headerBlockHash == (common.Hash{}) { if headerBlockHash == (common.Hash{}) {
return errors.New("ReadHeadBlockHash empry hash") return errors.New("ReadHeadBlockHash empty hash")
} }
blockHeader := rawdb.ReadHeader(db, headerBlockHash, blockNumber) blockHeader := rawdb.ReadHeader(db, headerBlockHash, blockNumber)
trieRootHash = blockHeader.Root trieRootHash = blockHeader.Root
@@ -437,6 +451,7 @@ func inspectTrie(ctx *cli.Context) error {
if dbScheme == rawdb.PathScheme { if dbScheme == rawdb.PathScheme {
config = &triedb.Config{ config = &triedb.Config{
PathDB: utils.PathDBConfigAddJournalFilePath(stack, pathdb.ReadOnly), PathDB: utils.PathDBConfigAddJournalFilePath(stack, pathdb.ReadOnly),
Cache: 0,
} }
} else if dbScheme == rawdb.HashScheme { } else if dbScheme == rawdb.HashScheme {
config = triedb.HashDefaults config = triedb.HashDefaults
@@ -448,7 +463,7 @@ func inspectTrie(ctx *cli.Context) error {
fmt.Printf("fail to new trie tree, err: %v, rootHash: %v\n", err, trieRootHash.String()) fmt.Printf("fail to new trie tree, err: %v, rootHash: %v\n", err, trieRootHash.String())
return err return err
} }
theInspect, err := trie.NewInspector(theTrie, triedb, trieRootHash, blockNumber, jobnum) theInspect, err := trie.NewInspector(theTrie, triedb, trieRootHash, blockNumber, jobnum, int(topN))
if err != nil { if err != nil {
return err return err
} }
@@ -493,7 +508,7 @@ func ancientInspect(ctx *cli.Context) error {
stack, _ := makeConfigNode(ctx) stack, _ := makeConfigNode(ctx)
defer stack.Close() defer stack.Close()
db := utils.MakeChainDatabase(ctx, stack, true, true) db := utils.MakeChainDatabase(ctx, stack, true, false)
defer db.Close() defer db.Close()
return rawdb.AncientInspect(db) return rawdb.AncientInspect(db)
} }

View File

@@ -25,6 +25,8 @@ import (
"strings" "strings"
"time" "time"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/accounts" "github.com/ethereum/go-ethereum/accounts"
"github.com/ethereum/go-ethereum/accounts/keystore" "github.com/ethereum/go-ethereum/accounts/keystore"
"github.com/ethereum/go-ethereum/cmd/utils" "github.com/ethereum/go-ethereum/cmd/utils"
@@ -71,12 +73,13 @@ var (
utils.SmartCardDaemonPathFlag, utils.SmartCardDaemonPathFlag,
utils.RialtoHash, utils.RialtoHash,
utils.OverrideCancun, utils.OverrideCancun,
utils.OverrideHaber,
utils.OverrideBohr,
utils.OverrideVerkle, utils.OverrideVerkle,
utils.OverrideFeynman,
utils.OverrideFeynmanFix,
utils.OverrideFullImmutabilityThreshold, utils.OverrideFullImmutabilityThreshold,
utils.OverrideMinBlocksForBlobRequests, utils.OverrideMinBlocksForBlobRequests,
utils.OverrideDefaultExtraReserveForBlobRequests, utils.OverrideDefaultExtraReserveForBlobRequests,
utils.OverrideBreatheBlockInterval,
utils.EnablePersonal, utils.EnablePersonal,
utils.TxPoolLocalsFlag, utils.TxPoolLocalsFlag,
utils.TxPoolNoLocalsFlag, utils.TxPoolNoLocalsFlag,
@@ -456,6 +459,10 @@ func startNode(ctx *cli.Context, stack *node.Node, backend ethapi.Backend, isCon
// Set the gas price to the limits from the CLI and start mining // Set the gas price to the limits from the CLI and start mining
gasprice := flags.GlobalBig(ctx, utils.MinerGasPriceFlag.Name) gasprice := flags.GlobalBig(ctx, utils.MinerGasPriceFlag.Name)
ethBackend.TxPool().SetGasTip(gasprice) ethBackend.TxPool().SetGasTip(gasprice)
gasCeil := ethBackend.Miner().GasCeil()
if gasCeil > params.SystemTxsGas {
ethBackend.TxPool().SetMaxGas(gasCeil - params.SystemTxsGas)
}
if err := ethBackend.StartMining(); err != nil { if err := ethBackend.StartMining(); err != nil {
utils.Fatalf("Failed to start mining: %v", err) utils.Fatalf("Failed to start mining: %v", err)
} }

View File

@@ -75,7 +75,7 @@ func NewLevelDBDatabaseWithFreezer(file string, cache int, handles int, ancient
if err != nil { if err != nil {
return nil, err return nil, err
} }
frdb, err := rawdb.NewDatabaseWithFreezer(kvdb, ancient, namespace, readonly, disableFreeze, isLastOffset, pruneAncientData) frdb, err := rawdb.NewDatabaseWithFreezer(kvdb, ancient, namespace, readonly, disableFreeze, isLastOffset, pruneAncientData, false)
if err != nil { if err != nil {
kvdb.Close() kvdb.Close()
return nil, err return nil, err
@@ -178,11 +178,10 @@ func BlockchainCreator(t *testing.T, chaindbPath, AncientPath string, blockRemai
// Force run a freeze cycle // Force run a freeze cycle
type freezer interface { type freezer interface {
Freeze() error Freeze(threshold uint64) error
Ancients() (uint64, error) Ancients() (uint64, error)
} }
blockchain.SetFinalized(blocks[len(blocks)-1].Header()) db.(freezer).Freeze(10)
db.(freezer).Freeze()
frozen, err := db.Ancients() frozen, err := db.Ancients()
//make sure there're frozen items //make sure there're frozen items

View File

@@ -43,9 +43,11 @@ import (
"github.com/ethereum/go-ethereum/log" "github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/metrics" "github.com/ethereum/go-ethereum/metrics"
"github.com/ethereum/go-ethereum/node" "github.com/ethereum/go-ethereum/node"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rlp" "github.com/ethereum/go-ethereum/rlp"
"github.com/ethereum/go-ethereum/trie" "github.com/ethereum/go-ethereum/trie"
"github.com/ethereum/go-ethereum/triedb" "github.com/ethereum/go-ethereum/triedb"
"github.com/ethereum/go-ethereum/triedb/pathdb"
cli "github.com/urfave/cli/v2" cli "github.com/urfave/cli/v2"
) )
@@ -245,7 +247,16 @@ func accessDb(ctx *cli.Context, stack *node.Node) (ethdb.Database, error) {
NoBuild: true, NoBuild: true,
AsyncBuild: false, AsyncBuild: false,
} }
snaptree, err := snapshot.New(snapconfig, chaindb, triedb.NewDatabase(chaindb, nil), headBlock.Root(), TriesInMemory, false) dbScheme := rawdb.ReadStateScheme(chaindb)
var config *triedb.Config
if dbScheme == rawdb.PathScheme {
config = &triedb.Config{
PathDB: utils.PathDBConfigAddJournalFilePath(stack, pathdb.ReadOnly),
}
} else if dbScheme == rawdb.HashScheme {
config = triedb.HashDefaults
}
snaptree, err := snapshot.New(snapconfig, chaindb, triedb.NewDatabase(chaindb, config), headBlock.Root(), TriesInMemory, false)
if err != nil { if err != nil {
log.Error("snaptree error", "err", err) log.Error("snaptree error", "err", err)
return nil, err // The relevant snapshot(s) might not exist return nil, err // The relevant snapshot(s) might not exist
@@ -333,6 +344,9 @@ func pruneBlock(ctx *cli.Context) error {
stack, config = makeConfigNode(ctx) stack, config = makeConfigNode(ctx)
defer stack.Close() defer stack.Close()
blockAmountReserved = ctx.Uint64(utils.BlockAmountReserved.Name) blockAmountReserved = ctx.Uint64(utils.BlockAmountReserved.Name)
if blockAmountReserved < params.FullImmutabilityThreshold {
return fmt.Errorf("block-amount-reserved must be greater than or equal to %d", params.FullImmutabilityThreshold)
}
chaindb, err = accessDb(ctx, stack) chaindb, err = accessDb(ctx, stack)
if err != nil { if err != nil {
return err return err
@@ -409,7 +423,7 @@ func pruneBlock(ctx *cli.Context) error {
} }
if _, err := os.Stat(newAncientPath); err == nil { if _, err := os.Stat(newAncientPath); err == nil {
// No file lock found for old ancientDB but new ancientDB exsisted, indicating the geth was interrupted // No file lock found for old ancientDB but new ancientDB existed, indicating the geth was interrupted
// after old ancientDB removal, this happened after backup successfully, so just rename the new ancientDB // after old ancientDB removal, this happened after backup successfully, so just rename the new ancientDB
if err := blockpruner.AncientDbReplacer(); err != nil { if err := blockpruner.AncientDbReplacer(); err != nil {
log.Error("Failed to rename new ancient directory") log.Error("Failed to rename new ancient directory")

View File

@@ -0,0 +1,51 @@
import { ethers } from "ethers";
import program from "commander";
// depends on ethjs v6.11.0+ for 4844, https://github.com/ethers-io/ethers.js/releases/tag/v6.11.0
// BSC testnet enabled 4844 on block: 39539137
// Usage:
// nvm use 20
// node check_blobtx.js --rpc https://data-seed-prebsc-1-s1.binance.org:8545 --startNum 39539137
// node check_blobtx.js --rpc https://data-seed-prebsc-1-s1.binance.org:8545 --startNum 39539137 --endNum 40345994
program.option("--rpc <Rpc>", "Rpc Server URL");
program.option("--startNum <Num>", "start block", 0);
program.option("--endNum <Num>", "end block", 0);
program.parse(process.argv);
const provider = new ethers.JsonRpcProvider(program.rpc);
const main = async () => {
var startBlock = parseInt(program.startNum)
var endBlock = parseInt(program.endNum)
if (isNaN(endBlock) || isNaN(startBlock) || startBlock == 0) {
console.error("invalid input, --startNum", program.startNum, "--end", program.endNum)
return
}
// if --endNum is not specified, set it to the latest block number.
if (endBlock == 0) {
endBlock = await provider.getBlockNumber();
}
if (startBlock > endBlock) {
console.error("invalid input, startBlock:",startBlock, " endBlock:", endBlock);
return
}
for (let i = startBlock; i <= endBlock; i++) {
let blockData = await provider.getBlock(i);
console.log("startBlock:",startBlock, "endBlock:", endBlock, "curBlock", i, "blobGasUsed", blockData.blobGasUsed);
if (blockData.blobGasUsed == 0) {
continue
}
for (let txIndex = 0; txIndex<= blockData.transactions.length - 1; txIndex++) {
let txHash = blockData.transactions[txIndex]
let txData = await provider.getTransaction(txHash);
if (txData.type == 3) {
console.log("BlobTx in block:",i, " txIndex:", txIndex, " txHash:", txHash);
}
}
}
};
main().then(() => process.exit(0))
.catch((error) => {
console.error(error);
process.exit(1);
});

View File

@@ -0,0 +1,49 @@
import { ethers } from "ethers";
import program from "commander";
// Usage:
// node faucet_request.js --rpc localhost:8545 --startNum 39539137
// node faucet_request.js --rpc localhost:8545 --startNum 39539137 --endNum 40345994
// node faucet_request.js --rpc https://data-seed-prebsc-1-s1.bnbchain.org:8545 --startNum 39539137 --endNum 40345994
program.option("--rpc <Rpc>", "Rpc Server URL");
program.option("--startNum <Num>", "start block", 0);
program.option("--endNum <Num>", "end block", 0);
program.parse(process.argv);
const provider = new ethers.JsonRpcProvider(program.rpc);
const main = async () => {
var startBlock = parseInt(program.startNum)
var endBlock = parseInt(program.endNum)
if (isNaN(endBlock) || isNaN(startBlock) || startBlock == 0) {
console.error("invalid input, --startNum", program.startNum, "--end", program.endNum)
return
}
// if --endNum is not specified, set it to the latest block number.
if (endBlock == 0) {
endBlock = await provider.getBlockNumber();
}
if (startBlock > endBlock) {
console.error("invalid input, startBlock:",startBlock, " endBlock:", endBlock);
return
}
let startBalance = await provider.getBalance("0xaa25Aa7a19f9c426E07dee59b12f944f4d9f1DD3", startBlock)
let endBalance = await provider.getBalance("0xaa25Aa7a19f9c426E07dee59b12f944f4d9f1DD3", endBlock)
const faucetAmount = BigInt(0.3 * 10**18); // Convert 0.3 ether to wei as a BigInt
const numFaucetRequest = (startBalance - endBalance) / faucetAmount;
// Convert BigInt to ether
const startBalanceEth = Number(startBalance) / 10**18;
const endBalanceEth = Number(endBalance) / 10**18;
console.log(`Start Balance: ${startBalanceEth} ETH`);
console.log(`End Balance: ${endBalanceEth} ETH`);
console.log("successful faucet request: ",numFaucetRequest);
};
main().then(() => process.exit(0))
.catch((error) => {
console.error(error);
process.exit(1);
});

View File

@@ -310,20 +310,21 @@ var (
Usage: "Manually specify the Cancun fork timestamp, overriding the bundled setting", Usage: "Manually specify the Cancun fork timestamp, overriding the bundled setting",
Category: flags.EthCategory, Category: flags.EthCategory,
} }
OverrideHaber = &cli.Uint64Flag{
Name: "override.haber",
Usage: "Manually specify the Haber fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideBohr = &cli.Uint64Flag{
Name: "override.bohr",
Usage: "Manually specify the Bohr fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideVerkle = &cli.Uint64Flag{ OverrideVerkle = &cli.Uint64Flag{
Name: "override.verkle", Name: "override.verkle",
Usage: "Manually specify the Verkle fork timestamp, overriding the bundled setting", Usage: "Manually specify the Verkle fork timestamp, overriding the bundled setting",
Category: flags.EthCategory, Category: flags.EthCategory,
} }
OverrideFeynman = &cli.Uint64Flag{
Name: "override.feynman",
Usage: "Manually specify the Feynman fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideFeynmanFix = &cli.Uint64Flag{
Name: "override.feynmanfix",
Usage: "Manually specify the FeynmanFix fork timestamp, overriding the bundled setting",
}
OverrideFullImmutabilityThreshold = &cli.Uint64Flag{ OverrideFullImmutabilityThreshold = &cli.Uint64Flag{
Name: "override.immutabilitythreshold", Name: "override.immutabilitythreshold",
Usage: "It is the number of blocks after which a chain segment is considered immutable, only for testing purpose", Usage: "It is the number of blocks after which a chain segment is considered immutable, only for testing purpose",
@@ -342,6 +343,12 @@ var (
Value: params.DefaultExtraReserveForBlobRequests, Value: params.DefaultExtraReserveForBlobRequests,
Category: flags.EthCategory, Category: flags.EthCategory,
} }
OverrideBreatheBlockInterval = &cli.Uint64Flag{
Name: "override.breatheblockinterval",
Usage: "It changes the interval between breathe blocks, only for testing purpose",
Value: params.BreatheBlockInterval,
Category: flags.EthCategory,
}
SyncModeFlag = &flags.TextMarshalerFlag{ SyncModeFlag = &flags.TextMarshalerFlag{
Name: "syncmode", Name: "syncmode",
Usage: `Blockchain sync mode ("snap" or "full")`, Usage: `Blockchain sync mode ("snap" or "full")`,
@@ -1080,6 +1087,7 @@ Please note that --` + MetricsHTTPFlag.Name + ` must be set to start the server.
Name: "block-amount-reserved", Name: "block-amount-reserved",
Usage: "Sets the expected remained amount of blocks for offline block prune", Usage: "Sets the expected remained amount of blocks for offline block prune",
Category: flags.BlockHistoryCategory, Category: flags.BlockHistoryCategory,
Value: params.FullImmutabilityThreshold,
} }
CheckSnapshotWithMPT = &cli.BoolFlag{ CheckSnapshotWithMPT = &cli.BoolFlag{

View File

@@ -163,7 +163,7 @@ func TestHistoryImportAndExport(t *testing.T) {
// Now import Era. // Now import Era.
freezer := t.TempDir() freezer := t.TempDir()
db2, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), freezer, "", false, false, false, false) db2, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), freezer, "", false, false, false, false, false)
if err != nil { if err != nil {
panic(err) panic(err)
} }

View File

@@ -15,14 +15,13 @@ import (
"github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/internal/ethapi" "github.com/ethereum/go-ethereum/internal/ethapi"
"github.com/ethereum/go-ethereum/log" "github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rpc" "github.com/ethereum/go-ethereum/rpc"
) )
const SecondsPerDay uint64 = 86400
// the params should be two blocks' time(timestamp) // the params should be two blocks' time(timestamp)
func sameDayInUTC(first, second uint64) bool { func sameDayInUTC(first, second uint64) bool {
return first/SecondsPerDay == second/SecondsPerDay return first/params.BreatheBlockInterval == second/params.BreatheBlockInterval
} }
func isBreatheBlock(lastBlockTime, blockTime uint64) bool { func isBreatheBlock(lastBlockTime, blockTime uint64) bool {

View File

@@ -66,6 +66,31 @@ func NewBlockValidator(config *params.ChainConfig, blockchain *BlockChain, engin
return validator return validator
} }
// ValidateListsInBody validates that UncleHash, WithdrawalsHash, and WithdrawalsHash correspond to the lists in the block body, respectively.
func ValidateListsInBody(block *types.Block) error {
header := block.Header()
if hash := types.CalcUncleHash(block.Uncles()); hash != header.UncleHash {
return fmt.Errorf("uncle root hash mismatch (header value %x, calculated %x)", header.UncleHash, hash)
}
if hash := types.DeriveSha(block.Transactions(), trie.NewStackTrie(nil)); hash != header.TxHash {
return fmt.Errorf("transaction root hash mismatch: have %x, want %x", hash, header.TxHash)
}
// Withdrawals are present after the Shanghai fork.
if header.WithdrawalsHash != nil {
// Withdrawals list must be present in body after Shanghai.
if block.Withdrawals() == nil {
return errors.New("missing withdrawals in block body")
}
if hash := types.DeriveSha(block.Withdrawals(), trie.NewStackTrie(nil)); hash != *header.WithdrawalsHash {
return fmt.Errorf("withdrawals root hash mismatch (header value %x, calculated %x)", *header.WithdrawalsHash, hash)
}
} else if block.Withdrawals() != nil { // Withdrawals turn into empty from nil when BlockBody has Sidecars
// Withdrawals are not allowed prior to shanghai fork
return errors.New("withdrawals present in block body")
}
return nil
}
// ValidateBody validates the given block's uncles and verifies the block // ValidateBody validates the given block's uncles and verifies the block
// header's transaction and uncle roots. The headers are assumed to be already // header's transaction and uncle roots. The headers are assumed to be already
// validated at this point. // validated at this point.
@@ -83,31 +108,12 @@ func (v *BlockValidator) ValidateBody(block *types.Block) error {
if err := v.engine.VerifyUncles(v.bc, block); err != nil { if err := v.engine.VerifyUncles(v.bc, block); err != nil {
return err return err
} }
if hash := types.CalcUncleHash(block.Uncles()); hash != header.UncleHash {
return fmt.Errorf("uncle root hash mismatch (header value %x, calculated %x)", header.UncleHash, hash)
}
validateFuns := []func() error{ validateFuns := []func() error{
func() error { func() error {
if hash := types.DeriveSha(block.Transactions(), trie.NewStackTrie(nil)); hash != header.TxHash { return ValidateListsInBody(block)
return fmt.Errorf("transaction root hash mismatch: have %x, want %x", hash, header.TxHash)
}
return nil
}, },
func() error { func() error {
// Withdrawals are present after the Shanghai fork.
if header.WithdrawalsHash != nil {
// Withdrawals list must be present in body after Shanghai.
if block.Withdrawals() == nil {
return errors.New("missing withdrawals in block body")
}
if hash := types.DeriveSha(block.Withdrawals(), trie.NewStackTrie(nil)); hash != *header.WithdrawalsHash {
return fmt.Errorf("withdrawals root hash mismatch (header value %x, calculated %x)", *header.WithdrawalsHash, hash)
}
} else if block.Withdrawals() != nil { // Withdrawals turn into empty from nil when BlockBody has Sidecars
// Withdrawals are not allowed prior to shanghai fork
return errors.New("withdrawals present in block body")
}
// Blob transactions may be present after the Cancun fork. // Blob transactions may be present after the Cancun fork.
var blobs int var blobs int
for i, tx := range block.Transactions() { for i, tx := range block.Transactions() {

View File

@@ -71,6 +71,8 @@ var (
justifiedBlockGauge = metrics.NewRegisteredGauge("chain/head/justified", nil) justifiedBlockGauge = metrics.NewRegisteredGauge("chain/head/justified", nil)
finalizedBlockGauge = metrics.NewRegisteredGauge("chain/head/finalized", nil) finalizedBlockGauge = metrics.NewRegisteredGauge("chain/head/finalized", nil)
blockInsertMgaspsGauge = metrics.NewRegisteredGauge("chain/insert/mgasps", nil)
chainInfoGauge = metrics.NewRegisteredGaugeInfo("chain/info", nil) chainInfoGauge = metrics.NewRegisteredGaugeInfo("chain/info", nil)
accountReadTimer = metrics.NewRegisteredTimer("chain/account/reads", nil) accountReadTimer = metrics.NewRegisteredTimer("chain/account/reads", nil)
@@ -299,6 +301,7 @@ type BlockChain struct {
diffLayerFreezerBlockLimit uint64 diffLayerFreezerBlockLimit uint64
wg sync.WaitGroup wg sync.WaitGroup
dbWg sync.WaitGroup
quit chan struct{} // shutdown signal, closed in Stop. quit chan struct{} // shutdown signal, closed in Stop.
stopping atomic.Bool // false if chain is running, true when stopped stopping atomic.Bool // false if chain is running, true when stopped
procInterrupt atomic.Bool // interrupt signaler for block processing procInterrupt atomic.Bool // interrupt signaler for block processing
@@ -667,7 +670,7 @@ func (bc *BlockChain) cacheBlock(hash common.Hash, block *types.Block) {
// into node seamlessly. // into node seamlessly.
func (bc *BlockChain) empty() bool { func (bc *BlockChain) empty() bool {
genesis := bc.genesisBlock.Hash() genesis := bc.genesisBlock.Hash()
for _, hash := range []common.Hash{rawdb.ReadHeadBlockHash(bc.db.BlockStore()), rawdb.ReadHeadHeaderHash(bc.db.BlockStore()), rawdb.ReadHeadFastBlockHash(bc.db)} { for _, hash := range []common.Hash{rawdb.ReadHeadBlockHash(bc.db.BlockStore()), rawdb.ReadHeadHeaderHash(bc.db.BlockStore()), rawdb.ReadHeadFastBlockHash(bc.db.BlockStore())} {
if hash != genesis { if hash != genesis {
return false return false
} }
@@ -736,7 +739,7 @@ func (bc *BlockChain) loadLastState() error {
bc.currentSnapBlock.Store(headBlock.Header()) bc.currentSnapBlock.Store(headBlock.Header())
headFastBlockGauge.Update(int64(headBlock.NumberU64())) headFastBlockGauge.Update(int64(headBlock.NumberU64()))
if head := rawdb.ReadHeadFastBlockHash(bc.db); head != (common.Hash{}) { if head := rawdb.ReadHeadFastBlockHash(bc.db.BlockStore()); head != (common.Hash{}) {
if block := bc.GetBlockByHash(head); block != nil { if block := bc.GetBlockByHash(head); block != nil {
bc.currentSnapBlock.Store(block.Header()) bc.currentSnapBlock.Store(block.Header())
headFastBlockGauge.Update(int64(block.NumberU64())) headFastBlockGauge.Update(int64(block.NumberU64()))
@@ -1135,7 +1138,7 @@ func (bc *BlockChain) setHeadBeyondRoot(head uint64, time uint64, root common.Ha
// If SetHead was only called as a chain reparation method, try to skip // If SetHead was only called as a chain reparation method, try to skip
// touching the header chain altogether, unless the freezer is broken // touching the header chain altogether, unless the freezer is broken
if repair { if repair {
if target, force := updateFn(bc.db, bc.CurrentBlock()); force { if target, force := updateFn(bc.db.BlockStore(), bc.CurrentBlock()); force {
bc.hc.SetHead(target.Number.Uint64(), updateFn, delFn) bc.hc.SetHead(target.Number.Uint64(), updateFn, delFn)
} }
} else { } else {
@@ -1296,19 +1299,33 @@ func (bc *BlockChain) ExportN(w io.Writer, first uint64, last uint64) error {
// //
// Note, this function assumes that the `mu` mutex is held! // Note, this function assumes that the `mu` mutex is held!
func (bc *BlockChain) writeHeadBlock(block *types.Block) { func (bc *BlockChain) writeHeadBlock(block *types.Block) {
// Add the block to the canonical chain number scheme and mark as the head bc.dbWg.Add(2)
rawdb.WriteCanonicalHash(bc.db.BlockStore(), block.Hash(), block.NumberU64()) defer bc.dbWg.Wait()
rawdb.WriteHeadHeaderHash(bc.db.BlockStore(), block.Hash()) go func() {
rawdb.WriteHeadBlockHash(bc.db.BlockStore(), block.Hash()) defer bc.dbWg.Done()
// Add the block to the canonical chain number scheme and mark as the head
blockBatch := bc.db.BlockStore().NewBatch()
rawdb.WriteCanonicalHash(blockBatch, block.Hash(), block.NumberU64())
rawdb.WriteHeadHeaderHash(blockBatch, block.Hash())
rawdb.WriteHeadBlockHash(blockBatch, block.Hash())
rawdb.WriteHeadFastBlockHash(blockBatch, block.Hash())
// Flush the whole batch into the disk, exit the node if failed
if err := blockBatch.Write(); err != nil {
log.Crit("Failed to update chain indexes and markers in block db", "err", err)
}
}()
go func() {
defer bc.dbWg.Done()
batch := bc.db.NewBatch() batch := bc.db.NewBatch()
rawdb.WriteHeadFastBlockHash(batch, block.Hash()) rawdb.WriteTxLookupEntriesByBlock(batch, block)
rawdb.WriteTxLookupEntriesByBlock(batch, block)
// Flush the whole batch into the disk, exit the node if failed
if err := batch.Write(); err != nil {
log.Crit("Failed to update chain indexes in chain db", "err", err)
}
}()
// Flush the whole batch into the disk, exit the node if failed
if err := batch.Write(); err != nil {
log.Crit("Failed to update chain indexes and markers", "err", err)
}
// Update all in-memory chain markers in the last step // Update all in-memory chain markers in the last step
bc.hc.SetCurrentHeader(block.Header()) bc.hc.SetCurrentHeader(block.Header())
@@ -1529,7 +1546,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
} else if !reorg { } else if !reorg {
return false return false
} }
rawdb.WriteHeadFastBlockHash(bc.db, head.Hash()) rawdb.WriteHeadFastBlockHash(bc.db.BlockStore(), head.Hash())
bc.currentSnapBlock.Store(head.Header()) bc.currentSnapBlock.Store(head.Header())
headFastBlockGauge.Update(int64(head.NumberU64())) headFastBlockGauge.Update(int64(head.NumberU64()))
return true return true
@@ -1772,7 +1789,6 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
wg := sync.WaitGroup{} wg := sync.WaitGroup{}
wg.Add(1) wg.Add(1)
go func() { go func() {
rawdb.WritePreimages(bc.db, state.Preimages())
blockBatch := bc.db.BlockStore().NewBatch() blockBatch := bc.db.BlockStore().NewBatch()
rawdb.WriteTd(blockBatch, block.Hash(), block.NumberU64(), externTd) rawdb.WriteTd(blockBatch, block.Hash(), block.NumberU64(), externTd)
rawdb.WriteBlock(blockBatch, block) rawdb.WriteBlock(blockBatch, block)
@@ -1781,7 +1797,11 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
if bc.chainConfig.IsCancun(block.Number(), block.Time()) { if bc.chainConfig.IsCancun(block.Number(), block.Time()) {
rawdb.WriteBlobSidecars(blockBatch, block.Hash(), block.NumberU64(), block.Sidecars()) rawdb.WriteBlobSidecars(blockBatch, block.Hash(), block.NumberU64(), block.Sidecars())
} }
rawdb.WritePreimages(blockBatch, state.Preimages()) if bc.db.StateStore() != nil {
rawdb.WritePreimages(bc.db.StateStore(), state.Preimages())
} else {
rawdb.WritePreimages(blockBatch, state.Preimages())
}
if err := blockBatch.Write(); err != nil { if err := blockBatch.Write(); err != nil {
log.Crit("Failed to write block into disk", "err", err) log.Crit("Failed to write block into disk", "err", err)
} }

View File

@@ -58,11 +58,13 @@ func (st *insertStats) report(chain []*types.Block, index int, snapDiffItems, sn
end := chain[index] end := chain[index]
// Assemble the log context and send it to the logger // Assemble the log context and send it to the logger
mgasps := float64(st.usedGas) * 1000 / float64(elapsed)
context := []interface{}{ context := []interface{}{
"number", end.Number(), "hash", end.Hash(), "miner", end.Coinbase(), "number", end.Number(), "hash", end.Hash(), "miner", end.Coinbase(),
"blocks", st.processed, "txs", txs, "blobs", blobs, "mgas", float64(st.usedGas) / 1000000, "blocks", st.processed, "txs", txs, "blobs", blobs, "mgas", float64(st.usedGas) / 1000000,
"elapsed", common.PrettyDuration(elapsed), "mgasps", float64(st.usedGas) * 1000 / float64(elapsed), "elapsed", common.PrettyDuration(elapsed), "mgasps", mgasps,
} }
blockInsertMgaspsGauge.Update(int64(mgasps))
if timestamp := time.Unix(int64(end.Time()), 0); time.Since(timestamp) > time.Minute { if timestamp := time.Unix(int64(end.Time()), 0); time.Since(timestamp) > time.Minute {
context = append(context, []interface{}{"age", common.PrettyAge(timestamp)}...) context = append(context, []interface{}{"age", common.PrettyAge(timestamp)}...)
} }

View File

@@ -511,3 +511,12 @@ func (bc *BlockChain) SubscribeBlockProcessingEvent(ch chan<- bool) event.Subscr
func (bc *BlockChain) SubscribeFinalizedHeaderEvent(ch chan<- FinalizedHeaderEvent) event.Subscription { func (bc *BlockChain) SubscribeFinalizedHeaderEvent(ch chan<- FinalizedHeaderEvent) event.Subscription {
return bc.scope.Track(bc.finalizedHeaderFeed.Subscribe(ch)) return bc.scope.Track(bc.finalizedHeaderFeed.Subscribe(ch))
} }
// AncientTail retrieves the tail the ancients blocks
func (bc *BlockChain) AncientTail() (uint64, error) {
tail, err := bc.db.Tail()
if err != nil {
return 0, err
}
return tail, nil
}

View File

@@ -1832,14 +1832,10 @@ func testRepairWithScheme(t *testing.T, tt *rewindTest, snapshots bool, scheme s
} }
// Force run a freeze cycle // Force run a freeze cycle
type freezer interface { type freezer interface {
Freeze() error Freeze(threshold uint64) error
Ancients() (uint64, error) Ancients() (uint64, error)
} }
if tt.freezeThreshold < uint64(tt.canonicalBlocks) { db.(freezer).Freeze(tt.freezeThreshold)
final := uint64(tt.canonicalBlocks) - tt.freezeThreshold
chain.SetFinalized(canonblocks[int(final)-1].Header())
}
db.(freezer).Freeze()
// Set the simulated pivot block // Set the simulated pivot block
if tt.pivotBlock != nil { if tt.pivotBlock != nil {

View File

@@ -2045,14 +2045,10 @@ func testSetHeadWithScheme(t *testing.T, tt *rewindTest, snapshots bool, scheme
// Force run a freeze cycle // Force run a freeze cycle
type freezer interface { type freezer interface {
Freeze() error Freeze(threshold uint64) error
Ancients() (uint64, error) Ancients() (uint64, error)
} }
if tt.freezeThreshold < uint64(tt.canonicalBlocks) { db.(freezer).Freeze(tt.freezeThreshold)
final := uint64(tt.canonicalBlocks) - tt.freezeThreshold
chain.SetFinalized(canonblocks[int(final)-1].Header())
}
db.(freezer).Freeze()
// Set the simulated pivot block // Set the simulated pivot block
if tt.pivotBlock != nil { if tt.pivotBlock != nil {

View File

@@ -974,7 +974,7 @@ func testFastVsFullChains(t *testing.T, scheme string) {
t.Fatalf("failed to insert receipt %d: %v", n, err) t.Fatalf("failed to insert receipt %d: %v", n, err)
} }
// Freezer style fast import the chain. // Freezer style fast import the chain.
ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create temp freezer db: %v", err) t.Fatalf("failed to create temp freezer db: %v", err)
} }
@@ -1069,7 +1069,7 @@ func testLightVsFastVsFullChainHeads(t *testing.T, scheme string) {
// makeDb creates a db instance for testing. // makeDb creates a db instance for testing.
makeDb := func() ethdb.Database { makeDb := func() ethdb.Database {
db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create temp freezer db: %v", err) t.Fatalf("failed to create temp freezer db: %v", err)
} }
@@ -1957,7 +1957,7 @@ func testLargeReorgTrieGC(t *testing.T, scheme string) {
competitor, _ := GenerateChain(genesis.Config, shared[len(shared)-1], engine, genDb, 2*TriesInMemory+1, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{3}) }) competitor, _ := GenerateChain(genesis.Config, shared[len(shared)-1], engine, genDb, 2*TriesInMemory+1, func(i int, b *BlockGen) { b.SetCoinbase(common.Address{3}) })
// Import the shared chain and the original canonical one // Import the shared chain and the original canonical one
db, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) db, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
defer db.Close() defer db.Close()
chain, err := NewBlockChain(db, DefaultCacheConfigWithScheme(scheme), genesis, nil, engine, vm.Config{}, nil, nil) chain, err := NewBlockChain(db, DefaultCacheConfigWithScheme(scheme), genesis, nil, engine, vm.Config{}, nil, nil)
@@ -2026,7 +2026,7 @@ func testBlockchainRecovery(t *testing.T, scheme string) {
_, blocks, receipts := GenerateChainWithGenesis(gspec, ethash.NewFaker(), int(height), nil) _, blocks, receipts := GenerateChainWithGenesis(gspec, ethash.NewFaker(), int(height), nil)
// Import the chain as a ancient-first node and ensure all pointers are updated // Import the chain as a ancient-first node and ensure all pointers are updated
ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create temp freezer db: %v", err) t.Fatalf("failed to create temp freezer db: %v", err)
} }
@@ -2097,7 +2097,7 @@ func testInsertReceiptChainRollback(t *testing.T, scheme string) {
} }
// Set up a BlockChain that uses the ancient store. // Set up a BlockChain that uses the ancient store.
ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) ancientDb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create temp freezer db: %v", err) t.Fatalf("failed to create temp freezer db: %v", err)
} }
@@ -2167,7 +2167,7 @@ func testLowDiffLongChain(t *testing.T, scheme string) {
}) })
// Import the canonical chain // Import the canonical chain
diskdb, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) diskdb, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
defer diskdb.Close() defer diskdb.Close()
chain, err := NewBlockChain(diskdb, DefaultCacheConfigWithScheme(scheme), genesis, nil, engine, vm.Config{}, nil, nil) chain, err := NewBlockChain(diskdb, DefaultCacheConfigWithScheme(scheme), genesis, nil, engine, vm.Config{}, nil, nil)
@@ -2384,7 +2384,7 @@ func testInsertKnownChainData(t *testing.T, typ string, scheme string) {
b.OffsetTime(-9) // A higher difficulty b.OffsetTime(-9) // A higher difficulty
}) })
// Import the shared chain and the original canonical one // Import the shared chain and the original canonical one
chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create temp freezer db: %v", err) t.Fatalf("failed to create temp freezer db: %v", err)
} }
@@ -2555,7 +2555,7 @@ func testInsertKnownChainDataWithMerging(t *testing.T, typ string, mergeHeight i
} }
}) })
// Import the shared chain and the original canonical one // Import the shared chain and the original canonical one
chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) chaindb, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create temp freezer db: %v", err) t.Fatalf("failed to create temp freezer db: %v", err)
} }
@@ -3858,7 +3858,7 @@ func testSetCanonical(t *testing.T, scheme string) {
} }
gen.AddTx(tx) gen.AddTx(tx)
}) })
diskdb, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false) diskdb, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), t.TempDir(), "", false, false, false, false, false)
defer diskdb.Close() defer diskdb.Close()
chain, err := NewBlockChain(diskdb, DefaultCacheConfigWithScheme(scheme), gspec, nil, engine, vm.Config{}, nil, nil) chain, err := NewBlockChain(diskdb, DefaultCacheConfigWithScheme(scheme), gspec, nil, engine, vm.Config{}, nil, nil)
@@ -4483,7 +4483,7 @@ func (c *mockParlia) CalcDifficulty(chain consensus.ChainHeaderReader, time uint
func TestParliaBlobFeeReward(t *testing.T) { func TestParliaBlobFeeReward(t *testing.T) {
// Have N headers in the freezer // Have N headers in the freezer
frdir := t.TempDir() frdir := t.TempDir()
db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create database with ancient backend") t.Fatalf("failed to create database with ancient backend")
} }

View File

@@ -114,7 +114,9 @@ func (f *ForkChoice) ReorgNeeded(current *types.Header, extern *types.Header) (b
if f.preserve != nil { if f.preserve != nil {
currentPreserve, externPreserve = f.preserve(current), f.preserve(extern) currentPreserve, externPreserve = f.preserve(current), f.preserve(extern)
} }
reorg = !currentPreserve && (externPreserve || f.rand.Float64() < 0.5) reorg = !currentPreserve && (externPreserve ||
extern.Time < current.Time ||
extern.Time == current.Time && f.rand.Float64() < 0.5)
} }
return reorg, nil return reorg, nil
} }

View File

@@ -216,10 +216,10 @@ func (e *GenesisMismatchError) Error() string {
// ChainOverrides contains the changes to chain config // ChainOverrides contains the changes to chain config
// Typically, these modifications involve hardforks that are not enabled on the BSC mainnet, intended for testing purposes. // Typically, these modifications involve hardforks that are not enabled on the BSC mainnet, intended for testing purposes.
type ChainOverrides struct { type ChainOverrides struct {
OverrideCancun *uint64 OverrideCancun *uint64
OverrideVerkle *uint64 OverrideHaber *uint64
OverrideFeynman *uint64 OverrideBohr *uint64
OverrideFeynmanFix *uint64 OverrideVerkle *uint64
} }
// SetupGenesisBlock writes or updates the genesis block in db. // SetupGenesisBlock writes or updates the genesis block in db.
@@ -248,15 +248,15 @@ func SetupGenesisBlockWithOverride(db ethdb.Database, triedb *triedb.Database, g
if overrides != nil && overrides.OverrideCancun != nil { if overrides != nil && overrides.OverrideCancun != nil {
config.CancunTime = overrides.OverrideCancun config.CancunTime = overrides.OverrideCancun
} }
if overrides != nil && overrides.OverrideHaber != nil {
config.HaberTime = overrides.OverrideHaber
}
if overrides != nil && overrides.OverrideBohr != nil {
config.BohrTime = overrides.OverrideBohr
}
if overrides != nil && overrides.OverrideVerkle != nil { if overrides != nil && overrides.OverrideVerkle != nil {
config.VerkleTime = overrides.OverrideVerkle config.VerkleTime = overrides.OverrideVerkle
} }
if overrides != nil && overrides.OverrideFeynman != nil {
config.FeynmanTime = overrides.OverrideFeynman
}
if overrides != nil && overrides.OverrideFeynmanFix != nil {
config.FeynmanFixTime = overrides.OverrideFeynmanFix
}
} }
} }
// Just commit the new block if there is no stored genesis block. // Just commit the new block if there is no stored genesis block.
@@ -498,7 +498,7 @@ func (g *Genesis) Commit(db ethdb.Database, triedb *triedb.Database) (*types.Blo
rawdb.WriteReceipts(db.BlockStore(), block.Hash(), block.NumberU64(), nil) rawdb.WriteReceipts(db.BlockStore(), block.Hash(), block.NumberU64(), nil)
rawdb.WriteCanonicalHash(db.BlockStore(), block.Hash(), block.NumberU64()) rawdb.WriteCanonicalHash(db.BlockStore(), block.Hash(), block.NumberU64())
rawdb.WriteHeadBlockHash(db.BlockStore(), block.Hash()) rawdb.WriteHeadBlockHash(db.BlockStore(), block.Hash())
rawdb.WriteHeadFastBlockHash(db, block.Hash()) rawdb.WriteHeadFastBlockHash(db.BlockStore(), block.Hash())
rawdb.WriteHeadHeaderHash(db.BlockStore(), block.Hash()) rawdb.WriteHeadHeaderHash(db.BlockStore(), block.Hash())
rawdb.WriteChainConfig(db, block.Hash(), config) rawdb.WriteChainConfig(db, block.Hash(), config)
return block, nil return block, nil

View File

@@ -668,7 +668,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
// first then remove the relative data from the database. // first then remove the relative data from the database.
// //
// Update head first(head fast block, head full block) before deleting the data. // Update head first(head fast block, head full block) before deleting the data.
markerBatch := hc.chainDb.NewBatch() markerBatch := hc.chainDb.BlockStore().NewBatch()
if updateFn != nil { if updateFn != nil {
newHead, force := updateFn(markerBatch, parent) newHead, force := updateFn(markerBatch, parent)
if force && ((headTime > 0 && newHead.Time < headTime) || (headTime == 0 && newHead.Number.Uint64() < headBlock)) { if force && ((headTime > 0 && newHead.Time < headTime) || (headTime == 0 && newHead.Number.Uint64() < headBlock)) {
@@ -677,7 +677,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
} }
} }
// Update head header then. // Update head header then.
rawdb.WriteHeadHeaderHash(hc.chainDb.BlockStore(), parentHash) rawdb.WriteHeadHeaderHash(markerBatch, parentHash)
if err := markerBatch.Write(); err != nil { if err := markerBatch.Write(); err != nil {
log.Crit("Failed to update chain markers", "error", err) log.Crit("Failed to update chain markers", "error", err)
} }

View File

@@ -316,8 +316,8 @@ func ReadHeaderRange(db ethdb.Reader, number uint64, count uint64) []rlp.RawValu
if count == 0 { if count == 0 {
return rlpHeaders return rlpHeaders
} }
// read remaining from ancients // read remaining from ancients, cap at 2M
data, err := db.AncientRange(ChainFreezerHeaderTable, i+1-count, count, 0) data, err := db.AncientRange(ChainFreezerHeaderTable, i+1-count, count, 2*1024*1024)
if err != nil { if err != nil {
log.Error("Failed to read headers from freezer", "err", err) log.Error("Failed to read headers from freezer", "err", err)
return rlpHeaders return rlpHeaders

View File

@@ -518,7 +518,7 @@ func checkBlobSidecarsRLP(have, want types.BlobSidecars) error {
func TestAncientStorage(t *testing.T) { func TestAncientStorage(t *testing.T) {
// Freezer style fast import the chain. // Freezer style fast import the chain.
frdir := t.TempDir() frdir := t.TempDir()
db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false) db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create database with ancient backend") t.Fatalf("failed to create database with ancient backend")
} }
@@ -657,7 +657,7 @@ func TestHashesInRange(t *testing.T) {
func BenchmarkWriteAncientBlocks(b *testing.B) { func BenchmarkWriteAncientBlocks(b *testing.B) {
// Open freezer database. // Open freezer database.
frdir := b.TempDir() frdir := b.TempDir()
db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false) db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false, false)
if err != nil { if err != nil {
b.Fatalf("failed to create database with ancient backend") b.Fatalf("failed to create database with ancient backend")
} }
@@ -1001,7 +1001,7 @@ func TestHeadersRLPStorage(t *testing.T) {
// Have N headers in the freezer // Have N headers in the freezer
frdir := t.TempDir() frdir := t.TempDir()
db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false) db, err := NewDatabaseWithFreezer(NewMemoryDatabase(), frdir, "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create database with ancient backend") t.Fatalf("failed to create database with ancient backend")
} }

View File

@@ -24,12 +24,12 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"github.com/ethereum/go-ethereum/rlp"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/ethdb" "github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/log" "github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params" "github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rlp"
) )
const ( const (
@@ -51,25 +51,31 @@ var (
// The background thread will keep moving ancient chain segments from key-value // The background thread will keep moving ancient chain segments from key-value
// database to flat files for saving space on live database. // database to flat files for saving space on live database.
type chainFreezer struct { type chainFreezer struct {
threshold atomic.Uint64 // Number of recent blocks not to freeze (params.FullImmutabilityThreshold apart from tests)
*Freezer *Freezer
quit chan struct{} quit chan struct{}
wg sync.WaitGroup wg sync.WaitGroup
trigger chan chan struct{} // Manual blocking freeze trigger, test determinism trigger chan chan struct{} // Manual blocking freeze trigger, test determinism
freezeEnv atomic.Value freezeEnv atomic.Value
multiDatabase bool
} }
// newChainFreezer initializes the freezer for ancient chain data. // newChainFreezer initializes the freezer for ancient chain data.
func newChainFreezer(datadir string, namespace string, readonly bool, offset uint64) (*chainFreezer, error) { func newChainFreezer(datadir string, namespace string, readonly bool, offset uint64, multiDatabase bool) (*chainFreezer, error) {
freezer, err := NewChainFreezer(datadir, namespace, readonly, offset) freezer, err := NewChainFreezer(datadir, namespace, readonly, offset)
if err != nil { if err != nil {
return nil, err return nil, err
} }
return &chainFreezer{ cf := chainFreezer{
Freezer: freezer, Freezer: freezer,
quit: make(chan struct{}), quit: make(chan struct{}),
trigger: make(chan chan struct{}), trigger: make(chan chan struct{}),
}, nil }
cf.threshold.Store(params.FullImmutabilityThreshold)
return &cf, nil
} }
// Close closes the chain freezer instance and terminates the background thread. // Close closes the chain freezer instance and terminates the background thread.
@@ -185,29 +191,101 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
continue continue
} }
threshold, err := f.freezeThreshold(nfdb) var (
if err != nil { frozen uint64
backoff = true threshold uint64
log.Debug("Current full block not old enough to freeze", "err", err) first uint64 // the first block to freeze
continue last uint64 // the last block to freeze
}
frozen := f.frozen.Load()
// Short circuit if the blocks below threshold are already frozen. hash common.Hash
if frozen != 0 && frozen-1 >= threshold { number *uint64
backoff = true head *types.Header
log.Debug("Ancient blocks frozen already", "threshold", threshold, "frozen", frozen) )
continue
// use finalized block as the chain freeze indicator was used for multiDatabase feature, if multiDatabase is false, keep 9W blocks in db
if f.multiDatabase {
threshold, err = f.freezeThreshold(nfdb)
if err != nil {
backoff = true
log.Debug("Current full block not old enough to freeze", "err", err)
continue
}
frozen = f.frozen.Load()
// Short circuit if the blocks below threshold are already frozen.
if frozen != 0 && frozen-1 >= threshold {
backoff = true
log.Debug("Ancient blocks frozen already", "threshold", threshold, "frozen", frozen)
continue
}
hash = ReadHeadBlockHash(nfdb)
if hash == (common.Hash{}) {
log.Debug("Current full block hash unavailable") // new chain, empty database
backoff = true
continue
}
number = ReadHeaderNumber(nfdb, hash)
if number == nil {
log.Error("Current full block number unavailable", "hash", hash)
backoff = true
continue
}
head = ReadHeader(nfdb, hash, *number)
if head == nil {
log.Error("Current full block unavailable", "number", *number, "hash", hash)
backoff = true
continue
}
first = frozen
last = threshold
if last-first+1 > freezerBatchLimit {
last = freezerBatchLimit + first - 1
}
} else {
// Retrieve the freezing threshold.
hash = ReadHeadBlockHash(nfdb)
if hash == (common.Hash{}) {
log.Debug("Current full block hash unavailable") // new chain, empty database
backoff = true
continue
}
number = ReadHeaderNumber(nfdb, hash)
threshold = f.threshold.Load()
frozen = f.frozen.Load()
switch {
case number == nil:
log.Error("Current full block number unavailable", "hash", hash)
backoff = true
continue
case *number < threshold:
log.Debug("Current full block not old enough to freeze", "number", *number, "hash", hash, "delay", threshold)
backoff = true
continue
case *number-threshold <= frozen:
log.Debug("Ancient blocks frozen already", "number", *number, "hash", hash, "frozen", frozen)
backoff = true
continue
}
head = ReadHeader(nfdb, hash, *number)
if head == nil {
log.Error("Current full block unavailable", "number", *number, "hash", hash)
backoff = true
continue
}
first, _ = f.Ancients()
last = *number - threshold
if last-first > freezerBatchLimit {
last = first + freezerBatchLimit
}
} }
// Seems we have data ready to be frozen, process in usable batches // Seems we have data ready to be frozen, process in usable batches
var ( var (
start = time.Now() start = time.Now()
first = frozen // the first block to freeze
last = threshold // the last block to freeze
) )
if last-first+1 > freezerBatchLimit {
last = freezerBatchLimit + first - 1
}
ancients, err := f.freezeRangeWithBlobs(nfdb, first, last) ancients, err := f.freezeRangeWithBlobs(nfdb, first, last)
if err != nil { if err != nil {
@@ -295,24 +373,6 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
log.Debug("Deep froze chain segment", context...) log.Debug("Deep froze chain segment", context...)
env, _ := f.freezeEnv.Load().(*ethdb.FreezerEnv) env, _ := f.freezeEnv.Load().(*ethdb.FreezerEnv)
hash := ReadHeadBlockHash(nfdb)
if hash == (common.Hash{}) {
log.Debug("Current full block hash unavailable") // new chain, empty database
backoff = true
continue
}
number := ReadHeaderNumber(nfdb, hash)
if number == nil {
log.Error("Current full block number unavailable", "hash", hash)
backoff = true
continue
}
head := ReadHeader(nfdb, hash, *number)
if head == nil {
log.Error("Current full block unavailable", "number", *number, "hash", hash)
backoff = true
continue
}
// try prune blob data after cancun fork // try prune blob data after cancun fork
if isCancun(env, head.Number, head.Time) { if isCancun(env, head.Number, head.Time) {
f.tryPruneBlobAncientTable(env, *number) f.tryPruneBlobAncientTable(env, *number)

View File

@@ -81,7 +81,7 @@ func InitDatabaseFromFreezer(db ethdb.Database) {
batch.Reset() batch.Reset()
WriteHeadHeaderHash(db.BlockStore(), hash) WriteHeadHeaderHash(db.BlockStore(), hash)
WriteHeadFastBlockHash(db, hash) WriteHeadFastBlockHash(db.BlockStore(), hash)
log.Info("Initialized database from freezer", "blocks", frozen, "elapsed", common.PrettyDuration(time.Since(start))) log.Info("Initialized database from freezer", "blocks", frozen, "elapsed", common.PrettyDuration(time.Since(start)))
} }

View File

@@ -61,7 +61,7 @@ func (frdb *freezerdb) BlockStoreReader() ethdb.Reader {
} }
func (frdb *freezerdb) BlockStoreWriter() ethdb.Writer { func (frdb *freezerdb) BlockStoreWriter() ethdb.Writer {
//TODO implement me // TODO implement me
panic("implement me") panic("implement me")
} }
@@ -138,13 +138,22 @@ func (frdb *freezerdb) SetBlockStore(block ethdb.Database) {
frdb.blockStore = block frdb.blockStore = block
} }
func (frdb *freezerdb) HasSeparateBlockStore() bool {
return frdb.blockStore != nil
}
// Freeze is a helper method used for external testing to trigger and block until // Freeze is a helper method used for external testing to trigger and block until
// a freeze cycle completes, without having to sleep for a minute to trigger the // a freeze cycle completes, without having to sleep for a minute to trigger the
// automatic background run. // automatic background run.
func (frdb *freezerdb) Freeze() error { func (frdb *freezerdb) Freeze(threshold uint64) error {
if frdb.AncientStore.(*chainFreezer).readonly { if frdb.AncientStore.(*chainFreezer).readonly {
return errReadOnly return errReadOnly
} }
// Set the freezer threshold to a temporary value
defer func(old uint64) {
frdb.AncientStore.(*chainFreezer).threshold.Store(old)
}(frdb.AncientStore.(*chainFreezer).threshold.Load())
frdb.AncientStore.(*chainFreezer).threshold.Store(threshold)
// Trigger a freeze cycle and block until it's done // Trigger a freeze cycle and block until it's done
trigger := make(chan struct{}, 1) trigger := make(chan struct{}, 1)
frdb.AncientStore.(*chainFreezer).trigger <- trigger frdb.AncientStore.(*chainFreezer).trigger <- trigger
@@ -184,7 +193,7 @@ func (db *nofreezedb) Ancients() (uint64, error) {
return 0, errNotSupported return 0, errNotSupported
} }
// Ancients returns an error as we don't have a backing chain freezer. // ItemAmountInAncient returns an error as we don't have a backing chain freezer.
func (db *nofreezedb) ItemAmountInAncient() (uint64, error) { func (db *nofreezedb) ItemAmountInAncient() (uint64, error) {
return 0, errNotSupported return 0, errNotSupported
} }
@@ -263,6 +272,10 @@ func (db *nofreezedb) SetBlockStore(block ethdb.Database) {
db.blockStore = block db.blockStore = block
} }
func (db *nofreezedb) HasSeparateBlockStore() bool {
return db.blockStore != nil
}
func (db *nofreezedb) BlockStoreReader() ethdb.Reader { func (db *nofreezedb) BlockStoreReader() ethdb.Reader {
if db.blockStore != nil { if db.blockStore != nil {
return db.blockStore return db.blockStore
@@ -318,6 +331,110 @@ func NewDatabase(db ethdb.KeyValueStore) ethdb.Database {
return &nofreezedb{KeyValueStore: db} return &nofreezedb{KeyValueStore: db}
} }
type emptyfreezedb struct {
ethdb.KeyValueStore
}
// HasAncient returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) HasAncient(kind string, number uint64) (bool, error) {
return false, nil
}
// Ancient returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) Ancient(kind string, number uint64) ([]byte, error) {
return nil, nil
}
// AncientRange returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) AncientRange(kind string, start, max, maxByteSize uint64) ([][]byte, error) {
return nil, nil
}
// Ancients returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) Ancients() (uint64, error) {
return 0, nil
}
// ItemAmountInAncient returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) ItemAmountInAncient() (uint64, error) {
return 0, nil
}
// Tail returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) Tail() (uint64, error) {
return 0, nil
}
// AncientSize returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) AncientSize(kind string) (uint64, error) {
return 0, nil
}
// ModifyAncients returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) ModifyAncients(func(ethdb.AncientWriteOp) error) (int64, error) {
return 0, nil
}
// TruncateHead returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) TruncateHead(items uint64) (uint64, error) {
return 0, nil
}
// TruncateTail returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) TruncateTail(items uint64) (uint64, error) {
return 0, nil
}
// TruncateTableTail returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) TruncateTableTail(kind string, tail uint64) (uint64, error) {
return 0, nil
}
// ResetTable returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) ResetTable(kind string, startAt uint64, onlyEmpty bool) error {
return nil
}
// Sync returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) Sync() error {
return nil
}
func (db *emptyfreezedb) DiffStore() ethdb.KeyValueStore { return db }
func (db *emptyfreezedb) SetDiffStore(diff ethdb.KeyValueStore) {}
func (db *emptyfreezedb) StateStore() ethdb.Database { return db }
func (db *emptyfreezedb) SetStateStore(state ethdb.Database) {}
func (db *emptyfreezedb) StateStoreReader() ethdb.Reader { return db }
func (db *emptyfreezedb) BlockStore() ethdb.Database { return db }
func (db *emptyfreezedb) SetBlockStore(block ethdb.Database) {}
func (db *emptyfreezedb) HasSeparateBlockStore() bool { return false }
func (db *emptyfreezedb) BlockStoreReader() ethdb.Reader { return db }
func (db *emptyfreezedb) BlockStoreWriter() ethdb.Writer { return db }
func (db *emptyfreezedb) ReadAncients(fn func(reader ethdb.AncientReaderOp) error) (err error) {
return nil
}
func (db *emptyfreezedb) AncientOffSet() uint64 { return 0 }
// MigrateTable returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) MigrateTable(kind string, convert convertLegacyFn) error {
return nil
}
// AncientDatadir returns nil for pruned db that we don't have a backing chain freezer.
func (db *emptyfreezedb) AncientDatadir() (string, error) {
return "", nil
}
func (db *emptyfreezedb) SetupFreezerEnv(env *ethdb.FreezerEnv) error {
return nil
}
// NewEmptyFreezeDB is used for CLI such as `geth db inspect` in pruned db that we don't
// have a backing chain freezer.
// WARNING: it must be only used in the above case.
func NewEmptyFreezeDB(db ethdb.KeyValueStore) ethdb.Database {
return &emptyfreezedb{KeyValueStore: db}
}
// NewFreezerDb only create a freezer without statedb. // NewFreezerDb only create a freezer without statedb.
func NewFreezerDb(db ethdb.KeyValueStore, frz, namespace string, readonly bool, newOffSet uint64) (*Freezer, error) { func NewFreezerDb(db ethdb.KeyValueStore, frz, namespace string, readonly bool, newOffSet uint64) (*Freezer, error) {
// Create the idle freezer instance, this operation should be atomic to avoid mismatch between offset and acientDB. // Create the idle freezer instance, this operation should be atomic to avoid mismatch between offset and acientDB.
@@ -358,7 +475,7 @@ func resolveChainFreezerDir(ancient string) string {
// value data store with a freezer moving immutable chain segments into cold // value data store with a freezer moving immutable chain segments into cold
// storage. The passed ancient indicates the path of root ancient directory // storage. The passed ancient indicates the path of root ancient directory
// where the chain freezer can be opened. // where the chain freezer can be opened.
func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace string, readonly, disableFreeze, isLastOffset, pruneAncientData bool) (ethdb.Database, error) { func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace string, readonly, disableFreeze, isLastOffset, pruneAncientData, multiDatabase bool) (ethdb.Database, error) {
var offset uint64 var offset uint64
// The offset of ancientDB should be handled differently in different scenarios. // The offset of ancientDB should be handled differently in different scenarios.
if isLastOffset { if isLastOffset {
@@ -367,6 +484,12 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
offset = ReadOffSetOfCurrentAncientFreezer(db) offset = ReadOffSetOfCurrentAncientFreezer(db)
} }
// This case is used for someone who wants to execute geth db inspect CLI in a pruned db
if !disableFreeze && readonly && ReadAncientType(db) == PruneFreezerType {
log.Warn("Disk db is pruned, using an empty freezer db for CLI")
return NewEmptyFreezeDB(db), nil
}
if pruneAncientData && !disableFreeze && !readonly { if pruneAncientData && !disableFreeze && !readonly {
frdb, err := newPrunedFreezer(resolveChainFreezerDir(ancient), db, offset) frdb, err := newPrunedFreezer(resolveChainFreezerDir(ancient), db, offset)
if err != nil { if err != nil {
@@ -394,7 +517,7 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
} }
// Create the idle freezer instance // Create the idle freezer instance
frdb, err := newChainFreezer(resolveChainFreezerDir(ancient), namespace, readonly, offset) frdb, err := newChainFreezer(resolveChainFreezerDir(ancient), namespace, readonly, offset, multiDatabase)
if err != nil { if err != nil {
printChainMetadata(db) printChainMetadata(db)
return nil, err return nil, err
@@ -575,6 +698,8 @@ type OpenOptions struct {
// Ephemeral means that filesystem sync operations should be avoided: data integrity in the face of // Ephemeral means that filesystem sync operations should be avoided: data integrity in the face of
// a crash is not important. This option should typically be used in tests. // a crash is not important. This option should typically be used in tests.
Ephemeral bool Ephemeral bool
MultiDataBase bool
} }
// openKeyValueDatabase opens a disk-based key-value database, e.g. leveldb or pebble. // openKeyValueDatabase opens a disk-based key-value database, e.g. leveldb or pebble.
@@ -619,13 +744,13 @@ func Open(o OpenOptions) (ethdb.Database, error) {
} }
if ReadAncientType(kvdb) == PruneFreezerType { if ReadAncientType(kvdb) == PruneFreezerType {
if !o.PruneAncientData { if !o.PruneAncientData {
log.Warn("Disk db is pruned") log.Warn("NOTICE: You're opening a pruned disk db!")
} }
} }
if len(o.AncientsDirectory) == 0 { if len(o.AncientsDirectory) == 0 {
return kvdb, nil return kvdb, nil
} }
frdb, err := NewDatabaseWithFreezer(kvdb, o.AncientsDirectory, o.Namespace, o.ReadOnly, o.DisableFreeze, o.IsLastOffset, o.PruneAncientData) frdb, err := NewDatabaseWithFreezer(kvdb, o.AncientsDirectory, o.Namespace, o.ReadOnly, o.DisableFreeze, o.IsLastOffset, o.PruneAncientData, o.MultiDataBase)
if err != nil { if err != nil {
kvdb.Close() kvdb.Close()
return nil, err return nil, err
@@ -748,7 +873,7 @@ func DataTypeByKey(key []byte) DataType {
return StateDataType return StateDataType
} }
} }
for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey} { for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey, headBlockKey, headFastBlockKey} {
if bytes.Equal(key, meta) { if bytes.Equal(key, meta) {
return BlockDataType return BlockDataType
} }
@@ -769,7 +894,7 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
trieIter = db.StateStore().NewIterator(keyPrefix, nil) trieIter = db.StateStore().NewIterator(keyPrefix, nil)
defer trieIter.Release() defer trieIter.Release()
} }
if db.BlockStore() != db { if db.HasSeparateBlockStore() {
blockIter = db.BlockStore().NewIterator(keyPrefix, nil) blockIter = db.BlockStore().NewIterator(keyPrefix, nil)
defer blockIter.Release() defer blockIter.Release()
} }
@@ -963,7 +1088,7 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
hashNumPairings.Add(size) hashNumPairings.Add(size)
default: default:
var accounted bool var accounted bool
for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey} { for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey, headBlockKey, headFastBlockKey} {
if bytes.Equal(key, meta) { if bytes.Equal(key, meta) {
metadata.Add(size) metadata.Add(size)
accounted = true accounted = true
@@ -1157,7 +1282,7 @@ func ReadChainMetadataFromMultiDatabase(db ethdb.Database) [][]string {
data := [][]string{ data := [][]string{
{"databaseVersion", pp(ReadDatabaseVersion(db))}, {"databaseVersion", pp(ReadDatabaseVersion(db))},
{"headBlockHash", fmt.Sprintf("%v", ReadHeadBlockHash(db.BlockStore()))}, {"headBlockHash", fmt.Sprintf("%v", ReadHeadBlockHash(db.BlockStore()))},
{"headFastBlockHash", fmt.Sprintf("%v", ReadHeadFastBlockHash(db))}, {"headFastBlockHash", fmt.Sprintf("%v", ReadHeadFastBlockHash(db.BlockStore()))},
{"headHeaderHash", fmt.Sprintf("%v", ReadHeadHeaderHash(db.BlockStore()))}, {"headHeaderHash", fmt.Sprintf("%v", ReadHeadHeaderHash(db.BlockStore()))},
{"lastPivotNumber", pp(ReadLastPivotNumber(db))}, {"lastPivotNumber", pp(ReadLastPivotNumber(db))},
{"len(snapshotSyncStatus)", fmt.Sprintf("%d bytes", len(ReadSnapshotSyncStatus(db)))}, {"len(snapshotSyncStatus)", fmt.Sprintf("%d bytes", len(ReadSnapshotSyncStatus(db)))},

View File

@@ -239,7 +239,7 @@ func (f *Freezer) Ancient(kind string, number uint64) ([]byte, error) {
// - if maxBytes is not specified, 'count' items will be returned if they are present. // - if maxBytes is not specified, 'count' items will be returned if they are present.
func (f *Freezer) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error) { func (f *Freezer) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error) {
if table := f.tables[kind]; table != nil { if table := f.tables[kind]; table != nil {
return table.RetrieveItems(start, count, maxBytes) return table.RetrieveItems(start-f.offset, count, maxBytes)
} }
return nil, errUnknownTable return nil, errUnknownTable
} }
@@ -252,7 +252,7 @@ func (f *Freezer) Ancients() (uint64, error) {
func (f *Freezer) TableAncients(kind string) (uint64, error) { func (f *Freezer) TableAncients(kind string) (uint64, error) {
f.writeLock.RLock() f.writeLock.RLock()
defer f.writeLock.RUnlock() defer f.writeLock.RUnlock()
return f.tables[kind].items.Load(), nil return f.tables[kind].items.Load() + f.offset, nil
} }
// ItemAmountInAncient returns the actual length of current ancientDB. // ItemAmountInAncient returns the actual length of current ancientDB.

View File

@@ -43,6 +43,10 @@ func (t *table) SetBlockStore(block ethdb.Database) {
panic("not implement") panic("not implement")
} }
func (t *table) HasSeparateBlockStore() bool {
panic("not implement")
}
// NewTable returns a database object that prefixes all keys with a given string. // NewTable returns a database object that prefixes all keys with a given string.
func NewTable(db ethdb.Database, prefix string) ethdb.Database { func NewTable(db ethdb.Database, prefix string) ethdb.Database {
return &table{ return &table{

View File

@@ -382,7 +382,7 @@ func (p *BlockPruner) backUpOldDb(name string, cache, handles int, namespace str
log.Info("chainDB opened successfully") log.Info("chainDB opened successfully")
// Get the number of items in old ancient db. // Get the number of items in old ancient db.
itemsOfAncient, err := chainDb.ItemAmountInAncient() itemsOfAncient, err := chainDb.BlockStore().ItemAmountInAncient()
log.Info("the number of items in ancientDB is ", "itemsOfAncient", itemsOfAncient) log.Info("the number of items in ancientDB is ", "itemsOfAncient", itemsOfAncient)
// If we can't access the freezer or it's empty, abort. // If we can't access the freezer or it's empty, abort.
@@ -402,7 +402,7 @@ func (p *BlockPruner) backUpOldDb(name string, cache, handles int, namespace str
var oldOffSet uint64 var oldOffSet uint64
if interrupt { if interrupt {
// The interrupt scecario within this function is specific for old and new ancientDB exsisted concurrently, // The interrupt scecario within this function is specific for old and new ancientDB existed concurrently,
// should use last version of offset for oldAncientDB, because current offset is // should use last version of offset for oldAncientDB, because current offset is
// actually of the new ancientDB_Backup, but what we want is the offset of ancientDB being backup. // actually of the new ancientDB_Backup, but what we want is the offset of ancientDB being backup.
oldOffSet = rawdb.ReadOffSetOfLastAncientFreezer(chainDb) oldOffSet = rawdb.ReadOffSetOfLastAncientFreezer(chainDb)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -57,31 +57,3 @@ var (
//go:embed chapel/TokenRecoverPortalContract //go:embed chapel/TokenRecoverPortalContract
ChapelTokenRecoverPortalContract string ChapelTokenRecoverPortalContract string
) )
// contract codes for Rialto upgrade
var (
//go:embed rialto/ValidatorContract
RialtoValidatorContract string
//go:embed rialto/SlashContract
RialtoSlashContract string
//go:embed rialto/TokenHubContract
RialtoTokenHubContract string
//go:embed rialto/GovHubContract
RialtoGovHubContract string
//go:embed rialto/CrossChainContract
RialtoCrossChainContract string
//go:embed rialto/StakingContract
RialtoStakingContract string
//go:embed rialto/StakeHubContract
RialtoStakeHubContract string
//go:embed rialto/StakeCreditContract
RialtoStakeCreditContract string
//go:embed rialto/GovernorContract
RialtoGovernorContract string
//go:embed rialto/GovTokenContract
RialtoGovTokenContract string
//go:embed rialto/TimelockContract
RialtoTimelockContract string
//go:embed rialto/TokenRecoverPortalContract
RialtoTokenRecoverPortalContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,11 @@
package haber_fix
import _ "embed"
// contract codes for Chapel upgrade
var (
//go:embed chapel/ValidatorContract
ChapelValidatorContract string
//go:embed chapel/SlashContract
ChapelSlashContract string
)

View File

@@ -7,14 +7,12 @@ import (
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/state" "github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/core/systemcontracts/bruno" "github.com/ethereum/go-ethereum/core/systemcontracts/bruno"
"github.com/ethereum/go-ethereum/core/systemcontracts/euler" "github.com/ethereum/go-ethereum/core/systemcontracts/euler"
"github.com/ethereum/go-ethereum/core/systemcontracts/feynman" "github.com/ethereum/go-ethereum/core/systemcontracts/feynman"
feynmanFix "github.com/ethereum/go-ethereum/core/systemcontracts/feynman_fix" feynmanFix "github.com/ethereum/go-ethereum/core/systemcontracts/feynman_fix"
"github.com/ethereum/go-ethereum/core/systemcontracts/gibbs" "github.com/ethereum/go-ethereum/core/systemcontracts/gibbs"
haberFix "github.com/ethereum/go-ethereum/core/systemcontracts/haber_fix"
"github.com/ethereum/go-ethereum/core/systemcontracts/kepler" "github.com/ethereum/go-ethereum/core/systemcontracts/kepler"
"github.com/ethereum/go-ethereum/core/systemcontracts/luban" "github.com/ethereum/go-ethereum/core/systemcontracts/luban"
"github.com/ethereum/go-ethereum/core/systemcontracts/mirror" "github.com/ethereum/go-ethereum/core/systemcontracts/mirror"
@@ -23,6 +21,8 @@ import (
"github.com/ethereum/go-ethereum/core/systemcontracts/planck" "github.com/ethereum/go-ethereum/core/systemcontracts/planck"
"github.com/ethereum/go-ethereum/core/systemcontracts/plato" "github.com/ethereum/go-ethereum/core/systemcontracts/plato"
"github.com/ethereum/go-ethereum/core/systemcontracts/ramanujan" "github.com/ethereum/go-ethereum/core/systemcontracts/ramanujan"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
) )
type UpgradeConfig struct { type UpgradeConfig struct {
@@ -75,6 +75,8 @@ var (
feynmanUpgrade = make(map[string]*Upgrade) feynmanUpgrade = make(map[string]*Upgrade)
feynmanFixUpgrade = make(map[string]*Upgrade) feynmanFixUpgrade = make(map[string]*Upgrade)
haberFixUpgrade = make(map[string]*Upgrade)
) )
func init() { func init() {
@@ -680,72 +682,6 @@ func init() {
}, },
} }
feynmanUpgrade[rialtoNet] = &Upgrade{
UpgradeName: "feynman",
Configs: []*UpgradeConfig{
{
ContractAddr: common.HexToAddress(ValidatorContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoValidatorContract,
},
{
ContractAddr: common.HexToAddress(SlashContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoSlashContract,
},
{
ContractAddr: common.HexToAddress(TokenHubContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoTokenHubContract,
},
{
ContractAddr: common.HexToAddress(GovHubContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoGovHubContract,
},
{
ContractAddr: common.HexToAddress(CrossChainContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoCrossChainContract,
},
{
ContractAddr: common.HexToAddress(StakingContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoStakingContract,
},
{
ContractAddr: common.HexToAddress(StakeHubContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoStakeHubContract,
},
{
ContractAddr: common.HexToAddress(StakeCreditContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.MainnetStakeCreditContract,
},
{
ContractAddr: common.HexToAddress(GovernorContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoGovernorContract,
},
{
ContractAddr: common.HexToAddress(GovTokenContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.MainnetGovTokenContract,
},
{
ContractAddr: common.HexToAddress(TimelockContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoTimelockContract,
},
{
ContractAddr: common.HexToAddress(TokenRecoverPortalContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: feynman.RialtoTokenRecoverPortalContract,
},
},
}
// This upgrade is to fix an error on testnet only. So the upgrade config of mainnet is empty. // This upgrade is to fix an error on testnet only. So the upgrade config of mainnet is empty.
feynmanFixUpgrade[mainNet] = &Upgrade{ feynmanFixUpgrade[mainNet] = &Upgrade{
UpgradeName: "feynmanFix", UpgradeName: "feynmanFix",
@@ -768,9 +704,20 @@ func init() {
}, },
} }
feynmanFixUpgrade[rialtoNet] = &Upgrade{ haberFixUpgrade[chapelNet] = &Upgrade{
UpgradeName: "feynmanFix", UpgradeName: "haberFix",
Configs: []*UpgradeConfig{}, Configs: []*UpgradeConfig{
{
ContractAddr: common.HexToAddress(ValidatorContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: haberFix.ChapelValidatorContract,
},
{
ContractAddr: common.HexToAddress(SlashContract),
CommitUrl: "https://github.com/bnb-chain/bsc-genesis-contract/commit/2d6372ddba77902ef01e45887a425938376d5a5c",
Code: haberFix.ChapelSlashContract,
},
},
} }
} }
@@ -848,6 +795,10 @@ func UpgradeBuildInSystemContract(config *params.ChainConfig, blockNumber *big.I
applySystemContractUpgrade(feynmanFixUpgrade[network], blockNumber, statedb, logger) applySystemContractUpgrade(feynmanFixUpgrade[network], blockNumber, statedb, logger)
} }
if config.IsOnHaberFix(blockNumber, lastBlockTime, blockTime) {
applySystemContractUpgrade(haberFixUpgrade[network], blockNumber, statedb, logger)
}
/* /*
apply other upgrades apply other upgrades
*/ */

View File

@@ -26,7 +26,7 @@ func TestAllCodesHash(t *testing.T) {
allCodes := make([]byte, 0, 10_000_000) allCodes := make([]byte, 0, 10_000_000)
for _, hardfork := range upgradesList { for _, hardfork := range upgradesList {
for _, network := range []string{mainNet, chapelNet, rialtoNet} { for _, network := range []string{mainNet, chapelNet} {
allCodes = append(allCodes, []byte(network)...) allCodes = append(allCodes, []byte(network)...)
if hardfork[network] != nil { if hardfork[network] != nil {
for _, addressConfig := range hardfork[network].Configs { for _, addressConfig := range hardfork[network].Configs {
@@ -37,6 +37,5 @@ func TestAllCodesHash(t *testing.T) {
} }
} }
allCodeHash := sha256.Sum256(allCodes) allCodeHash := sha256.Sum256(allCodes)
require.Equal(t, allCodeHash[:], common.Hex2Bytes("833cc0fc87c46ad8a223e44ccfdc16a51a7e7383525136441bd0c730f06023df"))
require.Equal(t, allCodeHash[:], common.Hex2Bytes("3d68c07faa6b9385e981a45bd539f15d4cbb712426c604b9cab22591af446fc8"))
} }

View File

@@ -212,7 +212,7 @@ func TestTxIndexer(t *testing.T) {
} }
for _, c := range cases { for _, c := range cases {
frdir := t.TempDir() frdir := t.TempDir()
db, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) db, _ := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false, false)
rawdb.WriteAncientBlocks(db, append([]*types.Block{gspec.ToBlock()}, blocks...), append([]types.Receipts{{}}, receipts...), big.NewInt(0)) rawdb.WriteAncientBlocks(db, append([]*types.Block{gspec.ToBlock()}, blocks...), append([]types.Receipts{{}}, receipts...), big.NewInt(0))
// Index the initial blocks from ancient store // Index the initial blocks from ancient store

View File

@@ -27,6 +27,7 @@ import (
"path/filepath" "path/filepath"
"sort" "sort"
"sync" "sync"
"sync/atomic"
"time" "time"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
@@ -305,6 +306,7 @@ type BlobPool struct {
head *types.Header // Current head of the chain head *types.Header // Current head of the chain
state *state.StateDB // Current state at the head of the chain state *state.StateDB // Current state at the head of the chain
gasTip *uint256.Int // Currently accepted minimum gas tip gasTip *uint256.Int // Currently accepted minimum gas tip
maxGas atomic.Uint64 // Currently accepted max gas, it will be modified by MinerAPI
lookup map[common.Hash]uint64 // Lookup table mapping hashes to tx billy entries lookup map[common.Hash]uint64 // Lookup table mapping hashes to tx billy entries
index map[common.Address][]*blobTxMeta // Blob transactions grouped by accounts, sorted by nonce index map[common.Address][]*blobTxMeta // Blob transactions grouped by accounts, sorted by nonce
@@ -1098,6 +1100,7 @@ func (p *BlobPool) validateTx(tx *types.Transaction) error {
Accept: 1 << types.BlobTxType, Accept: 1 << types.BlobTxType,
MaxSize: txMaxSize, MaxSize: txMaxSize,
MinTip: p.gasTip.ToBig(), MinTip: p.gasTip.ToBig(),
MaxGas: p.GetMaxGas(),
} }
if err := txpool.ValidateTransaction(tx, p.head, p.signer, baseOpts); err != nil { if err := txpool.ValidateTransaction(tx, p.head, p.signer, baseOpts); err != nil {
return err return err
@@ -1671,3 +1674,11 @@ func (p *BlobPool) Status(hash common.Hash) txpool.TxStatus {
} }
return txpool.TxStatusUnknown return txpool.TxStatusUnknown
} }
func (p *BlobPool) SetMaxGas(maxGas uint64) {
p.maxGas.Store(maxGas)
}
func (p *BlobPool) GetMaxGas() uint64 {
return p.maxGas.Load()
}

View File

@@ -219,6 +219,7 @@ type LegacyPool struct {
scope event.SubscriptionScope scope event.SubscriptionScope
signer types.Signer signer types.Signer
mu sync.RWMutex mu sync.RWMutex
maxGas atomic.Uint64 // Currently accepted max gas, it will be modified by MinerAPI
currentHead atomic.Pointer[types.Header] // Current head of the blockchain currentHead atomic.Pointer[types.Header] // Current head of the blockchain
currentState *state.StateDB // Current state in the blockchain head currentState *state.StateDB // Current state in the blockchain head
@@ -670,6 +671,7 @@ func (pool *LegacyPool) validateTxBasics(tx *types.Transaction, local bool) erro
1<<types.DynamicFeeTxType, 1<<types.DynamicFeeTxType,
MaxSize: txMaxSize, MaxSize: txMaxSize,
MinTip: pool.gasTip.Load().ToBig(), MinTip: pool.gasTip.Load().ToBig(),
MaxGas: pool.GetMaxGas(),
} }
if local { if local {
opts.MinTip = new(big.Int) opts.MinTip = new(big.Int)
@@ -1769,6 +1771,14 @@ func (pool *LegacyPool) demoteUnexecutables() {
} }
} }
func (pool *LegacyPool) GetMaxGas() uint64 {
return pool.maxGas.Load()
}
func (pool *LegacyPool) SetMaxGas(maxGas uint64) {
pool.maxGas.Store(maxGas)
}
// addressByHeartbeat is an account address tagged with its last activity timestamp. // addressByHeartbeat is an account address tagged with its last activity timestamp.
type addressByHeartbeat struct { type addressByHeartbeat struct {
address common.Address address common.Address

View File

@@ -166,4 +166,7 @@ type SubPool interface {
// Status returns the known status (unknown/pending/queued) of a transaction // Status returns the known status (unknown/pending/queued) of a transaction
// identified by their hashes. // identified by their hashes.
Status(hash common.Hash) TxStatus Status(hash common.Hash) TxStatus
// SetMaxGas limit max acceptable tx gas when mine is enabled
SetMaxGas(maxGas uint64)
} }

View File

@@ -284,6 +284,12 @@ func (p *TxPool) SetGasTip(tip *big.Int) {
} }
} }
func (p *TxPool) SetMaxGas(gas uint64) {
for _, subpool := range p.subpools {
subpool.SetMaxGas(gas)
}
}
// Has returns an indicator whether the pool has a transaction cached with the // Has returns an indicator whether the pool has a transaction cached with the
// given hash. // given hash.
func (p *TxPool) Has(hash common.Hash) bool { func (p *TxPool) Has(hash common.Hash) bool {

View File

@@ -45,6 +45,7 @@ type ValidationOptions struct {
Accept uint8 // Bitmap of transaction types that should be accepted for the calling pool Accept uint8 // Bitmap of transaction types that should be accepted for the calling pool
MaxSize uint64 // Maximum size of a transaction that the caller can meaningfully handle MaxSize uint64 // Maximum size of a transaction that the caller can meaningfully handle
MinTip *big.Int // Minimum gas tip needed to allow a transaction into the caller pool MinTip *big.Int // Minimum gas tip needed to allow a transaction into the caller pool
MaxGas uint64 // Max acceptable transaction gas in the txpool
} }
// ValidateTransaction is a helper method to check whether a transaction is valid // ValidateTransaction is a helper method to check whether a transaction is valid
@@ -86,6 +87,12 @@ func ValidateTransaction(tx *types.Transaction, head *types.Header, signer types
if head.GasLimit < tx.Gas() { if head.GasLimit < tx.Gas() {
return ErrGasLimit return ErrGasLimit
} }
// Ensure the transaction doesn't exceed the current miner max acceptable limit gas
if opts.MaxGas > 0 && opts.MaxGas < tx.Gas() {
return ErrGasLimit
}
// Sanity check for extremely large numbers (supported by RLP or RPC) // Sanity check for extremely large numbers (supported by RLP or RPC)
if tx.GasFeeCap().BitLen() > 256 { if tx.GasFeeCap().BitLen() > 256 {
return core.ErrFeeCapVeryHigh return core.ErrFeeCapVeryHigh

View File

@@ -6,6 +6,8 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
mapset "github.com/deckarep/golang-set/v2"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil" "github.com/ethereum/go-ethereum/common/hexutil"
"github.com/ethereum/go-ethereum/crypto" "github.com/ethereum/go-ethereum/crypto"
@@ -40,6 +42,12 @@ func (b *BidArgs) ToBid(builder common.Address, signer Signer) (*Bid, error) {
return nil, err return nil, err
} }
if len(b.RawBid.UnRevertible) > len(txs) {
return nil, fmt.Errorf("expect NonRevertible no more than %d", len(txs))
}
unRevertibleHashes := mapset.NewThreadUnsafeSetWithSize[common.Hash](len(b.RawBid.UnRevertible))
unRevertibleHashes.Append(b.RawBid.UnRevertible...)
if len(b.PayBidTx) != 0 { if len(b.PayBidTx) != 0 {
var payBidTx = new(Transaction) var payBidTx = new(Transaction)
err = payBidTx.UnmarshalBinary(b.PayBidTx) err = payBidTx.UnmarshalBinary(b.PayBidTx)
@@ -51,14 +59,15 @@ func (b *BidArgs) ToBid(builder common.Address, signer Signer) (*Bid, error) {
} }
bid := &Bid{ bid := &Bid{
Builder: builder, Builder: builder,
BlockNumber: b.RawBid.BlockNumber, BlockNumber: b.RawBid.BlockNumber,
ParentHash: b.RawBid.ParentHash, ParentHash: b.RawBid.ParentHash,
Txs: txs, Txs: txs,
GasUsed: b.RawBid.GasUsed + b.PayBidTxGasUsed, UnRevertible: unRevertibleHashes,
GasFee: b.RawBid.GasFee, GasUsed: b.RawBid.GasUsed + b.PayBidTxGasUsed,
BuilderFee: b.RawBid.BuilderFee, GasFee: b.RawBid.GasFee,
rawBid: *b.RawBid, BuilderFee: b.RawBid.BuilderFee,
rawBid: *b.RawBid,
} }
if bid.BuilderFee == nil { if bid.BuilderFee == nil {
@@ -70,12 +79,13 @@ func (b *BidArgs) ToBid(builder common.Address, signer Signer) (*Bid, error) {
// RawBid represents a raw bid from builder directly. // RawBid represents a raw bid from builder directly.
type RawBid struct { type RawBid struct {
BlockNumber uint64 `json:"blockNumber"` BlockNumber uint64 `json:"blockNumber"`
ParentHash common.Hash `json:"parentHash"` ParentHash common.Hash `json:"parentHash"`
Txs []hexutil.Bytes `json:"txs"` Txs []hexutil.Bytes `json:"txs"`
GasUsed uint64 `json:"gasUsed"` UnRevertible []common.Hash `json:"unRevertible"`
GasFee *big.Int `json:"gasFee"` GasUsed uint64 `json:"gasUsed"`
BuilderFee *big.Int `json:"builderFee"` GasFee *big.Int `json:"gasFee"`
BuilderFee *big.Int `json:"builderFee"`
hash atomic.Value hash atomic.Value
} }
@@ -154,13 +164,14 @@ func (b *RawBid) Hash() common.Hash {
// Bid represents a bid. // Bid represents a bid.
type Bid struct { type Bid struct {
Builder common.Address Builder common.Address
BlockNumber uint64 BlockNumber uint64
ParentHash common.Hash ParentHash common.Hash
Txs Transactions Txs Transactions
GasUsed uint64 UnRevertible mapset.Set[common.Hash]
GasFee *big.Int GasUsed uint64
BuilderFee *big.Int GasFee *big.Int
BuilderFee *big.Int
rawBid RawBid rawBid RawBid
} }
@@ -182,5 +193,7 @@ type MevParams struct {
ValidatorCommission uint64 // 100 means 1% ValidatorCommission uint64 // 100 means 1%
BidSimulationLeftOver time.Duration BidSimulationLeftOver time.Duration
GasCeil uint64 GasCeil uint64
GasPrice *big.Int // Minimum avg gas price for bid block
BuilderFeeCeil *big.Int BuilderFeeCeil *big.Int
Version string
} }

View File

@@ -36,6 +36,7 @@ import (
"github.com/ethereum/go-ethereum/crypto/bn256" "github.com/ethereum/go-ethereum/crypto/bn256"
"github.com/ethereum/go-ethereum/crypto/kzg4844" "github.com/ethereum/go-ethereum/crypto/kzg4844"
"github.com/ethereum/go-ethereum/crypto/secp256k1" "github.com/ethereum/go-ethereum/crypto/secp256k1"
"github.com/ethereum/go-ethereum/crypto/secp256r1"
"github.com/ethereum/go-ethereum/log" "github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params" "github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rlp" "github.com/ethereum/go-ethereum/rlp"
@@ -247,6 +248,36 @@ var PrecompiledContractsCancun = map[common.Address]PrecompiledContract{
common.BytesToAddress([]byte{105}): &secp256k1SignatureRecover{}, common.BytesToAddress([]byte{105}): &secp256k1SignatureRecover{},
} }
// PrecompiledContractsHaber contains the default set of pre-compiled Ethereum
// contracts used in the Haber release.
var PrecompiledContractsHaber = map[common.Address]PrecompiledContract{
common.BytesToAddress([]byte{1}): &ecrecover{},
common.BytesToAddress([]byte{2}): &sha256hash{},
common.BytesToAddress([]byte{3}): &ripemd160hash{},
common.BytesToAddress([]byte{4}): &dataCopy{},
common.BytesToAddress([]byte{5}): &bigModExp{eip2565: true},
common.BytesToAddress([]byte{6}): &bn256AddIstanbul{},
common.BytesToAddress([]byte{7}): &bn256ScalarMulIstanbul{},
common.BytesToAddress([]byte{8}): &bn256PairingIstanbul{},
common.BytesToAddress([]byte{9}): &blake2F{},
common.BytesToAddress([]byte{0x0a}): &kzgPointEvaluation{},
common.BytesToAddress([]byte{100}): &tmHeaderValidate{},
common.BytesToAddress([]byte{101}): &iavlMerkleProofValidatePlato{},
common.BytesToAddress([]byte{102}): &blsSignatureVerify{},
common.BytesToAddress([]byte{103}): &cometBFTLightBlockValidateHertz{},
common.BytesToAddress([]byte{104}): &verifyDoubleSignEvidence{},
common.BytesToAddress([]byte{105}): &secp256k1SignatureRecover{},
common.BytesToAddress([]byte{0x01, 0x00}): &p256Verify{},
}
// PrecompiledContractsP256Verify contains the precompiled Ethereum
// contract specified in EIP-7212. This is exported for testing purposes.
var PrecompiledContractsP256Verify = map[common.Address]PrecompiledContract{
common.BytesToAddress([]byte{0x01, 0x00}): &p256Verify{},
}
// PrecompiledContractsBLS contains the set of pre-compiled Ethereum // PrecompiledContractsBLS contains the set of pre-compiled Ethereum
// contracts specified in EIP-2537. These are exported for testing purposes. // contracts specified in EIP-2537. These are exported for testing purposes.
var PrecompiledContractsBLS = map[common.Address]PrecompiledContract{ var PrecompiledContractsBLS = map[common.Address]PrecompiledContract{
@@ -262,6 +293,7 @@ var PrecompiledContractsBLS = map[common.Address]PrecompiledContract{
} }
var ( var (
PrecompiledAddressesHaber []common.Address
PrecompiledAddressesCancun []common.Address PrecompiledAddressesCancun []common.Address
PrecompiledAddressesFeynman []common.Address PrecompiledAddressesFeynman []common.Address
PrecompiledAddressesHertz []common.Address PrecompiledAddressesHertz []common.Address
@@ -313,11 +345,16 @@ func init() {
for k := range PrecompiledContractsCancun { for k := range PrecompiledContractsCancun {
PrecompiledAddressesCancun = append(PrecompiledAddressesCancun, k) PrecompiledAddressesCancun = append(PrecompiledAddressesCancun, k)
} }
for k := range PrecompiledContractsHaber {
PrecompiledAddressesHaber = append(PrecompiledAddressesHaber, k)
}
} }
// ActivePrecompiles returns the precompiles enabled with the current configuration. // ActivePrecompiles returns the precompiles enabled with the current configuration.
func ActivePrecompiles(rules params.Rules) []common.Address { func ActivePrecompiles(rules params.Rules) []common.Address {
switch { switch {
case rules.IsHaber:
return PrecompiledAddressesHaber
case rules.IsCancun: case rules.IsCancun:
return PrecompiledAddressesCancun return PrecompiledAddressesCancun
case rules.IsFeynman: case rules.IsFeynman:
@@ -1389,6 +1426,40 @@ func kZGToVersionedHash(kzg kzg4844.Commitment) common.Hash {
return h return h
} }
// P256VERIFY (secp256r1 signature verification)
// implemented as a native contract
type p256Verify struct{}
// RequiredGas returns the gas required to execute the precompiled contract
func (c *p256Verify) RequiredGas(input []byte) uint64 {
return params.P256VerifyGas
}
// Run executes the precompiled contract with given 160 bytes of param, returning the output and the used gas
func (c *p256Verify) Run(input []byte) ([]byte, error) {
// Required input length is 160 bytes
const p256VerifyInputLength = 160
// Check the input length
if len(input) != p256VerifyInputLength {
// Input length is invalid
return nil, nil
}
// Extract the hash, r, s, x, y from the input
hash := input[0:32]
r, s := new(big.Int).SetBytes(input[32:64]), new(big.Int).SetBytes(input[64:96])
x, y := new(big.Int).SetBytes(input[96:128]), new(big.Int).SetBytes(input[128:160])
// Verify the secp256r1 signature
if secp256r1.Verify(hash, r, s, x, y) {
// Signature is valid
return common.LeftPadBytes(common.Big1.Bytes(), 32), nil
} else {
// Signature is invalid
return nil, nil
}
}
// verifyDoubleSignEvidence implements bsc header verification precompile. // verifyDoubleSignEvidence implements bsc header verification precompile.
type verifyDoubleSignEvidence struct{} type verifyDoubleSignEvidence struct{}

View File

@@ -46,17 +46,19 @@ type precompiledFailureTest struct {
// allPrecompiles does not map to the actual set of precompiles, as it also contains // allPrecompiles does not map to the actual set of precompiles, as it also contains
// repriced versions of precompiles at certain slots // repriced versions of precompiles at certain slots
var allPrecompiles = map[common.Address]PrecompiledContract{ var allPrecompiles = map[common.Address]PrecompiledContract{
common.BytesToAddress([]byte{1}): &ecrecover{}, common.BytesToAddress([]byte{1}): &ecrecover{},
common.BytesToAddress([]byte{2}): &sha256hash{}, common.BytesToAddress([]byte{2}): &sha256hash{},
common.BytesToAddress([]byte{3}): &ripemd160hash{}, common.BytesToAddress([]byte{3}): &ripemd160hash{},
common.BytesToAddress([]byte{4}): &dataCopy{}, common.BytesToAddress([]byte{4}): &dataCopy{},
common.BytesToAddress([]byte{5}): &bigModExp{eip2565: false}, common.BytesToAddress([]byte{5}): &bigModExp{eip2565: false},
common.BytesToAddress([]byte{0xf5}): &bigModExp{eip2565: true}, common.BytesToAddress([]byte{0xf5}): &bigModExp{eip2565: true},
common.BytesToAddress([]byte{6}): &bn256AddIstanbul{}, common.BytesToAddress([]byte{6}): &bn256AddIstanbul{},
common.BytesToAddress([]byte{7}): &bn256ScalarMulIstanbul{}, common.BytesToAddress([]byte{7}): &bn256ScalarMulIstanbul{},
common.BytesToAddress([]byte{8}): &bn256PairingIstanbul{}, common.BytesToAddress([]byte{8}): &bn256PairingIstanbul{},
common.BytesToAddress([]byte{9}): &blake2F{}, common.BytesToAddress([]byte{9}): &blake2F{},
common.BytesToAddress([]byte{0x0a}): &kzgPointEvaluation{}, common.BytesToAddress([]byte{0x0a}): &kzgPointEvaluation{},
common.BytesToAddress([]byte{0x01, 0x00}): &p256Verify{},
common.BytesToAddress([]byte{0x0f, 0x0a}): &bls12381G1Add{}, common.BytesToAddress([]byte{0x0f, 0x0a}): &bls12381G1Add{},
common.BytesToAddress([]byte{0x0f, 0x0b}): &bls12381G1Mul{}, common.BytesToAddress([]byte{0x0f, 0x0b}): &bls12381G1Mul{},
common.BytesToAddress([]byte{0x0f, 0x0c}): &bls12381G1MultiExp{}, common.BytesToAddress([]byte{0x0f, 0x0c}): &bls12381G1MultiExp{},
@@ -407,6 +409,18 @@ func BenchmarkPrecompiledBLS12381G2MultiExpWorstCase(b *testing.B) {
benchmarkPrecompiled("0f", testcase, b) benchmarkPrecompiled("0f", testcase, b)
} }
// Benchmarks the sample inputs from the P256VERIFY precompile.
func BenchmarkPrecompiledP256Verify(bench *testing.B) {
t := precompiledTest{
Input: "4cee90eb86eaa050036147a12d49004b6b9c72bd725d39d4785011fe190f0b4da73bd4903f0ce3b639bbbf6e8e80d16931ff4bcf5993d58468e8fb19086e8cac36dbcd03009df8c59286b162af3bd7fcc0450c9aa81be5d10d312af6c66b1d604aebd3099c618202fcfe16ae7770b0c49ab5eadf74b754204a3bb6060e44eff37618b065f9832de4ca6ca971a7a1adc826d0f7c00181a5fb2ddf79ae00b4e10e",
Expected: "0000000000000000000000000000000000000000000000000000000000000001",
Name: "p256Verify",
}
benchmarkPrecompiled("100", t, bench)
}
func TestPrecompiledP256Verify(t *testing.T) { testJson("p256Verify", "100", t) }
func TestDoubleSignSlash(t *testing.T) { func TestDoubleSignSlash(t *testing.T) {
tc := precompiledTest{ tc := precompiledTest{
Input: "f906278202cab9030ff9030ca01062d3d5015b9242bc193a9b0769f3d3780ecb55f97f40a752ae26d0b68cd0d8a0fae1a05fcb14bfd9b8a9f2b65007a9b6c2000de0627a73be644dd993d32342c494976ea74026e726554db657fa54763abd0c3a0aa9a0f385cc58ed297ff0d66eb5580b02853d3478ba418b1819ac659ee05df49b9794a0bf88464af369ed6b8cf02db00f0b9556ffa8d49cd491b00952a7f83431446638a00a6d0870e586a76278fbfdcedf76ef6679af18fc1f9137cfad495f434974ea81b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001820cdf830f4240830f4240846555fa64b90111d983010301846765746888676f312e32302e378664617277696e00007abd731ef8ae07b86091cb8836d58f5444b883422a18825d899035d3e6ea39ad1a50069bf0b86da8b5573dde1cb4a0a34f19ce94e0ef78ff7518c80265b8a3ca56e3c60167523590d4e8dcc324900559465fc0fa403774096614e135de280949b58a45cc96f2ba9e17f848820d41a08429d0d8b33ee72a84f750fefea846cbca54e487129c7961c680bb72309ca888820d42a08c9db14d938b19f9e2261bbeca2679945462be2b58103dfff73665d0d150fb8a804ae755e0fe64b59753f4db6308a1f679747bce186aa2c62b95fa6eeff3fbd08f3b0667e45428a54ade15bad19f49641c499b431b36f65803ea71b379e6b61de501a0232c9ba2d41b40d36ed794c306747bcbc49bf61a0f37409c18bfe2b5bef26a2d880000000000000000b9030ff9030ca01062d3d5015b9242bc193a9b0769f3d3780ecb55f97f40a752ae26d0b68cd0d8a0b2789a5357827ed838335283e15c4dcc42b9bebcbf2919a18613246787e2f96094976ea74026e726554db657fa54763abd0c3a0aa9a071ce4c09ee275206013f0063761bc19c93c13990582f918cc57333634c94ce89a00e095703e5c9b149f253fe89697230029e32484a410b4b1f2c61442d73c3095aa0d317ae19ede7c8a2d3ac9ef98735b049bcb7278d12f48c42b924538b60a25e12b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001820cdf830f4240830f4240846555fa64b90111d983010301846765746888676f312e32302e378664617277696e00007abd731ef8ae07b86091cb8836d58f5444b883422a18825d899035d3e6ea39ad1a50069bf0b86da8b5573dde1cb4a0a34f19ce94e0ef78ff7518c80265b8a3ca56e3c60167523590d4e8dcc324900559465fc0fa403774096614e135de280949b58a45cc96f2ba9e17f848820d41a08429d0d8b33ee72a84f750fefea846cbca54e487129c7961c680bb72309ca888820d42a08c9db14d938b19f9e2261bbeca2679945462be2b58103dfff73665d0d150fb8a80c0b17bfe88534296ff064cb7156548f6deba2d6310d5044ed6485f087dc6ef232e051c28e1909c2b50a3b4f29345d66681c319bef653e52e5d746480d5a3983b00a0b56228685be711834d0f154292d07826dea42a0fad3e4f56c31470b7fbfbea26880000000000000000", Input: "f906278202cab9030ff9030ca01062d3d5015b9242bc193a9b0769f3d3780ecb55f97f40a752ae26d0b68cd0d8a0fae1a05fcb14bfd9b8a9f2b65007a9b6c2000de0627a73be644dd993d32342c494976ea74026e726554db657fa54763abd0c3a0aa9a0f385cc58ed297ff0d66eb5580b02853d3478ba418b1819ac659ee05df49b9794a0bf88464af369ed6b8cf02db00f0b9556ffa8d49cd491b00952a7f83431446638a00a6d0870e586a76278fbfdcedf76ef6679af18fc1f9137cfad495f434974ea81b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001820cdf830f4240830f4240846555fa64b90111d983010301846765746888676f312e32302e378664617277696e00007abd731ef8ae07b86091cb8836d58f5444b883422a18825d899035d3e6ea39ad1a50069bf0b86da8b5573dde1cb4a0a34f19ce94e0ef78ff7518c80265b8a3ca56e3c60167523590d4e8dcc324900559465fc0fa403774096614e135de280949b58a45cc96f2ba9e17f848820d41a08429d0d8b33ee72a84f750fefea846cbca54e487129c7961c680bb72309ca888820d42a08c9db14d938b19f9e2261bbeca2679945462be2b58103dfff73665d0d150fb8a804ae755e0fe64b59753f4db6308a1f679747bce186aa2c62b95fa6eeff3fbd08f3b0667e45428a54ade15bad19f49641c499b431b36f65803ea71b379e6b61de501a0232c9ba2d41b40d36ed794c306747bcbc49bf61a0f37409c18bfe2b5bef26a2d880000000000000000b9030ff9030ca01062d3d5015b9242bc193a9b0769f3d3780ecb55f97f40a752ae26d0b68cd0d8a0b2789a5357827ed838335283e15c4dcc42b9bebcbf2919a18613246787e2f96094976ea74026e726554db657fa54763abd0c3a0aa9a071ce4c09ee275206013f0063761bc19c93c13990582f918cc57333634c94ce89a00e095703e5c9b149f253fe89697230029e32484a410b4b1f2c61442d73c3095aa0d317ae19ede7c8a2d3ac9ef98735b049bcb7278d12f48c42b924538b60a25e12b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001820cdf830f4240830f4240846555fa64b90111d983010301846765746888676f312e32302e378664617277696e00007abd731ef8ae07b86091cb8836d58f5444b883422a18825d899035d3e6ea39ad1a50069bf0b86da8b5573dde1cb4a0a34f19ce94e0ef78ff7518c80265b8a3ca56e3c60167523590d4e8dcc324900559465fc0fa403774096614e135de280949b58a45cc96f2ba9e17f848820d41a08429d0d8b33ee72a84f750fefea846cbca54e487129c7961c680bb72309ca888820d42a08c9db14d938b19f9e2261bbeca2679945462be2b58103dfff73665d0d150fb8a80c0b17bfe88534296ff064cb7156548f6deba2d6310d5044ed6485f087dc6ef232e051c28e1909c2b50a3b4f29345d66681c319bef653e52e5d746480d5a3983b00a0b56228685be711834d0f154292d07826dea42a0fad3e4f56c31470b7fbfbea26880000000000000000",

View File

@@ -48,6 +48,8 @@ type (
func (evm *EVM) precompile(addr common.Address) (PrecompiledContract, bool) { func (evm *EVM) precompile(addr common.Address) (PrecompiledContract, bool) {
var precompiles map[common.Address]PrecompiledContract var precompiles map[common.Address]PrecompiledContract
switch { switch {
case evm.chainRules.IsHaber:
precompiles = PrecompiledContractsHaber
case evm.chainRules.IsCancun: case evm.chainRules.IsCancun:
precompiles = PrecompiledContractsCancun precompiles = PrecompiledContractsCancun
case evm.chainRules.IsFeynman: case evm.chainRules.IsFeynman:

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,21 @@
package secp256r1
import (
"crypto/ecdsa"
"crypto/elliptic"
"math/big"
)
// Generates appropriate public key format from given coordinates
func newPublicKey(x, y *big.Int) *ecdsa.PublicKey {
// Check if the given coordinates are valid
if x == nil || y == nil || !elliptic.P256().IsOnCurve(x, y) {
return nil
}
return &ecdsa.PublicKey{
Curve: elliptic.P256(),
X: x,
Y: y,
}
}

View File

@@ -0,0 +1,22 @@
package secp256r1
import (
"crypto/ecdsa"
"math/big"
)
// Verify verifies the given signature (r, s) for the given hash and public key (x, y).
// It returns true if the signature is valid, false otherwise.
func Verify(hash []byte, r, s, x, y *big.Int) bool {
// Create the public key format
publicKey := newPublicKey(x, y)
// Check if they are invalid public key coordinates
if publicKey == nil {
return false
}
// Verify the signature with the public key,
// then return true if it's valid, false otherwise
return ecdsa.Verify(publicKey, hash, r, s)
}

View File

@@ -24,7 +24,6 @@ import (
"os" "os"
"strings" "strings"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core" "github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/rlp" "github.com/ethereum/go-ethereum/rlp"
@@ -142,31 +141,3 @@ func (api *AdminAPI) ImportChain(file string) (bool, error) {
} }
return true, nil return true, nil
} }
// MevRunning returns true if the validator accept bids from builder
func (api *AdminAPI) MevRunning() bool {
return api.eth.APIBackend.MevRunning()
}
// StartMev starts mev. It notifies the miner to start to receive bids.
func (api *AdminAPI) StartMev() {
api.eth.APIBackend.StartMev()
}
// StopMev stops mev. It notifies the miner to stop receiving bids from this moment,
// but the bids before this moment would still been taken into consideration by mev.
func (api *AdminAPI) StopMev() {
api.eth.APIBackend.StopMev()
}
// AddBuilder adds a builder to the bid simulator.
// url is the endpoint of the builder, for example, "https://mev-builder.amazonaws.com",
// if validator is equipped with sentry, ignore the url.
func (api *AdminAPI) AddBuilder(builder common.Address, url string) error {
return api.eth.APIBackend.AddBuilder(builder, url)
}
// RemoveBuilder removes a builder from the bid simulator.
func (api *AdminAPI) RemoveBuilder(builder common.Address) error {
return api.eth.APIBackend.RemoveBuilder(builder)
}

View File

@@ -484,6 +484,10 @@ func (b *EthAPIBackend) RemoveBuilder(builder common.Address) error {
return b.Miner().RemoveBuilder(builder) return b.Miner().RemoveBuilder(builder)
} }
func (b *EthAPIBackend) HasBuilder(builder common.Address) bool {
return b.Miner().HasBuilder(builder)
}
func (b *EthAPIBackend) SendBid(ctx context.Context, bid *types.BidArgs) (common.Hash, error) { func (b *EthAPIBackend) SendBid(ctx context.Context, bid *types.BidArgs) (common.Hash, error) {
return b.Miner().SendBid(ctx, bid) return b.Miner().SendBid(ctx, bid)
} }

View File

@@ -20,6 +20,8 @@ import (
"math/big" "math/big"
"time" "time"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil" "github.com/ethereum/go-ethereum/common/hexutil"
) )
@@ -71,6 +73,9 @@ func (api *MinerAPI) SetGasPrice(gasPrice hexutil.Big) bool {
// SetGasLimit sets the gaslimit to target towards during mining. // SetGasLimit sets the gaslimit to target towards during mining.
func (api *MinerAPI) SetGasLimit(gasLimit hexutil.Uint64) bool { func (api *MinerAPI) SetGasLimit(gasLimit hexutil.Uint64) bool {
api.e.Miner().SetGasCeil(uint64(gasLimit)) api.e.Miner().SetGasCeil(uint64(gasLimit))
if api.e.Miner().Mining() && uint64(gasLimit) > params.SystemTxsGas {
api.e.TxPool().SetMaxGas(uint64(gasLimit) - params.SystemTxsGas)
}
return true return true
} }
@@ -84,3 +89,31 @@ func (api *MinerAPI) SetEtherbase(etherbase common.Address) bool {
func (api *MinerAPI) SetRecommitInterval(interval int) { func (api *MinerAPI) SetRecommitInterval(interval int) {
api.e.Miner().SetRecommitInterval(time.Duration(interval) * time.Millisecond) api.e.Miner().SetRecommitInterval(time.Duration(interval) * time.Millisecond)
} }
// MevRunning returns true if the validator accept bids from builder
func (api *MinerAPI) MevRunning() bool {
return api.e.APIBackend.MevRunning()
}
// StartMev starts mev. It notifies the miner to start to receive bids.
func (api *MinerAPI) StartMev() {
api.e.APIBackend.StartMev()
}
// StopMev stops mev. It notifies the miner to stop receiving bids from this moment,
// but the bids before this moment would still been taken into consideration by mev.
func (api *MinerAPI) StopMev() {
api.e.APIBackend.StopMev()
}
// AddBuilder adds a builder to the bid simulator.
// url is the endpoint of the builder, for example, "https://mev-builder.amazonaws.com",
// if validator is equipped with sentry, ignore the url.
func (api *MinerAPI) AddBuilder(builder common.Address, url string) error {
return api.e.APIBackend.AddBuilder(builder, url)
}
// RemoveBuilder removes a builder from the bid simulator.
func (api *MinerAPI) RemoveBuilder(builder common.Address) error {
return api.e.APIBackend.RemoveBuilder(builder)
}

View File

@@ -161,12 +161,18 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) {
// Optimize memory distribution by reallocating surplus allowance from the // Optimize memory distribution by reallocating surplus allowance from the
// dirty cache to the clean cache. // dirty cache to the clean cache.
if config.StateScheme == rawdb.PathScheme && config.TrieDirtyCache > pathdb.MaxDirtyBufferSize/1024/1024 { if config.StateScheme == rawdb.PathScheme && config.TrieDirtyCache > pathdb.MaxDirtyBufferSize/1024/1024 {
log.Info("Capped dirty cache size", "provided", common.StorageSize(config.TrieDirtyCache)*1024*1024, "adjusted", common.StorageSize(pathdb.MaxDirtyBufferSize)) log.Info("Capped dirty cache size", "provided", common.StorageSize(config.TrieDirtyCache)*1024*1024,
log.Info("Clean cache size", "provided", common.StorageSize(config.TrieCleanCache)*1024*1024) "adjusted", common.StorageSize(pathdb.MaxDirtyBufferSize))
log.Info("Clean cache size", "provided", common.StorageSize(config.TrieCleanCache)*1024*1024,
"adjusted", common.StorageSize(config.TrieCleanCache+config.TrieDirtyCache-pathdb.MaxDirtyBufferSize/1024/1024)*1024*1024)
config.TrieCleanCache += config.TrieDirtyCache - pathdb.MaxDirtyBufferSize/1024/1024
config.TrieDirtyCache = pathdb.MaxDirtyBufferSize / 1024 / 1024 config.TrieDirtyCache = pathdb.MaxDirtyBufferSize / 1024 / 1024
} }
log.Info("Allocated trie memory caches", "clean", common.StorageSize(config.TrieCleanCache)*1024*1024, "dirty", common.StorageSize(config.TrieDirtyCache)*1024*1024) log.Info("Allocated memory caches",
"state_scheme", config.StateScheme,
"trie_clean_cache", common.StorageSize(config.TrieCleanCache)*1024*1024,
"trie_dirty_cache", common.StorageSize(config.TrieDirtyCache)*1024*1024,
"snapshot_cache", common.StorageSize(config.SnapshotCache)*1024*1024)
// Try to recover offline state pruning only in hash-based. // Try to recover offline state pruning only in hash-based.
if config.StateScheme == rawdb.HashScheme { if config.StateScheme == rawdb.HashScheme {
if err := pruner.RecoverPruning(stack.ResolvePath(""), chainDb, config.TriesInMemory); err != nil { if err := pruner.RecoverPruning(stack.ResolvePath(""), chainDb, config.TriesInMemory); err != nil {
@@ -183,18 +189,18 @@ func New(stack *node.Node, config *ethconfig.Config) (*Ethereum, error) {
chainConfig.CancunTime = config.OverrideCancun chainConfig.CancunTime = config.OverrideCancun
overrides.OverrideCancun = config.OverrideCancun overrides.OverrideCancun = config.OverrideCancun
} }
if config.OverrideHaber != nil {
chainConfig.HaberTime = config.OverrideHaber
overrides.OverrideHaber = config.OverrideHaber
}
if config.OverrideBohr != nil {
chainConfig.BohrTime = config.OverrideBohr
overrides.OverrideBohr = config.OverrideBohr
}
if config.OverrideVerkle != nil { if config.OverrideVerkle != nil {
chainConfig.VerkleTime = config.OverrideVerkle chainConfig.VerkleTime = config.OverrideVerkle
overrides.OverrideVerkle = config.OverrideVerkle overrides.OverrideVerkle = config.OverrideVerkle
} }
if config.OverrideFeynman != nil {
chainConfig.FeynmanTime = config.OverrideFeynman
overrides.OverrideFeynman = config.OverrideFeynman
}
if config.OverrideFeynmanFix != nil {
chainConfig.FeynmanFixTime = config.OverrideFeynmanFix
overrides.OverrideFeynmanFix = config.OverrideFeynmanFix
}
// startup ancient freeze // startup ancient freeze
if err = chainDb.SetupFreezerEnv(&ethdb.FreezerEnv{ if err = chainDb.SetupFreezerEnv(&ethdb.FreezerEnv{

View File

@@ -209,6 +209,9 @@ type BlockChain interface {
// UpdateChasingHead update remote best chain head, used by DA check now. // UpdateChasingHead update remote best chain head, used by DA check now.
UpdateChasingHead(head *types.Header) UpdateChasingHead(head *types.Header)
// AncientTail retrieves the tail the ancients blocks
AncientTail() (uint64, error)
} }
type DownloadOption func(downloader *Downloader) *Downloader type DownloadOption func(downloader *Downloader) *Downloader
@@ -797,6 +800,11 @@ func (d *Downloader) findAncestor(p *peerConnection, localHeight uint64, remoteH
// We're above the max reorg threshold, find the earliest fork point // We're above the max reorg threshold, find the earliest fork point
floor = int64(localHeight - maxForkAncestry) floor = int64(localHeight - maxForkAncestry)
} }
// if we have pruned too much history, reset the floor
if tail, err := d.blockchain.AncientTail(); err == nil && tail > uint64(floor) {
floor = int64(tail)
}
// If we're doing a light sync, ensure the floor doesn't go below the CHT, as // If we're doing a light sync, ensure the floor doesn't go below the CHT, as
// all headers before that point will be missing. // all headers before that point will be missing.
if mode == LightSync { if mode == LightSync {

View File

@@ -60,7 +60,7 @@ func newTester(t *testing.T) *downloadTester {
// newTester creates a new downloader test mocker. // newTester creates a new downloader test mocker.
func newTesterWithNotification(t *testing.T, success func()) *downloadTester { func newTesterWithNotification(t *testing.T, success func()) *downloadTester {
freezer := t.TempDir() freezer := t.TempDir()
db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), freezer, "", false, false, false, false) db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), freezer, "", false, false, false, false, false)
if err != nil { if err != nil {
panic(err) panic(err)
} }

View File

@@ -191,15 +191,15 @@ type Config struct {
// OverrideCancun (TODO: remove after the fork) // OverrideCancun (TODO: remove after the fork)
OverrideCancun *uint64 `toml:",omitempty"` OverrideCancun *uint64 `toml:",omitempty"`
// OverrideHaber (TODO: remove after the fork)
OverrideHaber *uint64 `toml:",omitempty"`
// OverrideBohr (TODO: remove after the fork)
OverrideBohr *uint64 `toml:",omitempty"`
// OverrideVerkle (TODO: remove after the fork) // OverrideVerkle (TODO: remove after the fork)
OverrideVerkle *uint64 `toml:",omitempty"` OverrideVerkle *uint64 `toml:",omitempty"`
// OverrideFeynman (TODO: remove after the fork)
OverrideFeynman *uint64 `toml:",omitempty"`
// OverrideFeynmanFix (TODO: remove after the fork)
OverrideFeynmanFix *uint64 `toml:",omitempty"`
// blob setting // blob setting
BlobExtraReserve uint64 BlobExtraReserve uint64
} }

View File

@@ -32,11 +32,12 @@ func (c Config) MarshalTOML() (interface{}, error) {
EnableTrustProtocol bool EnableTrustProtocol bool
PipeCommit bool PipeCommit bool
RangeLimit bool RangeLimit bool
TxLookupLimit uint64 `toml:",omitempty"` TxLookupLimit uint64 `toml:",omitempty"`
TransactionHistory uint64 `toml:",omitempty"` TransactionHistory uint64 `toml:",omitempty"`
StateHistory uint64 `toml:",omitempty"` StateHistory uint64 `toml:",omitempty"`
StateScheme string `toml:",omitempty"` StateScheme string `toml:",omitempty"`
PathSyncFlush bool `toml:",omitempty"` PathSyncFlush bool `toml:",omitempty"`
JournalFileEnabled bool
RequiredBlocks map[uint64]common.Hash `toml:"-"` RequiredBlocks map[uint64]common.Hash `toml:"-"`
LightServ int `toml:",omitempty"` LightServ int `toml:",omitempty"`
LightIngress int `toml:",omitempty"` LightIngress int `toml:",omitempty"`
@@ -70,9 +71,10 @@ func (c Config) MarshalTOML() (interface{}, error) {
RPCEVMTimeout time.Duration RPCEVMTimeout time.Duration
RPCTxFeeCap float64 RPCTxFeeCap float64
OverrideCancun *uint64 `toml:",omitempty"` OverrideCancun *uint64 `toml:",omitempty"`
OverrideHaber *uint64 `toml:",omitempty"`
OverrideBohr *uint64 `toml:",omitempty"`
OverrideVerkle *uint64 `toml:",omitempty"` OverrideVerkle *uint64 `toml:",omitempty"`
OverrideFeynman *uint64 `toml:",omitempty"` BlobExtraReserve uint64
OverrideFeynmanFix *uint64 `toml:",omitempty"`
} }
var enc Config var enc Config
enc.Genesis = c.Genesis enc.Genesis = c.Genesis
@@ -95,6 +97,7 @@ func (c Config) MarshalTOML() (interface{}, error) {
enc.StateHistory = c.StateHistory enc.StateHistory = c.StateHistory
enc.StateScheme = c.StateScheme enc.StateScheme = c.StateScheme
enc.PathSyncFlush = c.PathSyncFlush enc.PathSyncFlush = c.PathSyncFlush
enc.JournalFileEnabled = c.JournalFileEnabled
enc.RequiredBlocks = c.RequiredBlocks enc.RequiredBlocks = c.RequiredBlocks
enc.LightServ = c.LightServ enc.LightServ = c.LightServ
enc.LightIngress = c.LightIngress enc.LightIngress = c.LightIngress
@@ -128,9 +131,10 @@ func (c Config) MarshalTOML() (interface{}, error) {
enc.RPCEVMTimeout = c.RPCEVMTimeout enc.RPCEVMTimeout = c.RPCEVMTimeout
enc.RPCTxFeeCap = c.RPCTxFeeCap enc.RPCTxFeeCap = c.RPCTxFeeCap
enc.OverrideCancun = c.OverrideCancun enc.OverrideCancun = c.OverrideCancun
enc.OverrideHaber = c.OverrideHaber
enc.OverrideBohr = c.OverrideBohr
enc.OverrideVerkle = c.OverrideVerkle enc.OverrideVerkle = c.OverrideVerkle
enc.OverrideFeynman = c.OverrideFeynman enc.BlobExtraReserve = c.BlobExtraReserve
enc.OverrideFeynmanFix = c.OverrideFeynmanFix
return &enc, nil return &enc, nil
} }
@@ -152,11 +156,12 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error {
EnableTrustProtocol *bool EnableTrustProtocol *bool
PipeCommit *bool PipeCommit *bool
RangeLimit *bool RangeLimit *bool
TxLookupLimit *uint64 `toml:",omitempty"` TxLookupLimit *uint64 `toml:",omitempty"`
TransactionHistory *uint64 `toml:",omitempty"` TransactionHistory *uint64 `toml:",omitempty"`
StateHistory *uint64 `toml:",omitempty"` StateHistory *uint64 `toml:",omitempty"`
StateScheme *string `toml:",omitempty"` StateScheme *string `toml:",omitempty"`
PathSyncFlush *bool `toml:",omitempty"` PathSyncFlush *bool `toml:",omitempty"`
JournalFileEnabled *bool
RequiredBlocks map[uint64]common.Hash `toml:"-"` RequiredBlocks map[uint64]common.Hash `toml:"-"`
LightServ *int `toml:",omitempty"` LightServ *int `toml:",omitempty"`
LightIngress *int `toml:",omitempty"` LightIngress *int `toml:",omitempty"`
@@ -190,9 +195,10 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error {
RPCEVMTimeout *time.Duration RPCEVMTimeout *time.Duration
RPCTxFeeCap *float64 RPCTxFeeCap *float64
OverrideCancun *uint64 `toml:",omitempty"` OverrideCancun *uint64 `toml:",omitempty"`
OverrideHaber *uint64 `toml:",omitempty"`
OverrideBohr *uint64 `toml:",omitempty"`
OverrideVerkle *uint64 `toml:",omitempty"` OverrideVerkle *uint64 `toml:",omitempty"`
OverrideFeynman *uint64 `toml:",omitempty"` BlobExtraReserve *uint64
OverrideFeynmanFix *uint64 `toml:",omitempty"`
} }
var dec Config var dec Config
if err := unmarshal(&dec); err != nil { if err := unmarshal(&dec); err != nil {
@@ -258,6 +264,9 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error {
if dec.PathSyncFlush != nil { if dec.PathSyncFlush != nil {
c.PathSyncFlush = *dec.PathSyncFlush c.PathSyncFlush = *dec.PathSyncFlush
} }
if dec.JournalFileEnabled != nil {
c.JournalFileEnabled = *dec.JournalFileEnabled
}
if dec.RequiredBlocks != nil { if dec.RequiredBlocks != nil {
c.RequiredBlocks = dec.RequiredBlocks c.RequiredBlocks = dec.RequiredBlocks
} }
@@ -357,14 +366,17 @@ func (c *Config) UnmarshalTOML(unmarshal func(interface{}) error) error {
if dec.OverrideCancun != nil { if dec.OverrideCancun != nil {
c.OverrideCancun = dec.OverrideCancun c.OverrideCancun = dec.OverrideCancun
} }
if dec.OverrideHaber != nil {
c.OverrideHaber = dec.OverrideHaber
}
if dec.OverrideBohr != nil {
c.OverrideBohr = dec.OverrideBohr
}
if dec.OverrideVerkle != nil { if dec.OverrideVerkle != nil {
c.OverrideVerkle = dec.OverrideVerkle c.OverrideVerkle = dec.OverrideVerkle
} }
if dec.OverrideFeynman != nil { if dec.BlobExtraReserve != nil {
c.OverrideFeynman = dec.OverrideFeynman c.BlobExtraReserve = *dec.BlobExtraReserve
}
if dec.OverrideFeynmanFix != nil {
c.OverrideFeynmanFix = dec.OverrideFeynmanFix
} }
return nil return nil
} }

View File

@@ -22,6 +22,7 @@ import (
"math/rand" "math/rand"
"time" "time"
mapset "github.com/deckarep/golang-set/v2"
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/prque" "github.com/ethereum/go-ethereum/common/prque"
"github.com/ethereum/go-ethereum/consensus" "github.com/ethereum/go-ethereum/consensus"
@@ -66,6 +67,10 @@ var (
headerFilterOutMeter = metrics.NewRegisteredMeter("eth/fetcher/block/filter/headers/out", nil) headerFilterOutMeter = metrics.NewRegisteredMeter("eth/fetcher/block/filter/headers/out", nil)
bodyFilterInMeter = metrics.NewRegisteredMeter("eth/fetcher/block/filter/bodies/in", nil) bodyFilterInMeter = metrics.NewRegisteredMeter("eth/fetcher/block/filter/bodies/in", nil)
bodyFilterOutMeter = metrics.NewRegisteredMeter("eth/fetcher/block/filter/bodies/out", nil) bodyFilterOutMeter = metrics.NewRegisteredMeter("eth/fetcher/block/filter/bodies/out", nil)
blockInsertFailRecords = mapset.NewSet[common.Hash]()
blockInsertFailRecordslimit = 1000
blockInsertFailGauge = metrics.NewRegisteredGauge("chain/insert/failed", nil)
) )
var errTerminated = errors.New("terminated") var errTerminated = errors.New("terminated")
@@ -726,9 +731,6 @@ func (f *BlockFetcher) loop() {
matched = true matched = true
if f.getBlock(hash) == nil { if f.getBlock(hash) == nil {
block := types.NewBlockWithHeader(announce.header).WithBody(task.transactions[i], task.uncles[i]) block := types.NewBlockWithHeader(announce.header).WithBody(task.transactions[i], task.uncles[i])
if block.Header().EmptyWithdrawalsHash() {
block = block.WithWithdrawals(make([]*types.Withdrawal, 0))
}
block = block.WithSidecars(task.sidecars[i]) block = block.WithSidecars(task.sidecars[i])
block.ReceivedAt = task.time block.ReceivedAt = task.time
blocks = append(blocks, block) blocks = append(blocks, block)
@@ -914,6 +916,10 @@ func (f *BlockFetcher) importBlocks(op *blockOrHeaderInject) {
return return
} }
if block.Header().EmptyWithdrawalsHash() {
block = block.WithWithdrawals(make([]*types.Withdrawal, 0))
}
defer func() { f.done <- hash }() defer func() { f.done <- hash }()
// Quickly validate the header and propagate the block if it passes // Quickly validate the header and propagate the block if it passes
switch err := f.verifyHeader(block.Header()); err { switch err := f.verifyHeader(block.Header()); err {
@@ -934,6 +940,10 @@ func (f *BlockFetcher) importBlocks(op *blockOrHeaderInject) {
} }
// Run the actual import and log any issues // Run the actual import and log any issues
if _, err := f.insertChain(types.Blocks{block}); err != nil { if _, err := f.insertChain(types.Blocks{block}); err != nil {
if blockInsertFailRecords.Cardinality() < blockInsertFailRecordslimit {
blockInsertFailRecords.Add(block.Hash())
blockInsertFailGauge.Update(int64(blockInsertFailRecords.Cardinality()))
}
log.Debug("Propagated block import failed", "peer", peer, "number", block.Number(), "hash", hash, "err", err) log.Debug("Propagated block import failed", "peer", peer, "number", block.Number(), "hash", hash, "err", err)
return return
} }

View File

@@ -158,7 +158,7 @@ func (f *fetcherTester) chainFinalizedHeight() uint64 {
return f.blocks[f.hashes[len(f.hashes)-3]].NumberU64() return f.blocks[f.hashes[len(f.hashes)-3]].NumberU64()
} }
// insertChain injects a new headers into the simulated chain. // insertHeaders injects a new headers into the simulated chain.
func (f *fetcherTester) insertHeaders(headers []*types.Header) (int, error) { func (f *fetcherTester) insertHeaders(headers []*types.Header) (int, error) {
f.lock.Lock() f.lock.Lock()
defer f.lock.Unlock() defer f.lock.Unlock()

View File

@@ -44,6 +44,9 @@ var (
// The maximum number of topic criteria allowed, vm.LOG4 - vm.LOG0 // The maximum number of topic criteria allowed, vm.LOG4 - vm.LOG0
const maxTopics = 4 const maxTopics = 4
// The maximum number of allowed topics within a topic criteria
const maxSubTopics = 1000
// filter is a helper struct that holds meta information over the filter type // filter is a helper struct that holds meta information over the filter type
// and associated subscription in the event system. // and associated subscription in the event system.
type filter struct { type filter struct {
@@ -671,6 +674,9 @@ func (args *FilterCriteria) UnmarshalJSON(data []byte) error {
return errors.New("invalid addresses in query") return errors.New("invalid addresses in query")
} }
} }
if len(raw.Topics) > maxTopics {
return errExceedMaxTopics
}
// topics is an array consisting of strings and/or arrays of strings. // topics is an array consisting of strings and/or arrays of strings.
// JSON null values are converted to common.Hash{} and ignored by the filter manager. // JSON null values are converted to common.Hash{} and ignored by the filter manager.
@@ -691,6 +697,9 @@ func (args *FilterCriteria) UnmarshalJSON(data []byte) error {
case []interface{}: case []interface{}:
// or case e.g. [null, "topic0", "topic1"] // or case e.g. [null, "topic0", "topic1"]
if len(topic) > maxSubTopics {
return errExceedMaxTopics
}
for _, rawTopic := range topic { for _, rawTopic := range topic {
if rawTopic == nil { if rawTopic == nil {
// null component, match all // null component, match all

View File

@@ -318,7 +318,30 @@ func newHandler(config *handlerConfig) (*handler, error) {
} }
return h.chain.InsertChain(blocks) return h.chain.InsertChain(blocks)
} }
h.blockFetcher = fetcher.NewBlockFetcher(false, nil, h.chain.GetBlockByHash, validator, h.BroadcastBlock,
broadcastBlockWithCheck := func(block *types.Block, propagate bool) {
if propagate {
checkErrs := make(chan error, 2)
go func() {
checkErrs <- core.ValidateListsInBody(block)
}()
go func() {
checkErrs <- core.IsDataAvailable(h.chain, block)
}()
for i := 0; i < cap(checkErrs); i++ {
err := <-checkErrs
if err != nil {
log.Error("Propagating invalid block", "number", block.Number(), "hash", block.Hash(), "err", err)
return
}
}
}
h.BroadcastBlock(block, propagate)
}
h.blockFetcher = fetcher.NewBlockFetcher(false, nil, h.chain.GetBlockByHash, validator, broadcastBlockWithCheck,
heighter, finalizeHeighter, nil, inserter, h.removePeer) heighter, finalizeHeighter, nil, inserter, h.removePeer)
fetchTx := func(peer string, hashes []common.Hash) error { fetchTx := func(peer string, hashes []common.Hash) error {

View File

@@ -633,6 +633,9 @@ func testBroadcastBlock(t *testing.T, peers, bcasts int) {
go source.handler.runEthPeer(sourcePeer, func(peer *eth.Peer) error { go source.handler.runEthPeer(sourcePeer, func(peer *eth.Peer) error {
return eth.Handle((*ethHandler)(source.handler), peer) return eth.Handle((*ethHandler)(source.handler), peer)
}) })
// Wait a bit for the above handlers to start
time.Sleep(100 * time.Millisecond)
if err := sinkPeer.Handshake(1, td, genesis.Hash(), genesis.Hash(), forkid.NewIDWithChain(source.chain), forkid.NewFilter(source.chain), nil); err != nil { if err := sinkPeer.Handshake(1, td, genesis.Hash(), genesis.Hash(), forkid.NewIDWithChain(source.chain), forkid.NewFilter(source.chain), nil); err != nil {
t.Fatalf("failed to run protocol handshake") t.Fatalf("failed to run protocol handshake")
} }

View File

@@ -258,7 +258,7 @@ func (c *mockParlia) CalcDifficulty(chain consensus.ChainHeaderReader, time uint
func newTestParliaHandlerAfterCancun(t *testing.T, config *params.ChainConfig, mode downloader.SyncMode, preCancunBlks, postCancunBlks uint64) *testHandler { func newTestParliaHandlerAfterCancun(t *testing.T, config *params.ChainConfig, mode downloader.SyncMode, preCancunBlks, postCancunBlks uint64) *testHandler {
// Have N headers in the freezer // Have N headers in the freezer
frdir := t.TempDir() frdir := t.TempDir()
db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false) db, err := rawdb.NewDatabaseWithFreezer(rawdb.NewMemoryDatabase(), frdir, "", false, false, false, false, false)
if err != nil { if err != nil {
t.Fatalf("failed to create database with ancient backend") t.Fatalf("failed to create database with ancient backend")
} }

View File

@@ -151,6 +151,8 @@ func testChainSyncWithBlobs(t *testing.T, mode downloader.SyncMode, preCancunBlk
go full.handler.runEthPeer(fullPeerEth, func(peer *eth.Peer) error { go full.handler.runEthPeer(fullPeerEth, func(peer *eth.Peer) error {
return eth.Handle((*ethHandler)(full.handler), peer) return eth.Handle((*ethHandler)(full.handler), peer)
}) })
// Wait a bit for the above handlers to start
time.Sleep(250 * time.Millisecond)
emptyPipeSnap, fullPipeSnap := p2p.MsgPipe() emptyPipeSnap, fullPipeSnap := p2p.MsgPipe()
defer emptyPipeSnap.Close() defer emptyPipeSnap.Close()

View File

@@ -143,7 +143,7 @@ func (ec *Client) BlobSidecars(ctx context.Context, blockNrOrHash rpc.BlockNumbe
// BlobSidecarByTxHash return a sidecar of a given blob transaction // BlobSidecarByTxHash return a sidecar of a given blob transaction
func (ec *Client) BlobSidecarByTxHash(ctx context.Context, hash common.Hash) (*types.BlobTxSidecar, error) { func (ec *Client) BlobSidecarByTxHash(ctx context.Context, hash common.Hash) (*types.BlobTxSidecar, error) {
var r *types.BlobTxSidecar var r *types.BlobTxSidecar
err := ec.c.CallContext(ctx, &r, "eth_getBlockSidecarByTxHash", hash) err := ec.c.CallContext(ctx, &r, "eth_getBlobSidecarByTxHash", hash)
if err == nil && r == nil { if err == nil && r == nil {
return nil, ethereum.NotFound return nil, ethereum.NotFound
} }
@@ -752,6 +752,13 @@ func (ec *Client) MevRunning(ctx context.Context) (bool, error) {
return result, err return result, err
} }
// HasBuilder returns whether the builder is registered
func (ec *Client) HasBuilder(ctx context.Context, address common.Address) (bool, error) {
var result bool
err := ec.c.CallContext(ctx, &result, "mev_hasBuilder", address)
return result, err
}
// SendBid sends a bid // SendBid sends a bid
func (ec *Client) SendBid(ctx context.Context, args types.BidArgs) (common.Hash, error) { func (ec *Client) SendBid(ctx context.Context, args types.BidArgs) (common.Hash, error) {
var hash common.Hash var hash common.Hash

View File

@@ -183,6 +183,7 @@ type StateStoreReader interface {
type BlockStore interface { type BlockStore interface {
BlockStore() Database BlockStore() Database
SetBlockStore(block Database) SetBlockStore(block Database)
HasSeparateBlockStore() bool
} }
type BlockStoreReader interface { type BlockStoreReader interface {

View File

@@ -48,6 +48,9 @@ const (
// numLevels is the level number of pebble sst files // numLevels is the level number of pebble sst files
numLevels = 7 numLevels = 7
// degradationWarnInterval specifies how often warning should be printed if the
// leveldb database cannot keep up with requested writes.
degradationWarnInterval = time.Minute
) )
// Database is a persistent key-value store based on the pebble storage engine. // Database is a persistent key-value store based on the pebble storage engine.
@@ -79,14 +82,16 @@ type Database struct {
log log.Logger // Contextual logger tracking the database path log log.Logger // Contextual logger tracking the database path
activeComp int // Current number of active compactions activeComp int // Current number of active compactions
compStartTime time.Time // The start time of the earliest currently-active compaction compStartTime time.Time // The start time of the earliest currently-active compaction
compTime atomic.Int64 // Total time spent in compaction in ns compTime atomic.Int64 // Total time spent in compaction in ns
level0Comp atomic.Uint32 // Total number of level-zero compactions level0Comp atomic.Uint32 // Total number of level-zero compactions
nonLevel0Comp atomic.Uint32 // Total number of non level-zero compactions nonLevel0Comp atomic.Uint32 // Total number of non level-zero compactions
writeDelayStartTime time.Time // The start time of the latest write stall
writeDelayCount atomic.Int64 // Total number of write stall counts writeStalled atomic.Bool // Flag whether the write is stalled
writeDelayTime atomic.Int64 // Total time spent in write stalls writeDelayStartTime time.Time // The start time of the latest write stall
writeDelayCount atomic.Int64 // Total number of write stall counts
writeDelayTime atomic.Int64 // Total time spent in write stalls
writeOptions *pebble.WriteOptions writeOptions *pebble.WriteOptions
} }
@@ -115,10 +120,13 @@ func (d *Database) onCompactionEnd(info pebble.CompactionInfo) {
func (d *Database) onWriteStallBegin(b pebble.WriteStallBeginInfo) { func (d *Database) onWriteStallBegin(b pebble.WriteStallBeginInfo) {
d.writeDelayStartTime = time.Now() d.writeDelayStartTime = time.Now()
d.writeDelayCount.Add(1)
d.writeStalled.Store(true)
} }
func (d *Database) onWriteStallEnd() { func (d *Database) onWriteStallEnd() {
d.writeDelayTime.Add(int64(time.Since(d.writeDelayStartTime))) d.writeDelayTime.Add(int64(time.Since(d.writeDelayStartTime)))
d.writeStalled.Store(false)
} }
// panicLogger is just a noop logger to disable Pebble's internal logger. // panicLogger is just a noop logger to disable Pebble's internal logger.
@@ -461,13 +469,15 @@ func (d *Database) meter(refresh time.Duration, namespace string) {
// Create storage and warning log tracer for write delay. // Create storage and warning log tracer for write delay.
var ( var (
compTimes [2]int64 compTimes [2]int64
writeDelayTimes [2]int64 compWrites [2]int64
writeDelayCounts [2]int64 compReads [2]int64
compWrites [2]int64
compReads [2]int64
nWrites [2]int64 nWrites [2]int64
writeDelayTimes [2]int64
writeDelayCounts [2]int64
lastWriteStallReport time.Time
) )
// Iterate ad infinitum and collect the stats // Iterate ad infinitum and collect the stats
@@ -507,6 +517,13 @@ func (d *Database) meter(refresh time.Duration, namespace string) {
if d.writeDelayMeter != nil { if d.writeDelayMeter != nil {
d.writeDelayMeter.Mark(writeDelayTimes[i%2] - writeDelayTimes[(i-1)%2]) d.writeDelayMeter.Mark(writeDelayTimes[i%2] - writeDelayTimes[(i-1)%2])
} }
// Print a warning log if writing has been stalled for a while. The log will
// be printed per minute to avoid overwhelming users.
if d.writeStalled.Load() && writeDelayCounts[i%2] == writeDelayCounts[(i-1)%2] &&
time.Now().After(lastWriteStallReport.Add(degradationWarnInterval)) {
d.log.Warn("Database compacting, degraded performance")
lastWriteStallReport = time.Now()
}
if d.compTimeMeter != nil { if d.compTimeMeter != nil {
d.compTimeMeter.Mark(compTimes[i%2] - compTimes[(i-1)%2]) d.compTimeMeter.Mark(compTimes[i%2] - compTimes[(i-1)%2])
} }
@@ -600,8 +617,8 @@ func (b *batch) Reset() {
func (b *batch) Replay(w ethdb.KeyValueWriter) error { func (b *batch) Replay(w ethdb.KeyValueWriter) error {
reader := b.b.Reader() reader := b.b.Reader()
for { for {
kind, k, v, ok := reader.Next() kind, k, v, ok, err := reader.Next()
if !ok { if !ok || err != nil {
break break
} }
// The (k,v) slices might be overwritten if the batch is reset/reused, // The (k,v) slices might be overwritten if the batch is reset/reused,

View File

@@ -44,6 +44,10 @@ func (db *Database) BlockStore() ethdb.Database {
return db return db
} }
func (db *Database) HasSeparateBlockStore() bool {
return false
}
func (db *Database) SetBlockStore(block ethdb.Database) { func (db *Database) SetBlockStore(block ethdb.Database) {
panic("not supported") panic("not supported")
} }

10
go.mod
View File

@@ -17,8 +17,8 @@ require (
github.com/btcsuite/btcd/btcec/v2 v2.3.2 github.com/btcsuite/btcd/btcec/v2 v2.3.2
github.com/cespare/cp v1.1.1 github.com/cespare/cp v1.1.1
github.com/cloudflare/cloudflare-go v0.79.0 github.com/cloudflare/cloudflare-go v0.79.0
github.com/cockroachdb/pebble v0.0.0-20230928194634-aa077af62593
github.com/cometbft/cometbft v0.37.0 github.com/cometbft/cometbft v0.37.0
github.com/cockroachdb/pebble v1.1.0
github.com/consensys/gnark-crypto v0.12.1 github.com/consensys/gnark-crypto v0.12.1
github.com/crate-crypto/go-ipa v0.0.0-20231025140028-3c0104f4b233 github.com/crate-crypto/go-ipa v0.0.0-20231025140028-3c0104f4b233
github.com/crate-crypto/go-kzg-4844 v0.7.0 github.com/crate-crypto/go-kzg-4844 v0.7.0
@@ -81,10 +81,10 @@ require (
github.com/wealdtech/go-eth2-wallet-encryptor-keystorev4 v1.1.3 github.com/wealdtech/go-eth2-wallet-encryptor-keystorev4 v1.1.3
github.com/willf/bitset v1.1.3 github.com/willf/bitset v1.1.3
go.uber.org/automaxprocs v1.5.2 go.uber.org/automaxprocs v1.5.2
golang.org/x/crypto v0.19.0 golang.org/x/crypto v0.21.0
golang.org/x/exp v0.0.0-20240213143201-ec583247a57a golang.org/x/exp v0.0.0-20240213143201-ec583247a57a
golang.org/x/sync v0.6.0 golang.org/x/sync v0.6.0
golang.org/x/sys v0.17.0 golang.org/x/sys v0.18.0
golang.org/x/text v0.14.0 golang.org/x/text v0.14.0
golang.org/x/time v0.5.0 golang.org/x/time v0.5.0
golang.org/x/tools v0.18.0 golang.org/x/tools v0.18.0
@@ -274,9 +274,9 @@ require (
go.uber.org/multierr v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect go.uber.org/zap v1.27.0 // indirect
golang.org/x/mod v0.15.0 // indirect golang.org/x/mod v0.15.0 // indirect
golang.org/x/net v0.21.0 // indirect golang.org/x/net v0.23.0 // indirect
golang.org/x/oauth2 v0.16.0 // indirect golang.org/x/oauth2 v0.16.0 // indirect
golang.org/x/term v0.17.0 // indirect golang.org/x/term v0.18.0 // indirect
google.golang.org/api v0.44.0 // indirect google.golang.org/api v0.44.0 // indirect
google.golang.org/appengine v1.6.7 // indirect google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect

20
go.sum
View File

@@ -275,8 +275,8 @@ github.com/cockroachdb/errors v1.11.1 h1:xSEW75zKaKCWzR3OfxXUxgrk/NtT4G1MiOv5lWZ
github.com/cockroachdb/errors v1.11.1/go.mod h1:8MUxA3Gi6b25tYlFEBGLf+D8aISL+M4MIpiWMSNRfxw= github.com/cockroachdb/errors v1.11.1/go.mod h1:8MUxA3Gi6b25tYlFEBGLf+D8aISL+M4MIpiWMSNRfxw=
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZeQy818SGhaone5OnYfxFR/+AzdY3sf5aE= github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZeQy818SGhaone5OnYfxFR/+AzdY3sf5aE=
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs= github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=
github.com/cockroachdb/pebble v0.0.0-20230928194634-aa077af62593 h1:aPEJyR4rPBvDmeyi+l/FS/VtA00IWvjeFvjen1m1l1A= github.com/cockroachdb/pebble v1.1.0 h1:pcFh8CdCIt2kmEpK0OIatq67Ln9uGDYY3d5XnE0LJG4=
github.com/cockroachdb/pebble v0.0.0-20230928194634-aa077af62593/go.mod h1:6hk1eMY/u5t+Cf18q5lFMUA1Rc+Sm5I6Ra1QuPyxXCo= github.com/cockroachdb/pebble v1.1.0/go.mod h1:sEHm5NOXxyiAoKWhoFxT8xMgd/f3RA6qUqQ1BXKrh2E=
github.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30= github.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30=
github.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg= github.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo= github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 h1:zuQyyAKVxetITBuuhv3BI9cMrmStnpT18zmgmTxunpo=
@@ -1743,8 +1743,8 @@ golang.org/x/crypto v0.0.0-20210813211128-0a44fdfbc16e/go.mod h1:GvvjBRRGRdwPK5y
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211117183948-ae814b36b871/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.0.0-20211117183948-ae814b36b871/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.19.0 h1:ENy+Az/9Y1vSrlrvBSyna3PITt4tiZLf7sgCjZBX7Wo= golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU= golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -1857,8 +1857,8 @@ golang.org/x/net v0.0.0-20210813160813-60bc85c4be6d/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.21.0 h1:AQyQV4dYCvJ7vGmJyKki9+PBdyvhkSd8EIx/qb0AYv4= golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/oauth2 v0.0.0-20170912212905-13449ad91cb2/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20170912212905-13449ad91cb2/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181017192945-9dcd33a902f4/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20181017192945-9dcd33a902f4/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@@ -2007,13 +2007,13 @@ golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y= golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.17.0 h1:mkTF7LCd6WGJNL3K1Ad7kwxNfYAW6a8a8QqtMblp/4U= golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk= golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

View File

@@ -1010,7 +1010,11 @@ func (s *BlockChainAPI) GetBlockReceipts(ctx context.Context, blockNrOrHash rpc.
return result, nil return result, nil
} }
func (s *BlockChainAPI) GetBlobSidecars(ctx context.Context, blockNrOrHash rpc.BlockNumberOrHash) ([]map[string]interface{}, error) { func (s *BlockChainAPI) GetBlobSidecars(ctx context.Context, blockNrOrHash rpc.BlockNumberOrHash, fullBlob *bool) ([]map[string]interface{}, error) {
showBlob := true
if fullBlob != nil {
showBlob = *fullBlob
}
header, err := s.b.HeaderByNumberOrHash(ctx, blockNrOrHash) header, err := s.b.HeaderByNumberOrHash(ctx, blockNrOrHash)
if header == nil || err != nil { if header == nil || err != nil {
// When the block doesn't exist, the RPC method should return JSON null // When the block doesn't exist, the RPC method should return JSON null
@@ -1023,12 +1027,16 @@ func (s *BlockChainAPI) GetBlobSidecars(ctx context.Context, blockNrOrHash rpc.B
} }
result := make([]map[string]interface{}, len(blobSidecars)) result := make([]map[string]interface{}, len(blobSidecars))
for i, sidecar := range blobSidecars { for i, sidecar := range blobSidecars {
result[i] = marshalBlobSidecar(sidecar) result[i] = marshalBlobSidecar(sidecar, showBlob)
} }
return result, nil return result, nil
} }
func (s *BlockChainAPI) GetBlobSidecarByTxHash(ctx context.Context, hash common.Hash) (map[string]interface{}, error) { func (s *BlockChainAPI) GetBlobSidecarByTxHash(ctx context.Context, hash common.Hash, fullBlob *bool) (map[string]interface{}, error) {
showBlob := true
if fullBlob != nil {
showBlob = *fullBlob
}
txTarget, blockHash, _, Index := rawdb.ReadTransaction(s.b.ChainDb(), hash) txTarget, blockHash, _, Index := rawdb.ReadTransaction(s.b.ChainDb(), hash)
if txTarget == nil { if txTarget == nil {
return nil, nil return nil, nil
@@ -1045,7 +1053,7 @@ func (s *BlockChainAPI) GetBlobSidecarByTxHash(ctx context.Context, hash common.
} }
for _, sidecar := range blobSidecars { for _, sidecar := range blobSidecars {
if sidecar.TxIndex == Index { if sidecar.TxIndex == Index {
return marshalBlobSidecar(sidecar), nil return marshalBlobSidecar(sidecar, showBlob), nil
} }
} }
@@ -2142,13 +2150,31 @@ func marshalReceipt(receipt *types.Receipt, blockHash common.Hash, blockNumber u
return fields return fields
} }
func marshalBlobSidecar(sidecar *types.BlobSidecar) map[string]interface{} { func marshalBlobSidecar(sidecar *types.BlobSidecar, fullBlob bool) map[string]interface{} {
fields := map[string]interface{}{ fields := map[string]interface{}{
"blockHash": sidecar.BlockHash, "blockHash": sidecar.BlockHash,
"blockNumber": hexutil.EncodeUint64(sidecar.BlockNumber.Uint64()), "blockNumber": hexutil.EncodeUint64(sidecar.BlockNumber.Uint64()),
"txHash": sidecar.TxHash, "txHash": sidecar.TxHash,
"txIndex": hexutil.EncodeUint64(sidecar.TxIndex), "txIndex": hexutil.EncodeUint64(sidecar.TxIndex),
"blobSidecar": sidecar.BlobTxSidecar, }
fields["blobSidecar"] = marshalBlob(sidecar.BlobTxSidecar, fullBlob)
return fields
}
func marshalBlob(blobTxSidecar types.BlobTxSidecar, fullBlob bool) map[string]interface{} {
fields := map[string]interface{}{
"blobs": blobTxSidecar.Blobs,
"commitments": blobTxSidecar.Commitments,
"proofs": blobTxSidecar.Proofs,
}
if !fullBlob {
var blobs []common.Hash
for _, blob := range blobTxSidecar.Blobs {
var value common.Hash
copy(value[:], blob[:32])
blobs = append(blobs, value)
}
fields["blobs"] = blobs
} }
return fields return fields
} }

View File

@@ -87,6 +87,10 @@ func (m *MevAPI) Params() *types.MevParams {
return m.b.MevParams() return m.b.MevParams()
} }
func (m *MevAPI) HasBuilder(builder common.Address) bool {
return m.b.HasBuilder(builder)
}
// Running returns true if mev is running // Running returns true if mev is running
func (m *MevAPI) Running() bool { func (m *MevAPI) Running() bool {
return m.b.MevRunning() return m.b.MevRunning()

View File

@@ -650,7 +650,8 @@ func (b testBackend) ServiceFilter(ctx context.Context, session *bloombits.Match
panic("implement me") panic("implement me")
} }
func (b *testBackend) MevRunning() bool { return false } func (b *testBackend) MevRunning() bool { return false }
func (b *testBackend) HasBuilder(builder common.Address) bool { return false }
func (b *testBackend) MevParams() *types.MevParams { func (b *testBackend) MevParams() *types.MevParams {
return &types.MevParams{} return &types.MevParams{}
} }
@@ -2139,48 +2140,63 @@ func TestRPCGetBlobSidecars(t *testing.T) {
} }
var testSuite = []struct { var testSuite = []struct {
test rpc.BlockNumberOrHash test rpc.BlockNumberOrHash
file string fullBlob bool
file string
}{ }{
// 1. block without any txs(number) // 1. block without any txs(number)
{ {
test: rpc.BlockNumberOrHashWithNumber(0), test: rpc.BlockNumberOrHashWithNumber(0),
file: "number-1", fullBlob: true,
file: "number-1",
}, },
// 2. earliest tag // 2. earliest tag
{ {
test: rpc.BlockNumberOrHashWithNumber(rpc.EarliestBlockNumber), test: rpc.BlockNumberOrHashWithNumber(rpc.EarliestBlockNumber),
file: "tag-earliest", fullBlob: true,
file: "tag-earliest",
}, },
// 3. latest tag // 3. latest tag
{ {
test: rpc.BlockNumberOrHashWithNumber(rpc.LatestBlockNumber), test: rpc.BlockNumberOrHashWithNumber(rpc.LatestBlockNumber),
file: "tag-latest", fullBlob: true,
file: "tag-latest",
}, },
// 4. block is empty // 4. block is empty
{ {
test: rpc.BlockNumberOrHashWithHash(common.Hash{}, false), test: rpc.BlockNumberOrHashWithHash(common.Hash{}, false),
file: "hash-empty", fullBlob: true,
file: "hash-empty",
}, },
// 5. block is not found // 5. block is not found
{ {
test: rpc.BlockNumberOrHashWithHash(common.HexToHash("deadbeef"), false), test: rpc.BlockNumberOrHashWithHash(common.HexToHash("deadbeef"), false),
file: "hash-notfound", fullBlob: true,
file: "hash-notfound",
}, },
// 6. block is not found // 6. block is not found
{ {
test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(genBlocks + 1)), test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(genBlocks + 1)),
file: "block-notfound", fullBlob: true,
file: "block-notfound",
}, },
// 7. block with blob tx // 7. block with blob tx
{ {
test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(6)), test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(6)),
file: "block-with-blob-tx", fullBlob: true,
file: "block-with-blob-tx",
}, },
// 8. block with sidecar // 8. block with sidecar
{ {
test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(7)), test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(7)),
file: "block-with-blobSidecars", fullBlob: true,
file: "block-with-blobSidecars",
},
// 9. block with sidecar but show little
{
test: rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(7)),
fullBlob: false,
file: "block-with-blobSidecars-show-little",
}, },
} }
@@ -2189,7 +2205,7 @@ func TestRPCGetBlobSidecars(t *testing.T) {
result interface{} result interface{}
err error err error
) )
result, err = api.GetBlobSidecars(context.Background(), tt.test) result, err = api.GetBlobSidecars(context.Background(), tt.test, &tt.fullBlob)
if err != nil { if err != nil {
t.Errorf("test %d: want no error, have %v", i, err) t.Errorf("test %d: want no error, have %v", i, err)
continue continue
@@ -2205,33 +2221,45 @@ func TestGetBlobSidecarByTxHash(t *testing.T) {
api = NewBlockChainAPI(backend) api = NewBlockChainAPI(backend)
) )
var testSuite = []struct { var testSuite = []struct {
test common.Hash test common.Hash
file string fullBlob bool
file string
}{ }{
// 0. txHash is empty // 0. txHash is empty
{ {
test: common.Hash{}, test: common.Hash{},
file: "hash-empty", fullBlob: true,
file: "hash-empty",
}, },
// 1. txHash is not found // 1. txHash is not found
{ {
test: common.HexToHash("deadbeef"), test: common.HexToHash("deadbeef"),
file: "hash-notfound", fullBlob: true,
file: "hash-notfound",
}, },
// 2. txHash is not blob tx // 2. txHash is not blob tx
{ {
test: common.HexToHash("deadbeef"), test: common.HexToHash("deadbeef"),
file: "not-blob-tx", fullBlob: true,
file: "not-blob-tx",
}, },
// 3. block with blob tx without sidecar // 3. block with blob tx without sidecar
{ {
test: txHashs[5], test: txHashs[5],
file: "block-with-blob-tx", fullBlob: true,
file: "block-with-blob-tx",
}, },
// 4. block with sidecar // 4. block with sidecar
{ {
test: txHashs[6], test: txHashs[6],
file: "block-with-blobSidecars", fullBlob: true,
file: "block-with-blobSidecars",
},
// 5. block show part blobs
{
test: txHashs[6],
fullBlob: false,
file: "block-with-blobSidecars-show-little",
}, },
} }
@@ -2240,7 +2268,7 @@ func TestGetBlobSidecarByTxHash(t *testing.T) {
result interface{} result interface{}
err error err error
) )
result, err = api.GetBlobSidecarByTxHash(context.Background(), tt.test) result, err = api.GetBlobSidecarByTxHash(context.Background(), tt.test, &tt.fullBlob)
if err != nil { if err != nil {
t.Errorf("test %d: want no error, have %v", i, err) t.Errorf("test %d: want no error, have %v", i, err)
continue continue

View File

@@ -115,6 +115,8 @@ type Backend interface {
AddBuilder(builder common.Address, builderUrl string) error AddBuilder(builder common.Address, builderUrl string) error
// RemoveBuilder removes a builder from the bid simulator. // RemoveBuilder removes a builder from the bid simulator.
RemoveBuilder(builder common.Address) error RemoveBuilder(builder common.Address) error
// HasBuilder returns true if the builder is in the builder list.
HasBuilder(builder common.Address) bool
// SendBid receives bid from the builders. // SendBid receives bid from the builders.
SendBid(ctx context.Context, bid *types.BidArgs) (common.Hash, error) SendBid(ctx context.Context, bid *types.BidArgs) (common.Hash, error)
// BestBidGasFee returns the gas fee of the best bid for the given parent hash. // BestBidGasFee returns the gas fee of the best bid for the given parent hash.

View File

@@ -0,0 +1,17 @@
{
"blobSidecar": {
"blobs": [
"0x0000000000000000000000000000000000000000000000000000000000000000"
],
"commitments": [
"0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
],
"proofs": [
"0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
]
},
"blockHash": "0x6fe5e7eea22c44f700c95da066d6e0740894b6c1e993825cc63f634ad4f74250",
"blockNumber": "0x7",
"txHash": "0xc520427e696154779f6b21ab03a0735769e1c029035a484f5876f60383a0a7ce",
"txIndex": "0x0"
}

View File

@@ -0,0 +1,19 @@
[
{
"blobSidecar": {
"blobs": [
"0x0000000000000000000000000000000000000000000000000000000000000000"
],
"commitments": [
"0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
],
"proofs": [
"0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
]
},
"blockHash": "0x6fe5e7eea22c44f700c95da066d6e0740894b6c1e993825cc63f634ad4f74250",
"blockNumber": "0x7",
"txHash": "0xc520427e696154779f6b21ab03a0735769e1c029035a484f5876f60383a0a7ce",
"txIndex": "0x0"
}
]

View File

@@ -204,7 +204,8 @@ func (args *TransactionArgs) setFeeDefaults(ctx context.Context, b Backend) erro
// Sanity check the EIP-1559 fee parameters if present. // Sanity check the EIP-1559 fee parameters if present.
if args.GasPrice == nil && eip1559ParamsSet { if args.GasPrice == nil && eip1559ParamsSet {
if args.MaxFeePerGas.ToInt().Sign() == 0 { if args.MaxFeePerGas.ToInt().Sign() == 0 {
return errors.New("maxFeePerGas must be non-zero") // return errors.New("maxFeePerGas must be non-zero")
log.Warn("EIP-1559 Tx with zero maxFeePerGas") // BSC accepts zero gas price.
} }
if args.MaxFeePerGas.ToInt().Cmp(args.MaxPriorityFeePerGas.ToInt()) < 0 { if args.MaxFeePerGas.ToInt().Cmp(args.MaxPriorityFeePerGas.ToInt()) < 0 {
return fmt.Errorf("maxFeePerGas (%v) < maxPriorityFeePerGas (%v)", args.MaxFeePerGas, args.MaxPriorityFeePerGas) return fmt.Errorf("maxFeePerGas (%v) < maxPriorityFeePerGas (%v)", args.MaxFeePerGas, args.MaxPriorityFeePerGas)
@@ -217,7 +218,8 @@ func (args *TransactionArgs) setFeeDefaults(ctx context.Context, b Backend) erro
if args.GasPrice != nil && !eip1559ParamsSet { if args.GasPrice != nil && !eip1559ParamsSet {
// Zero gas-price is not allowed after London fork // Zero gas-price is not allowed after London fork
if args.GasPrice.ToInt().Sign() == 0 && isLondon { if args.GasPrice.ToInt().Sign() == 0 && isLondon {
return errors.New("gasPrice must be non-zero after london fork") // return errors.New("gasPrice must be non-zero after london fork")
log.Warn("non EIP-1559 Tx with zero gasPrice") // BSC accepts zero gas price.
} }
return nil // No need to set anything, user already set GasPrice return nil // No need to set anything, user already set GasPrice
} }

View File

@@ -85,8 +85,8 @@ func TestSetFeeDefaults(t *testing.T) {
"legacy tx post-London with zero price", "legacy tx post-London with zero price",
"london", "london",
&TransactionArgs{GasPrice: zero}, &TransactionArgs{GasPrice: zero},
nil, &TransactionArgs{GasPrice: zero},
errors.New("gasPrice must be non-zero after london fork"), nil, // errors.New("gasPrice must be non-zero after london fork"),
}, },
// Access list txs // Access list txs
@@ -180,8 +180,8 @@ func TestSetFeeDefaults(t *testing.T) {
"dynamic fee tx post-London, explicit gas price", "dynamic fee tx post-London, explicit gas price",
"london", "london",
&TransactionArgs{MaxFeePerGas: zero, MaxPriorityFeePerGas: zero}, &TransactionArgs{MaxFeePerGas: zero, MaxPriorityFeePerGas: zero},
nil, &TransactionArgs{MaxFeePerGas: zero, MaxPriorityFeePerGas: zero},
errors.New("maxFeePerGas must be non-zero"), nil, // errors.New("maxFeePerGas must be non-zero"),
}, },
// Misc // Misc
@@ -416,7 +416,8 @@ func (b *backendMock) SubscribeRemovedLogsEvent(ch chan<- core.RemovedLogsEvent)
func (b *backendMock) Engine() consensus.Engine { return nil } func (b *backendMock) Engine() consensus.Engine { return nil }
func (b *backendMock) MevRunning() bool { return false } func (b *backendMock) MevRunning() bool { return false }
func (b *backendMock) HasBuilder(builder common.Address) bool { return false }
func (b *backendMock) MevParams() *types.MevParams { func (b *backendMock) MevParams() *types.MevParams {
return &types.MevParams{} return &types.MevParams{}
} }

View File

@@ -617,12 +617,12 @@ web3._extend({
new web3._extend.Method({ new web3._extend.Method({
name: 'getBlobSidecars', name: 'getBlobSidecars',
call: 'eth_getBlobSidecars', call: 'eth_getBlobSidecars',
params: 1, params: 2,
}), }),
new web3._extend.Method({ new web3._extend.Method({
name: 'getBlobSidecarByTxHash', name: 'getBlobSidecarByTxHash',
call: 'eth_getBlobSidecarByTxHash', call: 'eth_getBlobSidecarByTxHash',
params: 1, params: 2,
}), }),
], ],
properties: [ properties: [
@@ -659,6 +659,30 @@ web3._extend({
name: 'stop', name: 'stop',
call: 'miner_stop' call: 'miner_stop'
}), }),
new web3._extend.Method({
name: 'mevRunning',
call: 'miner_mevRunning'
}),
new web3._extend.Method({
name: 'startMev',
call: 'miner_startMev'
}),
new web3._extend.Method({
name: 'stopMev',
call: 'miner_stopMev'
}),
new web3._extend.Method({
name: 'addBuilder',
call: 'miner_addBuilder',
params: 2,
inputFormatter: [web3._extend.formatters.inputAddressFormatter, null]
}),
new web3._extend.Method({
name: 'removeBuilder',
call: 'miner_removeBuilder',
params: 1,
inputFormatter: [web3._extend.formatters.inputAddressFormatter]
}),
new web3._extend.Method({ new web3._extend.Method({
name: 'setEtherbase', name: 'setEtherbase',
call: 'miner_setEtherbase', call: 'miner_setEtherbase',

View File

@@ -29,11 +29,6 @@ import (
const ( const (
// maxBidPerBuilderPerBlock is the max bid number per builder // maxBidPerBuilderPerBlock is the max bid number per builder
maxBidPerBuilderPerBlock = 3 maxBidPerBuilderPerBlock = 3
// leftOverTimeRate is the rate of left over time to simulate a bid
leftOverTimeRate = 11
// leftOverTimeScale is the scale of left over time to simulate a bid
leftOverTimeScale = 10
) )
var ( var (
@@ -78,6 +73,7 @@ type simBidReq struct {
type bidSimulator struct { type bidSimulator struct {
config *MevConfig config *MevConfig
delayLeftOver time.Duration delayLeftOver time.Duration
minGasPrice *big.Int
chain *core.BlockChain chain *core.BlockChain
chainConfig *params.ChainConfig chainConfig *params.ChainConfig
engine consensus.Engine engine consensus.Engine
@@ -114,6 +110,7 @@ type bidSimulator struct {
func newBidSimulator( func newBidSimulator(
config *MevConfig, config *MevConfig,
delayLeftOver time.Duration, delayLeftOver time.Duration,
minGasPrice *big.Int,
chain *core.BlockChain, chain *core.BlockChain,
chainConfig *params.ChainConfig, chainConfig *params.ChainConfig,
engine consensus.Engine, engine consensus.Engine,
@@ -122,6 +119,7 @@ func newBidSimulator(
b := &bidSimulator{ b := &bidSimulator{
config: config, config: config,
delayLeftOver: delayLeftOver, delayLeftOver: delayLeftOver,
minGasPrice: minGasPrice,
chain: chain, chain: chain,
chainConfig: chainConfig, chainConfig: chainConfig,
engine: engine, engine: engine,
@@ -315,18 +313,6 @@ func (b *bidSimulator) newBidLoop() {
// commit aborts in-flight bid execution with given signal and resubmits a new one. // commit aborts in-flight bid execution with given signal and resubmits a new one.
commit := func(reason int32, bidRuntime *BidRuntime) { commit := func(reason int32, bidRuntime *BidRuntime) {
// if the left time is not enough to do simulation, return
var simDuration time.Duration
if lastBid := b.GetBestBid(bidRuntime.bid.ParentHash); lastBid != nil && lastBid.duration != 0 {
simDuration = lastBid.duration
}
if time.Until(b.bidMustBefore(bidRuntime.bid.ParentHash)) <= simDuration*leftOverTimeRate/leftOverTimeScale {
log.Debug("BidSimulator: abort commit, not enough time to simulate",
"builder", bidRuntime.bid.Builder, "bidHash", bidRuntime.bid.Hash().Hex())
return
}
if interruptCh != nil { if interruptCh != nil {
// each commit work will have its own interruptCh to stop work with a reason // each commit work will have its own interruptCh to stop work with a reason
interruptCh <- reason interruptCh <- reason
@@ -367,6 +353,7 @@ func (b *bidSimulator) newBidLoop() {
expectedValidatorReward: expectedValidatorReward, expectedValidatorReward: expectedValidatorReward,
packedBlockReward: big.NewInt(0), packedBlockReward: big.NewInt(0),
packedValidatorReward: big.NewInt(0), packedValidatorReward: big.NewInt(0),
finished: make(chan struct{}),
} }
simulatingBid := b.GetSimulatingBid(newBid.ParentHash) simulatingBid := b.GetSimulatingBid(newBid.ParentHash)
@@ -407,11 +394,6 @@ func (b *bidSimulator) newBidLoop() {
} }
} }
func (b *bidSimulator) bidMustBefore(parentHash common.Hash) time.Time {
parentHeader := b.chain.GetHeaderByHash(parentHash)
return bidutil.BidMustBefore(parentHeader, b.chainConfig.Parlia.Period, b.delayLeftOver)
}
func (b *bidSimulator) bidBetterBefore(parentHash common.Hash) time.Time { func (b *bidSimulator) bidBetterBefore(parentHash common.Hash) time.Time {
parentHeader := b.chain.GetHeaderByHash(parentHash) parentHeader := b.chain.GetHeaderByHash(parentHash)
return bidutil.BidBetterBefore(parentHeader, b.chainConfig.Parlia.Period, b.delayLeftOver, b.config.BidSimulationLeftOver) return bidutil.BidBetterBefore(parentHeader, b.chainConfig.Parlia.Period, b.delayLeftOver, b.config.BidSimulationLeftOver)
@@ -527,7 +509,6 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
// ensure simulation exited then start next simulation // ensure simulation exited then start next simulation
b.SetSimulatingBid(parentHash, bidRuntime) b.SetSimulatingBid(parentHash, bidRuntime)
start := time.Now()
defer func(simStart time.Time) { defer func(simStart time.Time) {
logCtx := []any{ logCtx := []any{
@@ -553,10 +534,11 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
} }
b.RemoveSimulatingBid(parentHash) b.RemoveSimulatingBid(parentHash)
bidSimTimer.UpdateSince(start) close(bidRuntime.finished)
if success { if success {
bidRuntime.duration = time.Since(simStart) bidRuntime.duration = time.Since(simStart)
bidSimTimer.UpdateSince(simStart)
// only recommit self bid when newBidCh is empty // only recommit self bid when newBidCh is empty
if len(b.newBidCh) > 0 { if len(b.newBidCh) > 0 {
@@ -580,6 +562,14 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
return return
} }
// if the left time is not enough to do simulation, return
delay := b.engine.Delay(b.chain, bidRuntime.env.header, &b.delayLeftOver)
if delay == nil || *delay <= 0 {
log.Info("BidSimulator: abort commit, not enough time to simulate",
"builder", bidRuntime.bid.Builder, "bidHash", bidRuntime.bid.Hash().Hex())
return
}
gasLimit := bidRuntime.env.header.GasLimit gasLimit := bidRuntime.env.header.GasLimit
if bidRuntime.env.gasPool == nil { if bidRuntime.env.gasPool == nil {
bidRuntime.env.gasPool = new(core.GasPool).AddGas(gasLimit) bidRuntime.env.gasPool = new(core.GasPool).AddGas(gasLimit)
@@ -592,6 +582,7 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
return return
} }
// commit transactions in bid
for _, tx := range bidRuntime.bid.Txs { for _, tx := range bidRuntime.bid.Txs {
select { select {
case <-interruptCh: case <-interruptCh:
@@ -609,7 +600,7 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
break break
} }
err = bidRuntime.commitTransaction(b.chain, b.chainConfig, tx) err = bidRuntime.commitTransaction(b.chain, b.chainConfig, tx, bidRuntime.bid.UnRevertible.Contains(tx.Hash()))
if err != nil { if err != nil {
log.Error("BidSimulator: failed to commit tx", "bidHash", bidRuntime.bid.Hash(), "tx", tx.Hash(), "err", err) log.Error("BidSimulator: failed to commit tx", "bidHash", bidRuntime.bid.Hash(), "tx", tx.Hash(), "err", err)
err = fmt.Errorf("invalid tx in bid, %v", err) err = fmt.Errorf("invalid tx in bid, %v", err)
@@ -617,26 +608,41 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
} }
} }
bidRuntime.packReward(b.config.ValidatorCommission) // check if bid reward is valid
{
// return if bid is invalid, reportIssue issue to mev-sentry/builder if simulation is fully done bidRuntime.packReward(b.config.ValidatorCommission)
if !bidRuntime.validReward() { if !bidRuntime.validReward() {
err = errors.New("reward does not achieve the expectation") err = errors.New("reward does not achieve the expectation")
return return
}
} }
// fill transactions from mempool // check if bid gas price is lower than min gas price
{
bidGasUsed := uint64(0)
bidGasFee := bidRuntime.env.state.GetBalance(consensus.SystemAddress)
for _, receipt := range bidRuntime.env.receipts {
bidGasUsed += receipt.GasUsed
}
bidGasPrice := new(big.Int).Div(bidGasFee.ToBig(), new(big.Int).SetUint64(bidGasUsed))
if bidGasPrice.Cmp(b.minGasPrice) < 0 {
err = errors.New("bid gas price is lower than min gas price")
return
}
}
// if enable greedy merge, fill bid env with transactions from mempool
if b.config.GreedyMergeTx { if b.config.GreedyMergeTx {
delay := b.engine.Delay(b.chain, bidRuntime.env.header, &b.delayLeftOver) delay := b.engine.Delay(b.chain, bidRuntime.env.header, &b.delayLeftOver)
if delay != nil && *delay > 0 { if delay != nil && *delay > 0 {
stopTimer := time.NewTimer(*delay)
bidTxsSet := mapset.NewSet[common.Hash]() bidTxsSet := mapset.NewSet[common.Hash]()
for _, tx := range bidRuntime.bid.Txs { for _, tx := range bidRuntime.bid.Txs {
bidTxsSet.Add(tx.Hash()) bidTxsSet.Add(tx.Hash())
} }
fillErr := b.bidWorker.fillTransactions(interruptCh, bidRuntime.env, stopTimer, bidTxsSet) fillErr := b.bidWorker.fillTransactions(interruptCh, bidRuntime.env, nil, bidTxsSet)
log.Trace("BidSimulator: greedy merge stopped", "block", bidRuntime.env.header.Number, log.Trace("BidSimulator: greedy merge stopped", "block", bidRuntime.env.header.Number,
"builder", bidRuntime.bid.Builder, "tx count", bidRuntime.env.tcount-bidTxLen+1, "err", fillErr) "builder", bidRuntime.bid.Builder, "tx count", bidRuntime.env.tcount-bidTxLen+1, "err", fillErr)
@@ -645,8 +651,9 @@ func (b *bidSimulator) simBid(interruptCh chan int32, bidRuntime *BidRuntime) {
} }
} }
// commit payBidTx at the end of the block
bidRuntime.env.gasPool.AddGas(params.PayBidTxGasLimit) bidRuntime.env.gasPool.AddGas(params.PayBidTxGasLimit)
err = bidRuntime.commitTransaction(b.chain, b.chainConfig, payBidTx) err = bidRuntime.commitTransaction(b.chain, b.chainConfig, payBidTx, true)
if err != nil { if err != nil {
log.Error("BidSimulator: failed to commit tx", "builder", bidRuntime.bid.Builder, log.Error("BidSimulator: failed to commit tx", "builder", bidRuntime.bid.Builder,
"bidHash", bidRuntime.bid.Hash(), "tx", payBidTx.Hash(), "err", err) "bidHash", bidRuntime.bid.Hash(), "tx", payBidTx.Hash(), "err", err)
@@ -711,6 +718,7 @@ type BidRuntime struct {
packedBlockReward *big.Int packedBlockReward *big.Int
packedValidatorReward *big.Int packedValidatorReward *big.Int
finished chan struct{}
duration time.Duration duration time.Duration
} }
@@ -727,12 +735,10 @@ func (r *BidRuntime) packReward(validatorCommission uint64) {
r.packedValidatorReward.Sub(r.packedValidatorReward, r.bid.BuilderFee) r.packedValidatorReward.Sub(r.packedValidatorReward, r.bid.BuilderFee)
} }
func (r *BidRuntime) commitTransaction(chain *core.BlockChain, chainConfig *params.ChainConfig, tx *types.Transaction) error { func (r *BidRuntime) commitTransaction(chain *core.BlockChain, chainConfig *params.ChainConfig, tx *types.Transaction, unRevertible bool) error {
var ( var (
env = r.env env = r.env
snap = env.state.Snapshot() sc *types.BlobSidecar
gp = env.gasPool.Gas()
sc *types.BlobSidecar
) )
// Start executing the transaction // Start executing the transaction
@@ -755,9 +761,9 @@ func (r *BidRuntime) commitTransaction(chain *core.BlockChain, chainConfig *para
receipt, err := core.ApplyTransaction(chainConfig, chain, &env.coinbase, env.gasPool, env.state, env.header, tx, receipt, err := core.ApplyTransaction(chainConfig, chain, &env.coinbase, env.gasPool, env.state, env.header, tx,
&env.header.GasUsed, *chain.GetVMConfig(), core.NewReceiptBloomGenerator()) &env.header.GasUsed, *chain.GetVMConfig(), core.NewReceiptBloomGenerator())
if err != nil { if err != nil {
env.state.RevertToSnapshot(snap)
env.gasPool.SetGas(gp)
return err return err
} else if unRevertible && receipt.Status == types.ReceiptStatusFailed {
return errors.New("no revertible transaction failed")
} }
if tx.Type() == types.BlobTxType { if tx.Type() == types.BlobTxType {

View File

@@ -102,7 +102,7 @@ func New(eth Backend, config *Config, chainConfig *params.ChainConfig, mux *even
worker: newWorker(config, chainConfig, engine, eth, mux, isLocalBlock, false), worker: newWorker(config, chainConfig, engine, eth, mux, isLocalBlock, false),
} }
miner.bidSimulator = newBidSimulator(&config.Mev, config.DelayLeftOver, eth.BlockChain(), chainConfig, engine, miner.worker) miner.bidSimulator = newBidSimulator(&config.Mev, config.DelayLeftOver, config.GasPrice, eth.BlockChain(), chainConfig, engine, miner.worker)
miner.worker.setBestBidFetcher(miner.bidSimulator) miner.worker.setBestBidFetcher(miner.bidSimulator)
miner.wg.Add(1) miner.wg.Add(1)
@@ -295,3 +295,7 @@ func (miner *Miner) SubscribePendingLogs(ch chan<- []*types.Log) event.Subscript
func (miner *Miner) BuildPayload(args *BuildPayloadArgs) (*Payload, error) { func (miner *Miner) BuildPayload(args *BuildPayloadArgs) (*Payload, error) {
return miner.worker.buildPayload(args) return miner.worker.buildPayload(args)
} }
func (miner *Miner) GasCeil() uint64 {
return miner.worker.getGasCeil()
}

View File

@@ -9,6 +9,7 @@ import (
"github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/log" "github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
) )
type BuilderConfig struct { type BuilderConfig struct {
@@ -59,6 +60,11 @@ func (miner *Miner) RemoveBuilder(builderAddr common.Address) error {
return miner.bidSimulator.RemoveBuilder(builderAddr) return miner.bidSimulator.RemoveBuilder(builderAddr)
} }
// HasBuilder returns true if the builder is in the builder list.
func (miner *Miner) HasBuilder(builder common.Address) bool {
return miner.bidSimulator.ExistBuilder(builder)
}
func (miner *Miner) SendBid(ctx context.Context, bidArgs *types.BidArgs) (common.Hash, error) { func (miner *Miner) SendBid(ctx context.Context, bidArgs *types.BidArgs) (common.Hash, error) {
builder, err := bidArgs.EcrecoverSender() builder, err := bidArgs.EcrecoverSender()
if err != nil { if err != nil {
@@ -117,6 +123,8 @@ func (miner *Miner) MevParams() *types.MevParams {
ValidatorCommission: miner.worker.config.Mev.ValidatorCommission, ValidatorCommission: miner.worker.config.Mev.ValidatorCommission,
BidSimulationLeftOver: miner.worker.config.Mev.BidSimulationLeftOver, BidSimulationLeftOver: miner.worker.config.Mev.BidSimulationLeftOver,
GasCeil: miner.worker.config.GasCeil, GasCeil: miner.worker.config.GasCeil,
GasPrice: miner.worker.config.GasPrice,
BuilderFeeCeil: builderFeeCeil, BuilderFeeCeil: builderFeeCeil,
Version: params.Version,
} }
} }

View File

@@ -67,6 +67,9 @@ const (
// the current 4 mining loops could have asynchronous risk of mining block with // the current 4 mining loops could have asynchronous risk of mining block with
// save height, keep recently mined blocks to avoid double sign for safety, // save height, keep recently mined blocks to avoid double sign for safety,
recentMinedCacheLimit = 20 recentMinedCacheLimit = 20
// the default to wait for the mev miner to finish
waitMEVMinerEndTimeLimit = 50 * time.Millisecond
) )
var ( var (
@@ -171,6 +174,7 @@ type getWorkReq struct {
type bidFetcher interface { type bidFetcher interface {
GetBestBid(parentHash common.Hash) *BidRuntime GetBestBid(parentHash common.Hash) *BidRuntime
GetSimulatingBid(prevBlockHash common.Hash) *BidRuntime
} }
// worker is the main object which takes care of submitting new work to consensus engine // worker is the main object which takes care of submitting new work to consensus engine
@@ -329,6 +333,12 @@ func (w *worker) setGasCeil(ceil uint64) {
w.config.GasCeil = ceil w.config.GasCeil = ceil
} }
func (w *worker) getGasCeil() uint64 {
w.mu.Lock()
defer w.mu.Unlock()
return w.config.GasCeil
}
// setExtra sets the content used to initialize the block extra field. // setExtra sets the content used to initialize the block extra field.
func (w *worker) setExtra(extra []byte) { func (w *worker) setExtra(extra []byte) {
w.mu.Lock() w.mu.Lock()
@@ -1330,6 +1340,15 @@ LOOP:
// when in-turn, compare with remote work. // when in-turn, compare with remote work.
from := bestWork.coinbase from := bestWork.coinbase
if w.bidFetcher != nil && bestWork.header.Difficulty.Cmp(diffInTurn) == 0 { if w.bidFetcher != nil && bestWork.header.Difficulty.Cmp(diffInTurn) == 0 {
if pendingBid := w.bidFetcher.GetSimulatingBid(bestWork.header.ParentHash); pendingBid != nil {
waitBidTimer := time.NewTimer(waitMEVMinerEndTimeLimit)
defer waitBidTimer.Stop()
select {
case <-waitBidTimer.C:
case <-pendingBid.finished:
}
}
bestBid := w.bidFetcher.GetBestBid(bestWork.header.ParentHash) bestBid := w.bidFetcher.GetBestBid(bestWork.header.ParentHash)
if bestBid != nil { if bestBid != nil {

View File

@@ -773,12 +773,13 @@ func (n *Node) OpenDatabase(name string, cache, handles int, namespace string, r
db = rawdb.NewMemoryDatabase() db = rawdb.NewMemoryDatabase()
} else { } else {
db, err = rawdb.Open(rawdb.OpenOptions{ db, err = rawdb.Open(rawdb.OpenOptions{
Type: n.config.DBEngine, Type: n.config.DBEngine,
Directory: n.ResolvePath(name), Directory: n.ResolvePath(name),
Namespace: namespace, Namespace: namespace,
Cache: cache, Cache: cache,
Handles: handles, Handles: handles,
ReadOnly: readonly, ReadOnly: readonly,
MultiDataBase: n.CheckIfMultiDataBase(),
}) })
} }

View File

@@ -147,13 +147,13 @@ var (
LondonBlock: big.NewInt(31302048), LondonBlock: big.NewInt(31302048),
HertzBlock: big.NewInt(31302048), HertzBlock: big.NewInt(31302048),
HertzfixBlock: big.NewInt(34140700), HertzfixBlock: big.NewInt(34140700),
// UnixTime: 1705996800 is January 23, 2024 8:00:00 AM UTC ShanghaiTime: newUint64(1705996800), // 2024-01-23 08:00:00 AM UTC
ShanghaiTime: newUint64(1705996800), KeplerTime: newUint64(1705996800), // 2024-01-23 08:00:00 AM UTC
KeplerTime: newUint64(1705996800), FeynmanTime: newUint64(1713419340), // 2024-04-18 05:49:00 AM UTC
FeynmanTime: newUint64(1713419340), FeynmanFixTime: newUint64(1713419340), // 2024-04-18 05:49:00 AM UTC
FeynmanFixTime: newUint64(1713419340), CancunTime: newUint64(1718863500), // 2024-06-20 06:05:00 AM UTC
// TODO(GalaIO): enable cancun fork time later HaberTime: newUint64(1718863500), // 2024-06-20 06:05:00 AM UTC
//CancunTime: newUint64(), BohrTime: nil,
Parlia: &ParliaConfig{ Parlia: &ParliaConfig{
Period: 3, Period: 3,
@@ -192,6 +192,8 @@ var (
FeynmanTime: newUint64(1710136800), // 2024-03-11 6:00:00 AM UTC FeynmanTime: newUint64(1710136800), // 2024-03-11 6:00:00 AM UTC
FeynmanFixTime: newUint64(1711342800), // 2024-03-25 5:00:00 AM UTC FeynmanFixTime: newUint64(1711342800), // 2024-03-25 5:00:00 AM UTC
CancunTime: newUint64(1713330442), // 2024-04-17 05:07:22 AM UTC CancunTime: newUint64(1713330442), // 2024-04-17 05:07:22 AM UTC
HaberTime: newUint64(1716962820), // 2024-05-29 06:07:00 AM UTC
BohrTime: nil,
Parlia: &ParliaConfig{ Parlia: &ParliaConfig{
Period: 3, Period: 3,
@@ -231,10 +233,13 @@ var (
FeynmanTime: newUint64(0), FeynmanTime: newUint64(0),
FeynmanFixTime: newUint64(0), FeynmanFixTime: newUint64(0),
CancunTime: newUint64(0), CancunTime: newUint64(0),
HaberTime: newUint64(0),
HaberFixTime: _rialto_upgrade_height_,
BohrTime: _rialto_upgrade_height_,
Parlia: &ParliaConfig{ Parlia: &ParliaConfig{
Period: 3, Period: _rialto_parlia_period_,
Epoch: 200, Epoch: _rialto_parlia_period_,
}, },
} }
@@ -269,6 +274,7 @@ var (
FeynmanTime: newUint64(0), FeynmanTime: newUint64(0),
FeynmanFixTime: newUint64(0), FeynmanFixTime: newUint64(0),
CancunTime: newUint64(0), CancunTime: newUint64(0),
Parlia: &ParliaConfig{ Parlia: &ParliaConfig{
Period: 3, Period: 3,
Epoch: 200, Epoch: 200,
@@ -506,6 +512,9 @@ type ChainConfig struct {
FeynmanTime *uint64 `json:"feynmanTime,omitempty"` // Feynman switch time (nil = no fork, 0 = already activated) FeynmanTime *uint64 `json:"feynmanTime,omitempty"` // Feynman switch time (nil = no fork, 0 = already activated)
FeynmanFixTime *uint64 `json:"feynmanFixTime,omitempty"` // FeynmanFix switch time (nil = no fork, 0 = already activated) FeynmanFixTime *uint64 `json:"feynmanFixTime,omitempty"` // FeynmanFix switch time (nil = no fork, 0 = already activated)
CancunTime *uint64 `json:"cancunTime,omitempty"` // Cancun switch time (nil = no fork, 0 = already on cancun) CancunTime *uint64 `json:"cancunTime,omitempty"` // Cancun switch time (nil = no fork, 0 = already on cancun)
HaberTime *uint64 `json:"haberTime,omitempty"` // Haber switch time (nil = no fork, 0 = already on haber)
HaberFixTime *uint64 `json:"haberFixTime,omitempty"` // HaberFix switch time (nil = no fork, 0 = already on haberFix)
BohrTime *uint64 `json:"bohrTime,omitempty"` // Bohr switch time (nil = no fork, 0 = already on bohr)
PragueTime *uint64 `json:"pragueTime,omitempty"` // Prague switch time (nil = no fork, 0 = already on prague) PragueTime *uint64 `json:"pragueTime,omitempty"` // Prague switch time (nil = no fork, 0 = already on prague)
VerkleTime *uint64 `json:"verkleTime,omitempty"` // Verkle switch time (nil = no fork, 0 = already on verkle) VerkleTime *uint64 `json:"verkleTime,omitempty"` // Verkle switch time (nil = no fork, 0 = already on verkle)
@@ -611,7 +620,22 @@ func (c *ChainConfig) String() string {
CancunTime = big.NewInt(0).SetUint64(*c.CancunTime) CancunTime = big.NewInt(0).SetUint64(*c.CancunTime)
} }
return fmt.Sprintf("{ChainID: %v Homestead: %v DAO: %v DAOSupport: %v EIP150: %v EIP155: %v EIP158: %v Byzantium: %v Constantinople: %v Petersburg: %v Istanbul: %v, Muir Glacier: %v, Ramanujan: %v, Niels: %v, MirrorSync: %v, Bruno: %v, Berlin: %v, YOLO v3: %v, CatalystBlock: %v, London: %v, ArrowGlacier: %v, MergeFork:%v, Euler: %v, Gibbs: %v, Nano: %v, Moran: %v, Planck: %v,Luban: %v, Plato: %v, Hertz: %v, Hertzfix: %v, ShanghaiTime: %v, KeplerTime: %v, FeynmanTime: %v, FeynmanFixTime: %v, CancunTime: %v, Engine: %v}", var HaberTime *big.Int
if c.HaberTime != nil {
HaberTime = big.NewInt(0).SetUint64(*c.HaberTime)
}
var HaberFixTime *big.Int
if c.HaberFixTime != nil {
HaberFixTime = big.NewInt(0).SetUint64(*c.HaberFixTime)
}
var BohrTime *big.Int
if c.BohrTime != nil {
BohrTime = big.NewInt(0).SetUint64(*c.BohrTime)
}
return fmt.Sprintf("{ChainID: %v Homestead: %v DAO: %v DAOSupport: %v EIP150: %v EIP155: %v EIP158: %v Byzantium: %v Constantinople: %v Petersburg: %v Istanbul: %v, Muir Glacier: %v, Ramanujan: %v, Niels: %v, MirrorSync: %v, Bruno: %v, Berlin: %v, YOLO v3: %v, CatalystBlock: %v, London: %v, ArrowGlacier: %v, MergeFork:%v, Euler: %v, Gibbs: %v, Nano: %v, Moran: %v, Planck: %v,Luban: %v, Plato: %v, Hertz: %v, Hertzfix: %v, ShanghaiTime: %v, KeplerTime: %v, FeynmanTime: %v, FeynmanFixTime: %v, CancunTime: %v, HaberTime: %v, HaberFixTime: %v, BohrTime: %v, Engine: %v}",
c.ChainID, c.ChainID,
c.HomesteadBlock, c.HomesteadBlock,
c.DAOForkBlock, c.DAOForkBlock,
@@ -648,6 +672,9 @@ func (c *ChainConfig) String() string {
FeynmanTime, FeynmanTime,
FeynmanFixTime, FeynmanFixTime,
CancunTime, CancunTime,
HaberTime,
HaberFixTime,
BohrTime,
engine, engine,
) )
} }
@@ -915,6 +942,39 @@ func (c *ChainConfig) IsCancun(num *big.Int, time uint64) bool {
return c.IsLondon(num) && isTimestampForked(c.CancunTime, time) return c.IsLondon(num) && isTimestampForked(c.CancunTime, time)
} }
// IsHaber returns whether time is either equal to the Haber fork time or greater.
func (c *ChainConfig) IsHaber(num *big.Int, time uint64) bool {
return c.IsLondon(num) && isTimestampForked(c.HaberTime, time)
}
// IsHaberFix returns whether time is either equal to the HaberFix fork time or greater.
func (c *ChainConfig) IsHaberFix(num *big.Int, time uint64) bool {
return c.IsLondon(num) && isTimestampForked(c.HaberFixTime, time)
}
// IsOnHaberFix returns whether currentBlockTime is either equal to the HaberFix fork time or greater firstly.
func (c *ChainConfig) IsOnHaberFix(currentBlockNumber *big.Int, lastBlockTime uint64, currentBlockTime uint64) bool {
lastBlockNumber := new(big.Int)
if currentBlockNumber.Cmp(big.NewInt(1)) >= 0 {
lastBlockNumber.Sub(currentBlockNumber, big.NewInt(1))
}
return !c.IsHaberFix(lastBlockNumber, lastBlockTime) && c.IsHaberFix(currentBlockNumber, currentBlockTime)
}
// IsBohr returns whether time is either equal to the Bohr fork time or greater.
func (c *ChainConfig) IsBohr(num *big.Int, time uint64) bool {
return c.IsLondon(num) && isTimestampForked(c.BohrTime, time)
}
// IsOnBohr returns whether currentBlockTime is either equal to the Bohr fork time or greater firstly.
func (c *ChainConfig) IsOnBohr(currentBlockNumber *big.Int, lastBlockTime uint64, currentBlockTime uint64) bool {
lastBlockNumber := new(big.Int)
if currentBlockNumber.Cmp(big.NewInt(1)) >= 0 {
lastBlockNumber.Sub(currentBlockNumber, big.NewInt(1))
}
return !c.IsBohr(lastBlockNumber, lastBlockTime) && c.IsBohr(currentBlockNumber, currentBlockTime)
}
// IsPrague returns whether num is either equal to the Prague fork time or greater. // IsPrague returns whether num is either equal to the Prague fork time or greater.
func (c *ChainConfig) IsPrague(num *big.Int, time uint64) bool { func (c *ChainConfig) IsPrague(num *big.Int, time uint64) bool {
return c.IsLondon(num) && isTimestampForked(c.PragueTime, time) return c.IsLondon(num) && isTimestampForked(c.PragueTime, time)
@@ -978,6 +1038,9 @@ func (c *ChainConfig) CheckConfigForkOrder() error {
{name: "feynmanTime", timestamp: c.FeynmanTime}, {name: "feynmanTime", timestamp: c.FeynmanTime},
{name: "feynmanFixTime", timestamp: c.FeynmanFixTime}, {name: "feynmanFixTime", timestamp: c.FeynmanFixTime},
{name: "cancunTime", timestamp: c.CancunTime}, {name: "cancunTime", timestamp: c.CancunTime},
{name: "haberTime", timestamp: c.HaberTime},
{name: "haberFixTime", timestamp: c.HaberFixTime},
{name: "bohrTime", timestamp: c.BohrTime},
{name: "pragueTime", timestamp: c.PragueTime, optional: true}, {name: "pragueTime", timestamp: c.PragueTime, optional: true},
{name: "verkleTime", timestamp: c.VerkleTime, optional: true}, {name: "verkleTime", timestamp: c.VerkleTime, optional: true},
} { } {
@@ -1125,6 +1188,12 @@ func (c *ChainConfig) checkCompatible(newcfg *ChainConfig, headNumber *big.Int,
if isForkTimestampIncompatible(c.CancunTime, newcfg.CancunTime, headTimestamp) { if isForkTimestampIncompatible(c.CancunTime, newcfg.CancunTime, headTimestamp) {
return newTimestampCompatError("Cancun fork timestamp", c.CancunTime, newcfg.CancunTime) return newTimestampCompatError("Cancun fork timestamp", c.CancunTime, newcfg.CancunTime)
} }
if isForkTimestampIncompatible(c.HaberTime, newcfg.HaberTime, headTimestamp) {
return newTimestampCompatError("Haber fork timestamp", c.HaberTime, newcfg.HaberTime)
}
if isForkTimestampIncompatible(c.HaberFixTime, newcfg.HaberFixTime, headTimestamp) {
return newTimestampCompatError("HaberFix fork timestamp", c.HaberFixTime, newcfg.HaberFixTime)
}
if isForkTimestampIncompatible(c.PragueTime, newcfg.PragueTime, headTimestamp) { if isForkTimestampIncompatible(c.PragueTime, newcfg.PragueTime, headTimestamp) {
return newTimestampCompatError("Prague fork timestamp", c.PragueTime, newcfg.PragueTime) return newTimestampCompatError("Prague fork timestamp", c.PragueTime, newcfg.PragueTime)
} }
@@ -1307,8 +1376,8 @@ type Rules struct {
IsPlato bool IsPlato bool
IsHertz bool IsHertz bool
IsHertzfix bool IsHertzfix bool
IsShanghai, IsKepler, IsFeynman, IsCancun, IsPrague bool IsShanghai, IsKepler, IsFeynman, IsCancun, IsHaber bool
IsVerkle bool IsBohr, IsPrague, IsVerkle bool
} }
// Rules ensures c's ChainID is not nil. // Rules ensures c's ChainID is not nil.
@@ -1343,6 +1412,8 @@ func (c *ChainConfig) Rules(num *big.Int, isMerge bool, timestamp uint64) Rules
IsKepler: c.IsKepler(num, timestamp), IsKepler: c.IsKepler(num, timestamp),
IsFeynman: c.IsFeynman(num, timestamp), IsFeynman: c.IsFeynman(num, timestamp),
IsCancun: c.IsCancun(num, timestamp), IsCancun: c.IsCancun(num, timestamp),
IsHaber: c.IsHaber(num, timestamp),
IsBohr: c.IsBohr(num, timestamp),
IsPrague: c.IsPrague(num, timestamp), IsPrague: c.IsPrague(num, timestamp),
IsVerkle: c.IsVerkle(num, timestamp), IsVerkle: c.IsVerkle(num, timestamp),
} }

View File

@@ -170,6 +170,8 @@ const (
Bls12381MapG1Gas uint64 = 5500 // Gas price for BLS12-381 mapping field element to G1 operation Bls12381MapG1Gas uint64 = 5500 // Gas price for BLS12-381 mapping field element to G1 operation
Bls12381MapG2Gas uint64 = 110000 // Gas price for BLS12-381 mapping field element to G2 operation Bls12381MapG2Gas uint64 = 110000 // Gas price for BLS12-381 mapping field element to G2 operation
P256VerifyGas uint64 = 3450 // secp256r1 elliptic curve signature verifier gas price
// The Refund Quotient is the cap on how much of the used gas can be refunded. Before EIP-3529, // The Refund Quotient is the cap on how much of the used gas can be refunded. Before EIP-3529,
// up to half the consumed gas could be refunded. Redefined as 1/5th in EIP-3529 // up to half the consumed gas could be refunded. Redefined as 1/5th in EIP-3529
RefundQuotient uint64 = 2 RefundQuotient uint64 = 2
@@ -189,6 +191,7 @@ const (
var ( var (
MinBlocksForBlobRequests uint64 = 524288 // it keeps blob data available for ~18.2 days in local, ref: https://github.com/bnb-chain/BEPs/blob/master/BEPs/BEP-336.md#51-parameters. MinBlocksForBlobRequests uint64 = 524288 // it keeps blob data available for ~18.2 days in local, ref: https://github.com/bnb-chain/BEPs/blob/master/BEPs/BEP-336.md#51-parameters.
DefaultExtraReserveForBlobRequests uint64 = 1 * (24 * 3600) / 3 // it adds more time for expired blobs for some request cases, like expiry blob when remote peer is syncing, default 1 day. DefaultExtraReserveForBlobRequests uint64 = 1 * (24 * 3600) / 3 // it adds more time for expired blobs for some request cases, like expiry blob when remote peer is syncing, default 1 day.
BreatheBlockInterval uint64 = 86400 // Controls the interval for updateValidatorSetV2
) )
// Gas discount table for BLS12-381 G1 and G2 multi exponentiation operations // Gas discount table for BLS12-381 G1 and G2 multi exponentiation operations

Some files were not shown because too many files have changed in this diff Show More