Compare commits

..

93 Commits

Author SHA1 Message Date
emailtovamos
be71d41aa5 eth: formatting 2024-05-23 14:04:22 +01:00
emailtovamos
5b46fe13e7 eth: make transaction propagation paths in the network deterministic (#29034)
* eth: make transaction propagation paths in the network deterministic

* eth: avoid potential division by 0

* eth: make tx propagation dependent on local node id too

* eth: fix review comments
2024-05-23 14:00:29 +01:00
Satyajit Das
b0146261c7 jsutils: faucet successful requests within blocks (#2470) 2024-05-23 15:08:36 +08:00
lx
d7b9866d3b Merge pull request #2487 from bnb-chain/master
merge PRs from master to develop branch
2024-05-22 13:56:19 +08:00
zzzckck
f5ba30ed47 release: prepare for release v1.4.8 (#2486) 2024-05-21 18:30:28 +08:00
Nathan
f190c49252 core/vm: add secp256r1 into PrecompiledContractsHaber (#2483) 2024-05-21 17:46:42 +08:00
Mars
08769ead2b dev: ensure consistency in BPS bundle result (#2479)
* dev: ensure consistency in BPS bundle result

* fix: remove env operation once the sim is discarded & rename
2024-05-21 12:24:41 +08:00
Matus Kysel
4d0f1e7117 RIP-7212: Precompile for secp256r1 Curve Support (#2400) 2024-05-21 11:44:21 +08:00
irrun
c77bb1110d fix: limit the gas price of the mev bid (#2473) 2024-05-20 14:33:47 +08:00
Mars
c856d21719 fix: move mev op to MinerAPI & add command to console (#2475) 2024-05-20 14:00:28 +08:00
Nathan
f45305b1ad cmd/utils: add a flag to change breathe block interval for testing (#2472) 2024-05-17 16:18:29 +08:00
Eric
d16532d678 internal/ethapi: add optional parameter for blobSidecars (#2468) 2024-05-16 19:07:14 +08:00
Eric
5edd032cdb internal/ethapi: add optional parameter for blobSidecars (#2467) 2024-05-16 19:06:49 +08:00
galaio
6b8cbbe172 sync: fix some sync issues caused by prune-block. (#2466) 2024-05-16 12:07:13 +08:00
setunapo
5ea2ada0ee utils: add check_blobtx.js (#2463) 2024-05-15 18:17:57 +08:00
Fynn
b230a02006 cmd: fix memory leak when big dataset (#2455) 2024-05-15 15:28:57 +08:00
Nathan
86e3a02490 cmd/utils: add a flag to change breathe block interval for testing (#2462) 2024-05-15 15:27:05 +08:00
Nathan
0c0958ff87 eth/handler: check lists in body before broadcast blocks (#2461) 2024-05-15 14:54:25 +08:00
buddho
c577ce3720 Merge pull request #2460 from bnb-chain/develop
merge some PRs for v1.4.7(2nd)
2024-05-14 19:59:57 +08:00
Eric
d436f9e2e8 eth/fetcher/block_fetcher.go: add metrics for import failed block (#2459) 2024-05-14 19:29:43 +08:00
Eric
97c3b9b267 internal/api.go: add choice about not show full blob (#2458) 2024-05-14 17:17:58 +08:00
Nathan
0560685460 core: clearup feynman testflag and rialto code (#2457) 2024-05-14 16:37:17 +08:00
rjl493456442
bf16a39876 ethdb/pebble: print warning log if pebble performance degrades (#29478) 2024-05-14 15:26:21 +08:00
Devon Bear
2c8720016d go.mod: bump pebble db to official release (#29038)
bump pebble
2024-05-14 15:26:21 +08:00
Nathan
f2ec3cc6a5 eth/handler: check blobs before broadcast blocks (#2450) 2024-05-13 17:21:45 +08:00
Martin HS
0a2e1282d2 core/rawdb: add sanity-limit to header accessor (#29534) 2024-05-13 17:21:45 +08:00
Nathan
adb5e8fe86 eth/filters: enforce topic-limit early on filter criterias (#29535) (#2448)
This PR adds a limit of 1000 to the "inner" topics in a filter-criteria

Co-authored-by: Martin HS <martin@swende.se>
2024-05-13 17:21:45 +08:00
Nathan
23f6194fad eth/handler: check blobs before broadcast blocks (#2450) 2024-05-13 16:20:50 +08:00
Martin HS
691d195526 core/rawdb: add sanity-limit to header accessor (#29534) 2024-05-11 10:24:39 +08:00
Nathan
b57c779759 eth/filters: enforce topic-limit early on filter criterias (#29535) (#2448)
This PR adds a limit of 1000 to the "inner" topics in a filter-criteria

Co-authored-by: Martin HS <martin@swende.se>
2024-05-11 10:24:19 +08:00
zzzckck
4ab1c865b2 Merge pull request #2441 from bnb-chain/develop
Draft release v1.4.7
2024-05-10 13:10:56 +08:00
zzzckck
a7d5b02919 release: prepare for release v1.4.7 (#2442)
* release: prepare for release v1.4.7

* code: avoid golang-lint error for TxLookupLimit
2024-05-09 19:47:52 +08:00
zzzckck
1ce9bb044d config: setup Mainnet Tycho(Cancun) hardfork date (#2439)
expected hard fork date:
Mainnet: 2024-06-20 06:05:00 AM UTC
2024-05-09 16:24:04 +08:00
dependabot[bot]
7948950f7a build(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 (#2411) 2024-05-09 16:06:28 +08:00
knowmost
0c101e618a chore: fix some typos (#2433) 2024-05-09 16:05:46 +08:00
zzzckck
571ea2c4b9 nancy: add files .nancy-ignore (#2440) 2024-05-09 15:58:06 +08:00
galaio
7bc5a3353d txpool: limit max gas when mining is enabled; (#2435) 2024-05-09 15:54:31 +08:00
Chris Li
901ea2e0d2 fix: performance issue when load journal (#2438) 2024-05-08 15:36:01 +08:00
buddho
1d81f3316f metrics: add blockInsertMgaspsGauge to trace mgasps (#2396) 2024-04-30 17:58:04 +08:00
zzzckck
43b2ffa63b Merge pull request #2427 from bnb-chain/develop
Draft release v1.4.6
2024-04-29 14:07:29 +08:00
zzzckck
0567715760 release: prepare for release v1.4.6 2024-04-29 10:47:43 +08:00
zzzckck
e32fcf5b93 Revert "github: add branch protect rule (#2343)"
This reverts commit e7c5ce2e94.
2024-04-29 10:47:43 +08:00
zzzckck
e55028d788 metrics: refine the double sign detect code 2024-04-29 10:47:43 +08:00
Satyajit Das
9d8df917b8 metrics: add doublesign counter (#2419) 2024-04-29 09:42:43 +08:00
irrun
9e170972f4 fix: oom caused by non-discarded mev simulation env (#2430) 2024-04-28 19:45:42 +08:00
irrun
ba6726325a feat: recommit bid when newBidCh is empty to maximize mev reward (#2424) 2024-04-28 11:05:09 +08:00
Chris Li
6573254a62 fix: adapt journal for cmd (#2425) 2024-04-28 11:02:14 +08:00
galaio
31d92c50ad chore: add metric & log for blobTx; (#2428) 2024-04-27 06:37:42 +08:00
Eric
7cab9c622c eth/gasprice: add query limit to defend DDOS attack (#2423) 2024-04-25 14:05:12 +08:00
irrun
2a0e399c38 Revert "fix: wrong way to get blob tx sidecar in `BidRuntime.commitTransactio…" (#2418)
This reverts commit 14023fae6d.
2024-04-23 11:41:56 +08:00
forcedebug
182c841374 fix: fix function names (#2416) 2024-04-23 10:06:50 +08:00
Roshan
14023fae6d fix: wrong way to get blob tx sidecar in BidRuntime.commitTransaction (#2417) 2024-04-22 18:46:44 +08:00
Chris Li
d653cda82e feat: adaptive for loading journal file or journal kv during loadJournal (#2406)
* core: check journalType before load journal
* fix: when delete trieJournal delete from kv & file
2024-04-19 16:23:28 +08:00
careworry
4b54601d5c chore: fix some typos in comments (#2408) 2024-04-19 13:17:58 +08:00
Ng Wei Han
3b7f0e4279 cmd, p2p: filter peers by regex on name (#2404) 2024-04-18 16:11:32 +08:00
TechVest
fe1fff8c77 chore: fix some typos in comments (#2399) 2024-04-18 15:43:57 +08:00
Chris Li
c0afdc9a98 core: separated databases for block data (#2227)
* core: use finalized block as the chain freeze indicator (#28683)
* core/rawdb: use max(finality, head-90k) as chain freezing threshold
* core: impl multi database for block data
* core: fix db inspect total size bug
* core: add tips for user who use multi-database
* core: adapt some cmd for multi-database
* core: adapter blockBlobSidecars
* core: fix freezer readHeader bug

---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
2024-04-18 15:12:05 +08:00
buddho
fb435eb5f1 fix: allow fast node to rewind after abnormal shutdown (#2401) 2024-04-18 13:45:02 +08:00
irrun
5cc253a2cd fix: NPE (#2403) 2024-04-18 13:25:38 +08:00
buddho
cbcd26c9a9 fix: no import blocks before or equal to the finalized height (#2398) 2024-04-17 14:19:50 +08:00
Chris Li
90eb5b33e8 fix: trieJournal format compatible old db format (#2395) 2024-04-16 11:49:04 +08:00
Ng Wei Han
837de88057 cmd/geth: fix importBlock (#2244) 2024-04-15 16:40:03 +08:00
buddho
b4fb2f6ffc fix: print value instead of pointer in ConfigCompatError (#2391) 2024-04-15 14:46:14 +08:00
dylanhuang
11503edeb2 chore: render system bytecode by go:embed (#2201) 2024-04-15 11:23:23 +08:00
Chris Li
3a6e3c67f2 core/trie: persist TrieJournal to journal file instead of kv database (#2341) 2024-04-15 10:47:54 +08:00
hugehope
335be39905 chore: fix function names in comment (#2390) 2024-04-11 19:01:43 +08:00
irrun
b7972bcd77 feat: greedy merge tx in bid (#2363) 2024-04-11 15:15:46 +08:00
buddho
4bb1bd1a77 deps: update prsym to solve warning about quic-go version (#2389) 2024-04-11 14:11:15 +08:00
zzzckck
a05724588f Merge pull request #2388 from bnb-chain/develop
merge develop to master for several 4844 bug fix.
2024-04-10 21:33:55 +08:00
Eric
009df5a121 internal/api: return empty array rather than nil (#2385) 2024-04-10 20:13:05 +08:00
buddho
d7836bfe98 fix: set withdrawals properly when fetching empty block (#2386) 2024-04-10 19:46:34 +08:00
clonefetch
f4bad20447 docs: remove repetitive words (#2381) 2024-04-10 17:06:12 +08:00
buddho
a75e82367d fix: ensure empty withdrawals after cancun (#2384) 2024-04-10 14:42:16 +08:00
GalaIO
26f50099f4 blobtx: fix some found issues; (#2383) 2024-04-10 14:31:23 +08:00
zzzckck
060e5c6b34 Merge pull request #2380 from bnb-chain/develop
Draft release v1.4.5
2024-04-09 11:05:30 +08:00
zzzckck
4e9f699068 release: prepare for release v1.4.5 (#2379) 2024-04-08 19:20:23 +08:00
zzzckck
42a0236587 config: setup Testnet Tycho(Cancun) hardfork date (#2378)
expected hard fork date:
Testnet: 2024-04-17 05:07:22 AM UTC
2024-04-08 19:10:38 +08:00
Ng Wei Han
48f58a50bb core: stateDb has no trie and no snap return err (#2369) 2024-04-07 12:07:25 +08:00
sellskin
e4688e4e7a remove code that will not be executed (#2333) 2024-04-03 14:02:34 +08:00
Ng Wei Han
75a03f420f triedb: do not open state freezer under notries (#2359) 2024-04-02 19:39:43 +08:00
Péter Szilágyi
0b1438c3df eth: make transaction propagation paths in the network deterministic (#29034)
* eth: make transaction propagation paths in the network deterministic

* eth: avoid potential division by 0

* eth: make tx propagation dependent on local node id too

* eth: fix review comments
2024-03-02 22:39:22 +02:00
Sina Mahmoodi
0a2f33946b eth/catalyst: update simulated beacon for cancun (#28829)
* eth/catalyst: update simulated beacon for cancun

* validate blob hashes

* compute hashes from commitment

* fix beacon root and payload version

* check commitment conversion

* fix random attr

* flip dev to cancun
2024-02-29 14:17:32 +02:00
Péter Szilágyi
865e1e9f57 cmd/utils, core/rawdb, triedb/pathdb: flip hash to path scheme (#29108)
* cmd/utils, core/rawdb, triedb/pathdb: flip hash to path scheme

* graphql: run tests in hash mode as the chain maker needs it
2024-02-29 12:40:59 +02:00
yzb
db4cf69166 all: replace fmt.Errorf() with errors.New() if no param required (#29126)
replace-fmt-errorf

Co-authored-by: yzb@example.cn <yzb@example.cn>
2024-02-29 11:56:46 +02:00
Ng Wei Han
28d55218f7 cmd/geth: parseDumpConfig should not return closed db (#29100)
* cmd: parseDumpConfig should not return closed db

* fix lint
2024-02-29 11:56:17 +02:00
cui fliter
dbc27a199f all: fix function names in docs (#29128)
Signed-off-by: cui fliter <imcusg@gmail.com>
2024-02-29 11:29:06 +02:00
lightclient
1883438964 eth/catalyst: return invalid payload attributes instead of invalid parms for bad fcu payload (#29115) 2024-02-28 19:59:16 +01:00
buddho
9986a69c25 internal/ethapi: pass in accesslist in test (#29089)
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
2024-02-28 18:38:21 +01:00
rjl493456442
5bae14f9df triedb/pathdb: fix panic in recoverable (#29107)
* triedb/pathdb: fix panic in recoverable

* triedb/pathdb: add todo

* triedb/pathdb: rename

* triedb/pathdb: rename
2024-02-28 14:40:28 +02:00
rjl493456442
49623bd469 core, triedb/pathdb: calculate the size for batch pre-allocation (#29106)
* core, triedb/pathdb: calculate the size for batch pre-allocation

* triedb/pathdb: address comment
2024-02-28 14:23:52 +02:00
Péter Szilágyi
170fcd80c6 params: being major version bump cycle 2024-02-28 10:01:52 +02:00
cui
02d77c98f9 core: using math.MaxUint64 instead of 0xffffffffffffffff (#29094) 2024-02-28 15:25:12 +08:00
Péter Szilágyi
57d2b552c7 params: begin v1.13.15 cycle 2024-02-27 13:53:30 +02:00
224 changed files with 8867 additions and 1270 deletions

20
.github/CODEOWNERS vendored
View File

@@ -1,3 +1,21 @@
# Lines starting with '#' are comments.
# Each line is a file pattern followed by one or more owners.
* @zzzckck @zjubfd
accounts/usbwallet @karalabe
accounts/scwallet @gballet
accounts/abi @gballet @MariusVanDerWijden
cmd/clef @holiman
consensus @karalabe
core/ @karalabe @holiman @rjl493456442
eth/ @karalabe @holiman @rjl493456442
eth/catalyst/ @gballet
eth/tracers/ @s1na
graphql/ @s1na
les/ @zsfelfoldi @rjl493456442
light/ @zsfelfoldi @rjl493456442
node/ @fjl
p2p/ @fjl @zsfelfoldi
rpc/ @fjl @holiman
p2p/simulations @fjl
p2p/protocols @fjl
p2p/testing @fjl
signer/ @holiman

1
.nancy-ignore Normal file
View File

@@ -0,0 +1 @@
CVE-2024-34478 # "CWE-754: Improper Check for Unusual or Exceptional Conditions." This vulnerability is BTC only, BSC does not have the issue.

View File

@@ -1,4 +1,70 @@
# Changelog
## v1.4.8
### FEATURE
* [\#2483](https://github.com/bnb-chain/bsc/pull/2483) core/vm: add secp256r1 into PrecompiledContractsHaber
* [\#2400](https://github.com/bnb-chain/bsc/pull/2400) RIP-7212: Precompile for secp256r1 Curve Support
### IMPROVEMENT
NA
### BUGFIX
NA
## v1.4.7
### FEATURE
* [\#2439](https://github.com/bnb-chain/bsc/pull/2439) config: setup Mainnet Tycho(Cancun) hardfork date
### IMPROVEMENT
* [\#2396](https://github.com/bnb-chain/bsc/pull/2396) metrics: add blockInsertMgaspsGauge to trace mgasps
* [\#2411](https://github.com/bnb-chain/bsc/pull/2411) build(deps): bump golang.org/x/net from 0.19.0 to 0.23.0
* [\#2435](https://github.com/bnb-chain/bsc/pull/2435) txpool: limit max gas when mining is enabled
* [\#2438](https://github.com/bnb-chain/bsc/pull/2438) fix: performance issue when load journal
* [\#2440](https://github.com/bnb-chain/bsc/pull/2440) nancy: add files .nancy-ignore
### BUGFIX
NA
## v1.4.6
### FEATURE
* [\#2227](https://github.com/bnb-chain/bsc/pull/2227) core: separated databases for block data
* [\#2404](https://github.com/bnb-chain/bsc/pull/2404) cmd, p2p: filter peers by regex on name
### IMPROVEMENT
* [\#2201](https://github.com/bnb-chain/bsc/pull/2201) chore: render system bytecode by go:embed
* [\#2363](https://github.com/bnb-chain/bsc/pull/2363) feat: greedy merge tx in bid
* [\#2389](https://github.com/bnb-chain/bsc/pull/2389) deps: update prsym to solve warning about quic-go version
* [\#2341](https://github.com/bnb-chain/bsc/pull/2341) core/trie: persist TrieJournal to journal file instead of kv database
* [\#2395](https://github.com/bnb-chain/bsc/pull/2395) fix: trieJournal format compatible old db format
* [\#2406](https://github.com/bnb-chain/bsc/pull/2406) feat: adaptive for loading journal file or journal kv during loadJournal
* [\#2390](https://github.com/bnb-chain/bsc/pull/2390) chore: fix function names in comment
* [\#2399](https://github.com/bnb-chain/bsc/pull/2399) chore: fix some typos in comments
* [\#2408](https://github.com/bnb-chain/bsc/pull/2408) chore: fix some typos in comments
* [\#2416](https://github.com/bnb-chain/bsc/pull/2416) fix: fix function names
* [\#2424](https://github.com/bnb-chain/bsc/pull/2424) feat: recommit bid when newBidCh is empty to maximize mev reward
* [\#2430](https://github.com/bnb-chain/bsc/pull/2430) fix: oom caused by non-discarded mev simulation env
* [\#2428](https://github.com/bnb-chain/bsc/pull/2428) chore: add metric & log for blobTx
* [\#2419](https://github.com/bnb-chain/bsc/pull/2419) metrics: add doublesign counter
### BUGFIX
* [\#2244](https://github.com/bnb-chain/bsc/pull/2244) cmd/geth: fix importBlock
* [\#2391](https://github.com/bnb-chain/bsc/pull/2391) fix: print value instead of pointer in ConfigCompatError
* [\#2398](https://github.com/bnb-chain/bsc/pull/2398) fix: no import blocks before or equal to the finalized height
* [\#2401](https://github.com/bnb-chain/bsc/pull/2401) fix: allow fast node to rewind after abnormal shutdown
* [\#2403](https://github.com/bnb-chain/bsc/pull/2403) fix: NPE
* [\#2423](https://github.com/bnb-chain/bsc/pull/2423) eth/gasprice: add query limit to defend DDOS attack
* [\#2425](https://github.com/bnb-chain/bsc/pull/2425) fix: adapt journal for cmd
## v1.4.5
### FEATURE
* [\#2378](https://github.com/bnb-chain/bsc/pull/2378) config: setup Testnet Tycho(Cancun) hardfork date
### IMPROVEMENT
* [\#2333](https://github.com/bnb-chain/bsc/pull/2333) remove code that will not be executed
* [\#2369](https://github.com/bnb-chain/bsc/pull/2369) core: stateDb has no trie and no snap return err
### BUGFIX
* [\#2359](https://github.com/bnb-chain/bsc/pull/2359) triedb: do not open state freezer under notries
## v1.4.4
### FEATURE
* [\#2279](https://github.com/bnb-chain/bsc/pull/2279) BlobTx: implement EIP-4844 on BSC

View File

@@ -26,7 +26,7 @@ COPY --from=builder /go-ethereum/build/bin/* /usr/local/bin/
EXPOSE 8545 8546 30303 30303/udp
# Add some metadata labels to help programatic image consumption
# Add some metadata labels to help programmatic image consumption
ARG COMMIT=""
ARG VERSION=""
ARG BUILDNUM=""

View File

@@ -155,7 +155,7 @@ Note: If you encounter difficulties downloading the chaindata snapshot and prefe
```shell
./geth --config ./config.toml --datadir ./node --cache 8000 --rpc.allow-unprotected-txs --history.transactions 0
## It is recommand to run fullnode with `--tries-verify-mode none` if you want high performance and care little about state consistency
## It is recommend to run fullnode with `--tries-verify-mode none` if you want high performance and care little about state consistency
## It will run with Hash-Base Storage Scheme by default
./geth --config ./config.toml --datadir ./node --cache 8000 --rpc.allow-unprotected-txs --history.transactions 0 --tries-verify-mode none

View File

@@ -163,7 +163,7 @@ func (api *ExternalSigner) SignData(account accounts.Account, mimeType string, d
hexutil.Encode(data)); err != nil {
return nil, err
}
// If V is on 27/28-form, convert to to 0/1 for Clique and Parlia
// If V is on 27/28-form, convert to 0/1 for Clique and Parlia
if (mimeType == accounts.MimetypeClique || mimeType == accounts.MimetypeParlia) && (res[64] == 27 || res[64] == 28) {
res[64] -= 27 // Transform V from 27/28 to 0/1 for Clique and Parlia use
}

View File

@@ -51,7 +51,7 @@ var (
}
)
// waitWatcherStarts waits up to 1s for the keystore watcher to start.
// waitWatcherStart waits up to 1s for the keystore watcher to start.
func waitWatcherStart(ks *KeyStore) bool {
// On systems where file watch is not supported, just return "ok".
if !ks.cache.watcher.enabled() {
@@ -358,7 +358,6 @@ func TestUpdatedKeyfileContents(t *testing.T) {
// Now replace file contents
if err := forceCopyFile(file, cachetestAccounts[1].URL.Path); err != nil {
t.Fatal(err)
return
}
wantAccounts = []accounts.Account{cachetestAccounts[1]}
wantAccounts[0].URL = accounts.URL{Scheme: KeyStoreScheme, Path: file}
@@ -374,7 +373,6 @@ func TestUpdatedKeyfileContents(t *testing.T) {
// Now replace file contents again
if err := forceCopyFile(file, cachetestAccounts[2].URL.Path); err != nil {
t.Fatal(err)
return
}
wantAccounts = []accounts.Account{cachetestAccounts[2]}
wantAccounts[0].URL = accounts.URL{Scheme: KeyStoreScheme, Path: file}
@@ -390,7 +388,6 @@ func TestUpdatedKeyfileContents(t *testing.T) {
// Now replace file contents with crap
if err := os.WriteFile(file, []byte("foo"), 0600); err != nil {
t.Fatal(err)
return
}
if err := waitForAccounts([]accounts.Account{}, ks); err != nil {
t.Errorf("Emptying account file failed")

View File

@@ -343,7 +343,7 @@ func TestWalletNotifications(t *testing.T) {
checkEvents(t, wantEvents, events)
}
// TestImportExport tests the import functionality of a keystore.
// TestImportECDSA tests the import functionality of a keystore.
func TestImportECDSA(t *testing.T) {
t.Parallel()
_, ks := tmpKeyStore(t, true)
@@ -362,7 +362,7 @@ func TestImportECDSA(t *testing.T) {
}
}
// TestImportECDSA tests the import and export functionality of a keystore.
// TestImportExport tests the import and export functionality of a keystore.
func TestImportExport(t *testing.T) {
t.Parallel()
_, ks := tmpKeyStore(t, true)

View File

@@ -525,7 +525,7 @@ func (f *faucet) apiHandler(w http.ResponseWriter, r *http.Request) {
}
f.lock.Unlock()
// Send an error if too frequent funding, othewise a success
// Send an error if too frequent funding, otherwise a success
if !fund {
if err = sendError(wsconn, fmt.Errorf("%s left until next allowance", common.PrettyDuration(time.Until(timeout)))); err != nil { // nolint: gosimple
log.Warn("Failed to send funding error to client", "err", err)

View File

@@ -48,7 +48,7 @@ func TestAttachWithHeaders(t *testing.T) {
// This is fixed in a follow-up PR.
}
// TestAttachWithHeaders tests that 'geth db --remotedb' with custom headers works, i.e
// TestRemoteDbWithHeaders tests that 'geth db --remotedb' with custom headers works, i.e
// that custom headers are forwarded to the target.
func TestRemoteDbWithHeaders(t *testing.T) {
t.Parallel()

View File

@@ -12,16 +12,16 @@ import (
"github.com/google/uuid"
"github.com/logrusorgru/aurora"
"github.com/prysmaticlabs/prysm/v4/crypto/bls"
"github.com/prysmaticlabs/prysm/v4/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v4/io/prompt"
validatorpb "github.com/prysmaticlabs/prysm/v4/proto/prysm/v1alpha1/validator-client"
"github.com/prysmaticlabs/prysm/v4/validator/accounts"
"github.com/prysmaticlabs/prysm/v4/validator/accounts/iface"
"github.com/prysmaticlabs/prysm/v4/validator/accounts/petnames"
"github.com/prysmaticlabs/prysm/v4/validator/accounts/wallet"
"github.com/prysmaticlabs/prysm/v4/validator/keymanager"
"github.com/prysmaticlabs/prysm/v4/validator/keymanager/local"
"github.com/prysmaticlabs/prysm/v5/crypto/bls"
"github.com/prysmaticlabs/prysm/v5/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/v5/io/prompt"
validatorpb "github.com/prysmaticlabs/prysm/v5/proto/prysm/v1alpha1/validator-client"
"github.com/prysmaticlabs/prysm/v5/validator/accounts"
"github.com/prysmaticlabs/prysm/v5/validator/accounts/iface"
"github.com/prysmaticlabs/prysm/v5/validator/accounts/petnames"
"github.com/prysmaticlabs/prysm/v5/validator/accounts/wallet"
"github.com/prysmaticlabs/prysm/v5/validator/keymanager"
"github.com/prysmaticlabs/prysm/v5/validator/keymanager/local"
"github.com/urfave/cli/v2"
keystorev4 "github.com/wealdtech/go-eth2-wallet-encryptor-keystorev4"

View File

@@ -41,6 +41,7 @@ import (
"github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/eth"
"github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/internal/era"
"github.com/ethereum/go-ethereum/internal/flags"
@@ -62,6 +63,7 @@ var (
Flags: flags.Merge([]cli.Flag{
utils.CachePreimagesFlag,
utils.OverrideCancun,
utils.OverrideHaber,
utils.OverrideVerkle,
}, utils.DatabaseFlags),
Description: `
@@ -255,6 +257,10 @@ func initGenesis(ctx *cli.Context) error {
v := ctx.Uint64(utils.OverrideCancun.Name)
overrides.OverrideCancun = &v
}
if ctx.IsSet(utils.OverrideHaber.Name) {
v := ctx.Uint64(utils.OverrideHaber.Name)
overrides.OverrideHaber = &v
}
if ctx.IsSet(utils.OverrideVerkle.Name) {
v := ctx.Uint64(utils.OverrideVerkle.Name)
overrides.OverrideVerkle = &v
@@ -267,15 +273,21 @@ func initGenesis(ctx *cli.Context) error {
defer chaindb.Close()
// if the trie data dir has been set, new trie db with a new state database
if ctx.IsSet(utils.SeparateDBFlag.Name) {
if ctx.IsSet(utils.MultiDataBaseFlag.Name) {
statediskdb, dbErr := stack.OpenDatabaseWithFreezer(name+"/state", 0, 0, "", "", false, false, false, false)
if dbErr != nil {
utils.Fatalf("Failed to open separate trie database: %v", dbErr)
}
chaindb.SetStateStore(statediskdb)
blockdb, err := stack.OpenDatabaseWithFreezer(name+"/block", 0, 0, "", "", false, false, false, false)
if err != nil {
utils.Fatalf("Failed to open separate block database: %v", err)
}
chaindb.SetBlockStore(blockdb)
log.Warn("Multi-database is an experimental feature")
}
triedb := utils.MakeTrieDatabase(ctx, chaindb, ctx.Bool(utils.CachePreimagesFlag.Name), false, genesis.IsVerkle())
triedb := utils.MakeTrieDatabase(ctx, stack, chaindb, ctx.Bool(utils.CachePreimagesFlag.Name), false, genesis.IsVerkle())
defer triedb.Close()
_, hash, err := core.SetupGenesisBlockWithOverride(chaindb, triedb, genesis, &overrides)
@@ -473,6 +485,13 @@ func dumpGenesis(ctx *cli.Context) error {
}
continue
}
// set the separate state & block database
if stack.CheckIfMultiDataBase() && err == nil {
stateDiskDb := utils.MakeStateDataBase(ctx, stack, true, false)
db.SetStateStore(stateDiskDb)
blockDb := utils.MakeBlockDatabase(ctx, stack, true, false)
db.SetBlockStore(blockDb)
}
genesis, err := core.ReadGenesis(db)
if err != nil {
utils.Fatalf("failed to read genesis: %s", err)
@@ -500,10 +519,16 @@ func importChain(ctx *cli.Context) error {
// Start system runtime metrics collection
go metrics.CollectProcessMetrics(3 * time.Second)
stack, _ := makeConfigNode(ctx)
stack, cfg := makeConfigNode(ctx)
defer stack.Close()
chain, db := utils.MakeChain(ctx, stack, false)
backend, err := eth.New(stack, &cfg.Eth)
if err != nil {
return err
}
chain := backend.BlockChain()
db := backend.ChainDb()
defer db.Close()
// Start periodically gathering memory profiles
@@ -739,7 +764,7 @@ func parseDumpConfig(ctx *cli.Context, stack *node.Node) (*state.DumpConfig, eth
arg := ctx.Args().First()
if hashish(arg) {
hash := common.HexToHash(arg)
if number := rawdb.ReadHeaderNumber(db, hash); number != nil {
if number := rawdb.ReadHeaderNumber(db.BlockStore(), hash); number != nil {
header = rawdb.ReadHeader(db, hash, *number)
} else {
return nil, nil, common.Hash{}, fmt.Errorf("block %x not found", hash)
@@ -758,7 +783,7 @@ func parseDumpConfig(ctx *cli.Context, stack *node.Node) (*state.DumpConfig, eth
} else {
// Use latest
if scheme == rawdb.PathScheme {
triedb := triedb.NewDatabase(db, &triedb.Config{PathDB: pathdb.ReadOnly})
triedb := triedb.NewDatabase(db, &triedb.Config{PathDB: utils.PathDBConfigAddJournalFilePath(stack, pathdb.ReadOnly)})
defer triedb.Close()
if stateRoot := triedb.Head(); stateRoot != (common.Hash{}) {
header.Root = stateRoot
@@ -808,7 +833,8 @@ func dump(ctx *cli.Context) error {
if err != nil {
return err
}
triedb := utils.MakeTrieDatabase(ctx, db, true, true, false) // always enable preimage lookup
defer db.Close()
triedb := utils.MakeTrieDatabase(ctx, stack, db, true, true, false) // always enable preimage lookup
defer triedb.Close()
state, err := state.New(root, state.NewDatabaseWithNodeDB(db, triedb), nil)
@@ -828,7 +854,7 @@ func dumpAllRootHashInPath(ctx *cli.Context) error {
defer stack.Close()
db := utils.MakeChainDatabase(ctx, stack, true, false)
defer db.Close()
triedb := triedb.NewDatabase(db, &triedb.Config{PathDB: pathdb.ReadOnly})
triedb := triedb.NewDatabase(db, &triedb.Config{PathDB: utils.PathDBConfigAddJournalFilePath(stack, pathdb.ReadOnly)})
defer triedb.Close()
scheme, err := rawdb.ParseStateScheme(ctx.String(utils.StateSchemeFlag.Name), db)

View File

@@ -189,18 +189,14 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
v := ctx.Uint64(utils.OverrideCancun.Name)
cfg.Eth.OverrideCancun = &v
}
if ctx.IsSet(utils.OverrideHaber.Name) {
v := ctx.Uint64(utils.OverrideHaber.Name)
cfg.Eth.OverrideHaber = &v
}
if ctx.IsSet(utils.OverrideVerkle.Name) {
v := ctx.Uint64(utils.OverrideVerkle.Name)
cfg.Eth.OverrideVerkle = &v
}
if ctx.IsSet(utils.OverrideFeynman.Name) {
v := ctx.Uint64(utils.OverrideFeynman.Name)
cfg.Eth.OverrideFeynman = &v
}
if ctx.IsSet(utils.OverrideFeynmanFix.Name) {
v := ctx.Uint64(utils.OverrideFeynmanFix.Name)
cfg.Eth.OverrideFeynmanFix = &v
}
if ctx.IsSet(utils.OverrideFullImmutabilityThreshold.Name) {
params.FullImmutabilityThreshold = ctx.Uint64(utils.OverrideFullImmutabilityThreshold.Name)
downloader.FullMaxForkAncestry = ctx.Uint64(utils.OverrideFullImmutabilityThreshold.Name)
@@ -211,9 +207,10 @@ func makeFullNode(ctx *cli.Context) (*node.Node, ethapi.Backend) {
if ctx.IsSet(utils.OverrideDefaultExtraReserveForBlobRequests.Name) {
params.DefaultExtraReserveForBlobRequests = ctx.Uint64(utils.OverrideDefaultExtraReserveForBlobRequests.Name)
}
if ctx.IsSet(utils.SeparateDBFlag.Name) && !stack.IsSeparatedDB() {
utils.Fatalf("Failed to locate separate database subdirectory when separatedb parameter has been set")
if ctx.IsSet(utils.OverrideBreatheBlockInterval.Name) {
params.BreatheBlockInterval = ctx.Uint64(utils.OverrideBreatheBlockInterval.Name)
}
backend, eth := utils.RegisterEthService(stack, &cfg.Eth)
// Create gauge with geth system and build information

View File

@@ -106,12 +106,12 @@ Remove blockchain and state databases`,
dbInspectTrieCmd = &cli.Command{
Action: inspectTrie,
Name: "inspect-trie",
ArgsUsage: "<blocknum> <jobnum>",
ArgsUsage: "<blocknum> <jobnum> <topn>",
Flags: []cli.Flag{
utils.DataDirFlag,
utils.SyncModeFlag,
},
Usage: "Inspect the MPT tree of the account and contract.",
Usage: "Inspect the MPT tree of the account and contract. 'blocknum' can be latest/snapshot/number. 'topn' means output the top N storage tries info ranked by the total number of TrieNodes",
Description: `This commands iterates the entrie WorldState.`,
}
dbCheckStateContentCmd = &cli.Command{
@@ -386,6 +386,7 @@ func inspectTrie(ctx *cli.Context) error {
blockNumber uint64
trieRootHash common.Hash
jobnum uint64
topN uint64
)
stack, _ := makeConfigNode(ctx)
@@ -396,8 +397,8 @@ func inspectTrie(ctx *cli.Context) error {
var headerBlockHash common.Hash
if ctx.NArg() >= 1 {
if ctx.Args().Get(0) == "latest" {
headerHash := rawdb.ReadHeadHeaderHash(db)
blockNumber = *(rawdb.ReadHeaderNumber(db, headerHash))
headerHash := rawdb.ReadHeadHeaderHash(db.BlockStore())
blockNumber = *(rawdb.ReadHeaderNumber(db.BlockStore(), headerHash))
} else if ctx.Args().Get(0) == "snapshot" {
trieRootHash = rawdb.ReadSnapshotRoot(db)
blockNumber = math.MaxUint64
@@ -411,12 +412,25 @@ func inspectTrie(ctx *cli.Context) error {
if ctx.NArg() == 1 {
jobnum = 1000
topN = 10
} else if ctx.NArg() == 2 {
var err error
jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64)
if err != nil {
return fmt.Errorf("failed to Parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
}
topN = 10
} else {
var err error
jobnum, err = strconv.ParseUint(ctx.Args().Get(1), 10, 64)
if err != nil {
return fmt.Errorf("failed to Parse jobnum, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
}
topN, err = strconv.ParseUint(ctx.Args().Get(2), 10, 64)
if err != nil {
return fmt.Errorf("failed to Parse topn, Args[1]: %v, err: %v", ctx.Args().Get(1), err)
}
}
if blockNumber != math.MaxUint64 {
@@ -436,7 +450,8 @@ func inspectTrie(ctx *cli.Context) error {
var config *triedb.Config
if dbScheme == rawdb.PathScheme {
config = &triedb.Config{
PathDB: pathdb.ReadOnly,
PathDB: utils.PathDBConfigAddJournalFilePath(stack, pathdb.ReadOnly),
Cache: 0,
}
} else if dbScheme == rawdb.HashScheme {
config = triedb.HashDefaults
@@ -448,7 +463,7 @@ func inspectTrie(ctx *cli.Context) error {
fmt.Printf("fail to new trie tree, err: %v, rootHash: %v\n", err, trieRootHash.String())
return err
}
theInspect, err := trie.NewInspector(theTrie, triedb, trieRootHash, blockNumber, jobnum)
theInspect, err := trie.NewInspector(theTrie, triedb, trieRootHash, blockNumber, jobnum, int(topN))
if err != nil {
return err
}
@@ -519,7 +534,7 @@ func checkStateContent(ctx *cli.Context) error {
db := utils.MakeChainDatabase(ctx, stack, true, false)
defer db.Close()
var (
it = rawdb.NewKeyLengthIterator(db.NewIterator(prefix, start), 32)
it ethdb.Iterator
hasher = crypto.NewKeccakState()
got = make([]byte, 32)
errs int
@@ -527,6 +542,11 @@ func checkStateContent(ctx *cli.Context) error {
startTime = time.Now()
lastLog = time.Now()
)
if stack.CheckIfMultiDataBase() {
it = rawdb.NewKeyLengthIterator(db.StateStore().NewIterator(prefix, start), 32)
} else {
it = rawdb.NewKeyLengthIterator(db.NewIterator(prefix, start), 32)
}
for it.Next() {
count++
k := it.Key()
@@ -573,9 +593,11 @@ func dbStats(ctx *cli.Context) error {
defer db.Close()
showLeveldbStats(db)
if db.StateStore() != nil {
if stack.CheckIfMultiDataBase() {
fmt.Println("show stats of state store")
showLeveldbStats(db.StateStore())
fmt.Println("show stats of block store")
showLeveldbStats(db.BlockStore())
}
return nil
@@ -591,10 +613,11 @@ func dbCompact(ctx *cli.Context) error {
log.Info("Stats before compaction")
showLeveldbStats(db)
statediskdb := db.StateStore()
if statediskdb != nil {
if stack.CheckIfMultiDataBase() {
fmt.Println("show stats of state store")
showLeveldbStats(statediskdb)
showLeveldbStats(db.StateStore())
fmt.Println("show stats of block store")
showLeveldbStats(db.BlockStore())
}
log.Info("Triggering compaction")
@@ -603,8 +626,12 @@ func dbCompact(ctx *cli.Context) error {
return err
}
if statediskdb != nil {
if err := statediskdb.Compact(nil, nil); err != nil {
if stack.CheckIfMultiDataBase() {
if err := db.StateStore().Compact(nil, nil); err != nil {
log.Error("Compact err", "error", err)
return err
}
if err := db.BlockStore().Compact(nil, nil); err != nil {
log.Error("Compact err", "error", err)
return err
}
@@ -612,9 +639,11 @@ func dbCompact(ctx *cli.Context) error {
log.Info("Stats after compaction")
showLeveldbStats(db)
if statediskdb != nil {
if stack.CheckIfMultiDataBase() {
fmt.Println("show stats of state store after compaction")
showLeveldbStats(statediskdb)
showLeveldbStats(db.StateStore())
fmt.Println("show stats of block store after compaction")
showLeveldbStats(db.BlockStore())
}
return nil
}
@@ -635,18 +664,18 @@ func dbGet(ctx *cli.Context) error {
log.Info("Could not decode the key", "error", err)
return err
}
statediskdb := db.StateStore()
data, err := db.Get(key)
if err != nil {
// if separate trie db exist, try to get it from separate db
if statediskdb != nil {
statedata, dberr := statediskdb.Get(key)
if dberr == nil {
fmt.Printf("key %#x: %#x\n", key, statedata)
return nil
}
opDb := db
if stack.CheckIfMultiDataBase() {
keyType := rawdb.DataTypeByKey(key)
if keyType == rawdb.StateDataType {
opDb = db.StateStore()
} else if keyType == rawdb.BlockDataType {
opDb = db.BlockStore()
}
}
data, err := opDb.Get(key)
if err != nil {
log.Info("Get operation failed", "key", fmt.Sprintf("%#x", key), "error", err)
return err
}
@@ -809,11 +838,21 @@ func dbDelete(ctx *cli.Context) error {
log.Info("Could not decode the key", "error", err)
return err
}
data, err := db.Get(key)
opDb := db
if stack.CheckIfMultiDataBase() {
keyType := rawdb.DataTypeByKey(key)
if keyType == rawdb.StateDataType {
opDb = db.StateStore()
} else if keyType == rawdb.BlockDataType {
opDb = db.BlockStore()
}
}
data, err := opDb.Get(key)
if err == nil {
fmt.Printf("Previous value: %#x\n", data)
}
if err = db.Delete(key); err != nil {
if err = opDb.Delete(key); err != nil {
log.Info("Delete operation returned an error", "key", fmt.Sprintf("%#x", key), "error", err)
return err
}
@@ -923,11 +962,22 @@ func dbPut(ctx *cli.Context) error {
log.Info("Could not decode the value", "error", err)
return err
}
data, err = db.Get(key)
opDb := db
if stack.CheckIfMultiDataBase() {
keyType := rawdb.DataTypeByKey(key)
if keyType == rawdb.StateDataType {
opDb = db.StateStore()
} else if keyType == rawdb.BlockDataType {
opDb = db.BlockStore()
}
}
data, err = opDb.Get(key)
if err == nil {
fmt.Printf("Previous value: %#x\n", data)
}
return db.Put(key, value)
return opDb.Put(key, value)
}
// dbDumpTrie shows the key-value slots of a given storage trie
@@ -940,8 +990,7 @@ func dbDumpTrie(ctx *cli.Context) error {
db := utils.MakeChainDatabase(ctx, stack, true, false)
defer db.Close()
triedb := utils.MakeTrieDatabase(ctx, db, false, true, false)
triedb := utils.MakeTrieDatabase(ctx, stack, db, false, true, false)
defer triedb.Close()
var (
@@ -1019,7 +1068,7 @@ func freezerInspect(ctx *cli.Context) error {
stack, _ := makeConfigNode(ctx)
ancient := stack.ResolveAncient("chaindata", ctx.String(utils.AncientFlag.Name))
stack.Close()
return rawdb.InspectFreezerTable(ancient, freezer, table, start, end)
return rawdb.InspectFreezerTable(ancient, freezer, table, start, end, stack.CheckIfMultiDataBase())
}
func importLDBdata(ctx *cli.Context) error {
@@ -1159,11 +1208,11 @@ func showMetaData(ctx *cli.Context) error {
defer stack.Close()
db := utils.MakeChainDatabase(ctx, stack, true, false)
defer db.Close()
ancients, err := db.Ancients()
ancients, err := db.BlockStore().Ancients()
if err != nil {
fmt.Fprintf(os.Stderr, "Error accessing ancients: %v", err)
}
data := rawdb.ReadChainMetadata(db)
data := rawdb.ReadChainMetadataFromMultiDatabase(db)
data = append(data, []string{"frozen", fmt.Sprintf("%d items", ancients)})
data = append(data, []string{"snapshotGenerator", snapshot.ParseGeneratorStatus(rawdb.ReadSnapshotGenerator(db))})
if b := rawdb.ReadHeadBlock(db); b != nil {
@@ -1224,8 +1273,8 @@ func hbss2pbss(ctx *cli.Context) error {
log.Info("hbss2pbss triedb", "scheme", triedb.Scheme())
defer triedb.Close()
headerHash := rawdb.ReadHeadHeaderHash(db)
blockNumber := rawdb.ReadHeaderNumber(db, headerHash)
headerHash := rawdb.ReadHeadHeaderHash(db.BlockStore())
blockNumber := rawdb.ReadHeaderNumber(db.BlockStore(), headerHash)
if blockNumber == nil {
log.Error("read header number failed.")
return fmt.Errorf("read header number failed")

View File

@@ -25,6 +25,8 @@ import (
"strings"
"time"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/accounts"
"github.com/ethereum/go-ethereum/accounts/keystore"
"github.com/ethereum/go-ethereum/cmd/utils"
@@ -71,12 +73,12 @@ var (
utils.SmartCardDaemonPathFlag,
utils.RialtoHash,
utils.OverrideCancun,
utils.OverrideHaber,
utils.OverrideVerkle,
utils.OverrideFeynman,
utils.OverrideFeynmanFix,
utils.OverrideFullImmutabilityThreshold,
utils.OverrideMinBlocksForBlobRequests,
utils.OverrideDefaultExtraReserveForBlobRequests,
utils.OverrideBreatheBlockInterval,
utils.EnablePersonal,
utils.TxPoolLocalsFlag,
utils.TxPoolNoLocalsFlag,
@@ -103,6 +105,7 @@ var (
utils.TransactionHistoryFlag,
utils.StateHistoryFlag,
utils.PathDBSyncFlag,
utils.JournalFileFlag,
utils.LightServeFlag, // deprecated
utils.LightIngressFlag, // deprecated
utils.LightEgressFlag, // deprecated
@@ -144,6 +147,7 @@ var (
// utils.MinerNewPayloadTimeout,
utils.NATFlag,
utils.NoDiscoverFlag,
utils.PeerFilterPatternsFlag,
utils.DiscoveryV4Flag,
utils.DiscoveryV5Flag,
utils.InstanceFlag,
@@ -454,6 +458,10 @@ func startNode(ctx *cli.Context, stack *node.Node, backend ethapi.Backend, isCon
// Set the gas price to the limits from the CLI and start mining
gasprice := flags.GlobalBig(ctx, utils.MinerGasPriceFlag.Name)
ethBackend.TxPool().SetGasTip(gasprice)
gasCeil := ethBackend.Miner().GasCeil()
if gasCeil > params.SystemTxsGas {
ethBackend.TxPool().SetMaxGas(gasCeil - params.SystemTxsGas)
}
if err := ethBackend.StartMining(); err != nil {
utils.Fatalf("Failed to start mining: %v", err)
}

View File

@@ -178,10 +178,11 @@ func BlockchainCreator(t *testing.T, chaindbPath, AncientPath string, blockRemai
// Force run a freeze cycle
type freezer interface {
Freeze(threshold uint64) error
Freeze() error
Ancients() (uint64, error)
}
db.(freezer).Freeze(10)
blockchain.SetFinalized(blocks[len(blocks)-1].Header())
db.(freezer).Freeze()
frozen, err := db.Ancients()
//make sure there're frozen items

View File

@@ -92,7 +92,7 @@ WARNING: it's only supported in hash mode(--state.scheme=hash)".
geth offline prune-block for block data in ancientdb.
The amount of blocks expected for remaining after prune can be specified via block-amount-reserved in this command,
will prune and only remain the specified amount of old block data in ancientdb.
the brief workflow is to backup the the number of this specified amount blocks backward in original ancientdb
the brief workflow is to backup the number of this specified amount blocks backward in original ancientdb
into new ancient_backup, then delete the original ancientdb dir and rename the ancient_backup to original one for replacement,
finally assemble the statedb and new ancientDb together.
The purpose of doing it is because the block data will be moved into the ancient store when it
@@ -355,7 +355,15 @@ func pruneBlock(ctx *cli.Context) error {
if !ctx.IsSet(utils.AncientFlag.Name) {
return errors.New("datadir.ancient must be set")
} else {
oldAncientPath = ctx.String(utils.AncientFlag.Name)
if stack.CheckIfMultiDataBase() {
ancientPath := ctx.String(utils.AncientFlag.Name)
index := strings.LastIndex(ancientPath, "/ancient/chain")
if index != -1 {
oldAncientPath = ancientPath[:index] + "/block/ancient/chain"
}
} else {
oldAncientPath = ctx.String(utils.AncientFlag.Name)
}
if !filepath.IsAbs(oldAncientPath) {
// force absolute paths, which often fail due to the splicing of relative paths
return errors.New("datadir.ancient not abs path")
@@ -401,7 +409,7 @@ func pruneBlock(ctx *cli.Context) error {
}
if _, err := os.Stat(newAncientPath); err == nil {
// No file lock found for old ancientDB but new ancientDB exsisted, indicating the geth was interrupted
// No file lock found for old ancientDB but new ancientDB existed, indicating the geth was interrupted
// after old ancientDB removal, this happened after backup successfully, so just rename the new ancientDB
if err := blockpruner.AncientDbReplacer(); err != nil {
log.Error("Failed to rename new ancient directory")
@@ -524,7 +532,7 @@ func verifyState(ctx *cli.Context) error {
log.Error("Failed to load head block")
return errors.New("no head block")
}
triedb := utils.MakeTrieDatabase(ctx, chaindb, false, true, false)
triedb := utils.MakeTrieDatabase(ctx, stack, chaindb, false, true, false)
defer triedb.Close()
snapConfig := snapshot.Config{
@@ -579,7 +587,7 @@ func traverseState(ctx *cli.Context) error {
chaindb := utils.MakeChainDatabase(ctx, stack, true, false)
defer chaindb.Close()
triedb := utils.MakeTrieDatabase(ctx, chaindb, false, true, false)
triedb := utils.MakeTrieDatabase(ctx, stack, chaindb, false, true, false)
defer triedb.Close()
headBlock := rawdb.ReadHeadBlock(chaindb)
@@ -688,7 +696,7 @@ func traverseRawState(ctx *cli.Context) error {
chaindb := utils.MakeChainDatabase(ctx, stack, true, false)
defer chaindb.Close()
triedb := utils.MakeTrieDatabase(ctx, chaindb, false, true, false)
triedb := utils.MakeTrieDatabase(ctx, stack, chaindb, false, true, false)
defer triedb.Close()
headBlock := rawdb.ReadHeadBlock(chaindb)
@@ -852,7 +860,8 @@ func dumpState(ctx *cli.Context) error {
if err != nil {
return err
}
triedb := utils.MakeTrieDatabase(ctx, db, false, true, false)
defer db.Close()
triedb := utils.MakeTrieDatabase(ctx, stack, db, false, true, false)
defer triedb.Close()
snapConfig := snapshot.Config{
@@ -935,7 +944,7 @@ func snapshotExportPreimages(ctx *cli.Context) error {
chaindb := utils.MakeChainDatabase(ctx, stack, true, false)
defer chaindb.Close()
triedb := utils.MakeTrieDatabase(ctx, chaindb, false, true, false)
triedb := utils.MakeTrieDatabase(ctx, stack, chaindb, false, true, false)
defer triedb.Close()
var root common.Hash

View File

@@ -0,0 +1,51 @@
import { ethers } from "ethers";
import program from "commander";
// depends on ethjs v6.11.0+ for 4844, https://github.com/ethers-io/ethers.js/releases/tag/v6.11.0
// BSC testnet enabled 4844 on block: 39539137
// Usage:
// nvm use 20
// node check_blobtx.js --rpc https://data-seed-prebsc-1-s1.binance.org:8545 --startNum 39539137
// node check_blobtx.js --rpc https://data-seed-prebsc-1-s1.binance.org:8545 --startNum 39539137 --endNum 40345994
program.option("--rpc <Rpc>", "Rpc Server URL");
program.option("--startNum <Num>", "start block", 0);
program.option("--endNum <Num>", "end block", 0);
program.parse(process.argv);
const provider = new ethers.JsonRpcProvider(program.rpc);
const main = async () => {
var startBlock = parseInt(program.startNum)
var endBlock = parseInt(program.endNum)
if (isNaN(endBlock) || isNaN(startBlock) || startBlock == 0) {
console.error("invalid input, --startNum", program.startNum, "--end", program.endNum)
return
}
// if --endNum is not specified, set it to the latest block number.
if (endBlock == 0) {
endBlock = await provider.getBlockNumber();
}
if (startBlock > endBlock) {
console.error("invalid input, startBlock:",startBlock, " endBlock:", endBlock);
return
}
for (let i = startBlock; i <= endBlock; i++) {
let blockData = await provider.getBlock(i);
console.log("startBlock:",startBlock, "endBlock:", endBlock, "curBlock", i, "blobGasUsed", blockData.blobGasUsed);
if (blockData.blobGasUsed == 0) {
continue
}
for (let txIndex = 0; txIndex<= blockData.transactions.length - 1; txIndex++) {
let txHash = blockData.transactions[txIndex]
let txData = await provider.getTransaction(txHash);
if (txData.type == 3) {
console.log("BlobTx in block:",i, " txIndex:", txIndex, " txHash:", txHash);
}
}
}
};
main().then(() => process.exit(0))
.catch((error) => {
console.error(error);
process.exit(1);
});

View File

@@ -0,0 +1,49 @@
import { ethers } from "ethers";
import program from "commander";
// Usage:
// node faucet_request.js --rpc localhost:8545 --startNum 39539137
// node faucet_request.js --rpc localhost:8545 --startNum 39539137 --endNum 40345994
// node faucet_request.js --rpc https://data-seed-prebsc-1-s1.bnbchain.org:8545 --startNum 39539137 --endNum 40345994
program.option("--rpc <Rpc>", "Rpc Server URL");
program.option("--startNum <Num>", "start block", 0);
program.option("--endNum <Num>", "end block", 0);
program.parse(process.argv);
const provider = new ethers.JsonRpcProvider(program.rpc);
const main = async () => {
var startBlock = parseInt(program.startNum)
var endBlock = parseInt(program.endNum)
if (isNaN(endBlock) || isNaN(startBlock) || startBlock == 0) {
console.error("invalid input, --startNum", program.startNum, "--end", program.endNum)
return
}
// if --endNum is not specified, set it to the latest block number.
if (endBlock == 0) {
endBlock = await provider.getBlockNumber();
}
if (startBlock > endBlock) {
console.error("invalid input, startBlock:",startBlock, " endBlock:", endBlock);
return
}
let startBalance = await provider.getBalance("0xaa25Aa7a19f9c426E07dee59b12f944f4d9f1DD3", startBlock)
let endBalance = await provider.getBalance("0xaa25Aa7a19f9c426E07dee59b12f944f4d9f1DD3", endBlock)
const faucetAmount = BigInt(0.3 * 10**18); // Convert 0.3 ether to wei as a BigInt
const numFaucetRequest = (startBalance - endBalance) / faucetAmount;
// Convert BigInt to ether
const startBalanceEth = Number(startBalance) / 10**18;
const endBalanceEth = Number(endBalance) / 10**18;
console.log(`Start Balance: ${startBalanceEth} ETH`);
console.log(`End Balance: ${endBalanceEth} ETH`);
console.log("successful faucet request: ",numFaucetRequest);
};
main().then(() => process.exit(0))
.catch((error) => {
console.error(error);
process.exit(1);
});

View File

@@ -93,10 +93,10 @@ var (
Value: flags.DirectoryString(node.DefaultDataDir()),
Category: flags.EthCategory,
}
SeparateDBFlag = &cli.BoolFlag{
Name: "separatedb",
Usage: "Enable a separated trie database, it will be created within a subdirectory called state, " +
"Users can copy this state directory to another directory or disk, and then create a symbolic link to the state directory under the chaindata",
MultiDataBaseFlag = &cli.BoolFlag{
Name: "multidatabase",
Usage: "Enable a separated state and block database, it will be created within two subdirectory called state and block, " +
"Users can copy this state or block directory to another directory or disk, and then create a symbolic link to the state directory under the chaindata",
Category: flags.EthCategory,
}
DirectBroadcastFlag = &cli.BoolFlag{
@@ -224,7 +224,7 @@ var (
// hbss2pbss command options
ForceFlag = &cli.BoolFlag{
Name: "force",
Usage: "Force convert hbss trie node to pbss trie node. Ingore any metadata",
Usage: "Force convert hbss trie node to pbss trie node. Ignore any metadata",
Value: false,
}
// Dump command options.
@@ -310,20 +310,16 @@ var (
Usage: "Manually specify the Cancun fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideHaber = &cli.Uint64Flag{
Name: "override.haber",
Usage: "Manually specify the Haber fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideVerkle = &cli.Uint64Flag{
Name: "override.verkle",
Usage: "Manually specify the Verkle fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideFeynman = &cli.Uint64Flag{
Name: "override.feynman",
Usage: "Manually specify the Feynman fork timestamp, overriding the bundled setting",
Category: flags.EthCategory,
}
OverrideFeynmanFix = &cli.Uint64Flag{
Name: "override.feynmanfix",
Usage: "Manually specify the FeynmanFix fork timestamp, overriding the bundled setting",
}
OverrideFullImmutabilityThreshold = &cli.Uint64Flag{
Name: "override.immutabilitythreshold",
Usage: "It is the number of blocks after which a chain segment is considered immutable, only for testing purpose",
@@ -342,6 +338,12 @@ var (
Value: params.DefaultExtraReserveForBlobRequests,
Category: flags.EthCategory,
}
OverrideBreatheBlockInterval = &cli.Uint64Flag{
Name: "override.breatheblockinterval",
Usage: "It changes the interval between breathe blocks, only for testing purpose",
Value: params.BreatheBlockInterval,
Category: flags.EthCategory,
}
SyncModeFlag = &flags.TextMarshalerFlag{
Name: "syncmode",
Usage: `Blockchain sync mode ("snap" or "full")`,
@@ -365,6 +367,12 @@ var (
Value: false,
Category: flags.StateCategory,
}
JournalFileFlag = &cli.BoolFlag{
Name: "journalfile",
Usage: "Enable using journal file to store the TrieJournal instead of KVDB in pbss (default = false)",
Value: false,
Category: flags.StateCategory,
}
StateHistoryFlag = &cli.Uint64Flag{
Name: "history.state",
Usage: "Number of recent blocks to retain state history for (default = 90,000 blocks, 0 = entire chain)",
@@ -876,6 +884,11 @@ var (
Usage: "Disables the peer discovery mechanism (manual peer addition)",
Category: flags.NetworkingCategory,
}
PeerFilterPatternsFlag = &cli.StringSliceFlag{
Name: "peerfilter",
Usage: "Disallow peers connection if peer name matches the given regular expressions",
Category: flags.NetworkingCategory,
}
DiscoveryV4Flag = &cli.BoolFlag{
Name: "discovery.v4",
Aliases: []string{"discv4"},
@@ -1144,7 +1157,7 @@ var (
DBEngineFlag,
StateSchemeFlag,
HttpHeaderFlag,
SeparateDBFlag,
MultiDataBaseFlag,
}
)
@@ -1542,6 +1555,9 @@ func SetP2PConfig(ctx *cli.Context, cfg *p2p.Config) {
if ctx.IsSet(NoDiscoverFlag.Name) {
cfg.NoDiscovery = true
}
if ctx.IsSet(PeerFilterPatternsFlag.Name) {
cfg.PeerFilterPatterns = ctx.StringSlice(PeerFilterPatternsFlag.Name)
}
CheckExclusive(ctx, DiscoveryV4Flag, NoDiscoverFlag)
CheckExclusive(ctx, DiscoveryV5Flag, NoDiscoverFlag)
@@ -1962,6 +1978,10 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) {
if ctx.IsSet(PathDBSyncFlag.Name) {
cfg.PathSyncFlush = true
}
if ctx.IsSet(JournalFileFlag.Name) {
cfg.JournalFileEnabled = true
}
if ctx.String(GCModeFlag.Name) == "archive" && cfg.TransactionHistory != 0 {
cfg.TransactionHistory = 0
log.Warn("Disabled transaction unindexing for archive node")
@@ -2377,9 +2397,11 @@ func MakeChainDatabase(ctx *cli.Context, stack *node.Node, readonly, disableFree
default:
chainDb, err = stack.OpenDatabaseWithFreezer("chaindata", cache, handles, ctx.String(AncientFlag.Name), "", readonly, disableFreeze, false, false)
// set the separate state database
if stack.IsSeparatedDB() && err == nil {
if stack.CheckIfMultiDataBase() && err == nil {
stateDiskDb := MakeStateDataBase(ctx, stack, readonly, false)
chainDb.SetStateStore(stateDiskDb)
blockDb := MakeBlockDatabase(ctx, stack, readonly, false)
chainDb.SetBlockStore(blockDb)
}
}
if err != nil {
@@ -2391,7 +2413,7 @@ func MakeChainDatabase(ctx *cli.Context, stack *node.Node, readonly, disableFree
// MakeStateDataBase open a separate state database using the flags passed to the client and will hard crash if it fails.
func MakeStateDataBase(ctx *cli.Context, stack *node.Node, readonly, disableFreeze bool) ethdb.Database {
cache := ctx.Int(CacheFlag.Name) * ctx.Int(CacheDatabaseFlag.Name) / 100
handles := MakeDatabaseHandles(ctx.Int(FDLimitFlag.Name)) / 2
handles := MakeDatabaseHandles(ctx.Int(FDLimitFlag.Name)) * 90 / 100
statediskdb, err := stack.OpenDatabaseWithFreezer("chaindata/state", cache, handles, "", "", readonly, disableFreeze, false, false)
if err != nil {
Fatalf("Failed to open separate trie database: %v", err)
@@ -2399,6 +2421,23 @@ func MakeStateDataBase(ctx *cli.Context, stack *node.Node, readonly, disableFree
return statediskdb
}
// MakeBlockDatabase open a separate block database using the flags passed to the client and will hard crash if it fails.
func MakeBlockDatabase(ctx *cli.Context, stack *node.Node, readonly, disableFreeze bool) ethdb.Database {
cache := ctx.Int(CacheFlag.Name) * ctx.Int(CacheDatabaseFlag.Name) / 100
handles := MakeDatabaseHandles(ctx.Int(FDLimitFlag.Name)) / 10
blockDb, err := stack.OpenDatabaseWithFreezer("chaindata/block", cache, handles, "", "", readonly, disableFreeze, false, false)
if err != nil {
Fatalf("Failed to open separate block database: %v", err)
}
return blockDb
}
func PathDBConfigAddJournalFilePath(stack *node.Node, config *pathdb.Config) *pathdb.Config {
path := fmt.Sprintf("%s/%s", stack.ResolvePath("chaindata"), eth.JournalFileName)
config.JournalFilePath = path
return config
}
// tryMakeReadOnlyDatabase try to open the chain database in read-only mode,
// or fallback to write mode if the database is not initialized.
//
@@ -2541,7 +2580,7 @@ func MakeConsolePreloads(ctx *cli.Context) []string {
}
// MakeTrieDatabase constructs a trie database based on the configured scheme.
func MakeTrieDatabase(ctx *cli.Context, disk ethdb.Database, preimage bool, readOnly bool, isVerkle bool) *triedb.Database {
func MakeTrieDatabase(ctx *cli.Context, stack *node.Node, disk ethdb.Database, preimage bool, readOnly bool, isVerkle bool) *triedb.Database {
config := &triedb.Config{
Preimages: preimage,
IsVerkle: isVerkle,
@@ -2562,6 +2601,7 @@ func MakeTrieDatabase(ctx *cli.Context, disk ethdb.Database, preimage bool, read
} else {
config.PathDB = pathdb.Defaults
}
config.PathDB.JournalFilePath = fmt.Sprintf("%s/%s", stack.ResolvePath("chaindata"), eth.JournalFileName)
return triedb.NewDatabase(disk, config)
}

View File

@@ -467,7 +467,6 @@ func (tt *cliqueTest) run(t *testing.T) {
for j := 0; j < len(batches)-1; j++ {
if k, err := chain.InsertChain(batches[j]); err != nil {
t.Fatalf("failed to import batch %d, block %d: %v", j, k, err)
break
}
}
if _, err = chain.InsertChain(batches[len(batches)-1]); err != tt.failure {

View File

@@ -15,14 +15,13 @@ import (
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/internal/ethapi"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
"github.com/ethereum/go-ethereum/rpc"
)
const SecondsPerDay uint64 = 86400
// the params should be two blocks' time(timestamp)
func sameDayInUTC(first, second uint64) bool {
return first/SecondsPerDay == second/SecondsPerDay
return first/params.BreatheBlockInterval == second/params.BreatheBlockInterval
}
func isBreatheBlock(lastBlockTime, blockTime uint64) bool {

View File

@@ -16,7 +16,7 @@ import (
lru "github.com/hashicorp/golang-lru"
"github.com/holiman/uint256"
"github.com/prysmaticlabs/prysm/v4/crypto/bls"
"github.com/prysmaticlabs/prysm/v5/crypto/bls"
"github.com/willf/bitset"
"golang.org/x/crypto/sha3"
@@ -48,8 +48,9 @@ import (
)
const (
inMemorySnapshots = 256 // Number of recent snapshots to keep in memory
inMemorySignatures = 4096 // Number of recent block signatures to keep in memory
inMemorySnapshots = 256 // Number of recent snapshots to keep in memory
inMemorySignatures = 4096 // Number of recent block signatures to keep in memory
inMemoryHeaders = 86400 // Number of recent headers to keep in memory for double sign detection,
checkpointInterval = 1024 // Number of blocks after which to save the snapshot to the database
defaultEpochLength = uint64(100) // Default number of blocks of checkpoint to update validatorSet from contract
@@ -80,6 +81,7 @@ var (
verifyVoteAttestationErrorCounter = metrics.NewRegisteredCounter("parlia/verifyVoteAttestation/error", nil)
updateAttestationErrorCounter = metrics.NewRegisteredCounter("parlia/updateAttestation/error", nil)
validVotesfromSelfCounter = metrics.NewRegisteredCounter("parlia/VerifyVote/self", nil)
doubleSignCounter = metrics.NewRegisteredCounter("parlia/doublesign", nil)
systemContracts = map[common.Address]bool{
common.HexToAddress(systemcontracts.ValidatorContract): true,
@@ -216,8 +218,11 @@ type Parlia struct {
genesisHash common.Hash
db ethdb.Database // Database to store and retrieve snapshot checkpoints
recentSnaps *lru.ARCCache // Snapshots for recent block to speed up
signatures *lru.ARCCache // Signatures of recent blocks to speed up mining
recentSnaps *lru.ARCCache // Snapshots for recent block to speed up
signatures *lru.ARCCache // Signatures of recent blocks to speed up mining
recentHeaders *lru.ARCCache //
// Recent headers to check for double signing: key includes block number and miner. value is the block header
// If same key's value already exists for different block header roots then double sign is detected
signer types.Signer
@@ -263,6 +268,10 @@ func New(
if err != nil {
panic(err)
}
recentHeaders, err := lru.NewARC(inMemoryHeaders)
if err != nil {
panic(err)
}
vABIBeforeLuban, err := abi.JSON(strings.NewReader(validatorSetABIBeforeLuban))
if err != nil {
panic(err)
@@ -286,6 +295,7 @@ func New(
db: db,
ethAPI: ethAPI,
recentSnaps: recentSnaps,
recentHeaders: recentHeaders,
signatures: signatures,
validatorSetABIBeforeLuban: vABIBeforeLuban,
validatorSetABI: vABI,
@@ -599,9 +609,7 @@ func (p *Parlia) verifyHeader(chain consensus.ChainHeaderReader, header *types.H
switch {
case header.ParentBeaconRoot != nil:
return fmt.Errorf("invalid parentBeaconRoot, have %#x, expected nil", header.ParentBeaconRoot)
// types.EmptyWithdrawalsHash represents a empty value when EIP-4895 enabled,
// here, EIP-4895 still be disabled, value expected to be `types.EmptyWithdrawalsHash` is only to feet the demand of rlp encode/decode
case header.WithdrawalsHash == nil || *header.WithdrawalsHash != types.EmptyWithdrawalsHash:
case !header.EmptyWithdrawalsHash():
return errors.New("header has wrong WithdrawalsHash")
}
if err := eip4844.VerifyEIP4844Header(parent, header); err != nil {
@@ -811,6 +819,17 @@ func (p *Parlia) verifySeal(chain consensus.ChainHeaderReader, header *types.Hea
return errCoinBaseMisMatch
}
// check for double sign & add to cache
key := proposalKey(*header)
preHash, ok := p.recentHeaders.Get(key)
if ok && preHash != header.Hash() {
doubleSignCounter.Inc(1)
log.Warn("DoubleSign detected", " block", header.Number, " miner", header.Coinbase,
"hash1", preHash.(common.Hash), "hash2", header.Hash())
} else {
p.recentHeaders.Add(key, header.Hash())
}
if _, ok := snap.Validators[signer]; !ok {
return errUnauthorizedValidator(signer.String())
}
@@ -2013,3 +2032,8 @@ func applyMessage(
}
return msg.Gas() - returnGas, err
}
// proposalKey build a key which is a combination of the block number and the proposer address.
func proposalKey(header types.Header) string {
return header.ParentHash.String() + header.Coinbase.String()
}

View File

@@ -66,6 +66,31 @@ func NewBlockValidator(config *params.ChainConfig, blockchain *BlockChain, engin
return validator
}
// ValidateListsInBody validates that UncleHash, WithdrawalsHash, and WithdrawalsHash correspond to the lists in the block body, respectively.
func ValidateListsInBody(block *types.Block) error {
header := block.Header()
if hash := types.CalcUncleHash(block.Uncles()); hash != header.UncleHash {
return fmt.Errorf("uncle root hash mismatch (header value %x, calculated %x)", header.UncleHash, hash)
}
if hash := types.DeriveSha(block.Transactions(), trie.NewStackTrie(nil)); hash != header.TxHash {
return fmt.Errorf("transaction root hash mismatch: have %x, want %x", hash, header.TxHash)
}
// Withdrawals are present after the Shanghai fork.
if header.WithdrawalsHash != nil {
// Withdrawals list must be present in body after Shanghai.
if block.Withdrawals() == nil {
return errors.New("missing withdrawals in block body")
}
if hash := types.DeriveSha(block.Withdrawals(), trie.NewStackTrie(nil)); hash != *header.WithdrawalsHash {
return fmt.Errorf("withdrawals root hash mismatch (header value %x, calculated %x)", *header.WithdrawalsHash, hash)
}
} else if block.Withdrawals() != nil { // Withdrawals turn into empty from nil when BlockBody has Sidecars
// Withdrawals are not allowed prior to shanghai fork
return errors.New("withdrawals present in block body")
}
return nil
}
// ValidateBody validates the given block's uncles and verifies the block
// header's transaction and uncle roots. The headers are assumed to be already
// validated at this point.
@@ -83,31 +108,12 @@ func (v *BlockValidator) ValidateBody(block *types.Block) error {
if err := v.engine.VerifyUncles(v.bc, block); err != nil {
return err
}
if hash := types.CalcUncleHash(block.Uncles()); hash != header.UncleHash {
return fmt.Errorf("uncle root hash mismatch (header value %x, calculated %x)", header.UncleHash, hash)
}
validateFuns := []func() error{
func() error {
if hash := types.DeriveSha(block.Transactions(), trie.NewStackTrie(nil)); hash != header.TxHash {
return fmt.Errorf("transaction root hash mismatch: have %x, want %x", hash, header.TxHash)
}
return nil
return ValidateListsInBody(block)
},
func() error {
// Withdrawals are present after the Shanghai fork.
if !header.EmptyWithdrawalsHash() {
// Withdrawals list must be present in body after Shanghai.
if block.Withdrawals() == nil {
return errors.New("missing withdrawals in block body")
}
if hash := types.DeriveSha(block.Withdrawals(), trie.NewStackTrie(nil)); hash != *header.WithdrawalsHash {
return fmt.Errorf("withdrawals root hash mismatch (header value %x, calculated %x)", *header.WithdrawalsHash, hash)
}
} else if len(block.Withdrawals()) != 0 { // Withdrawals turn into empty from nil when BlockBody has Sidecars
// Withdrawals are not allowed prior to shanghai fork
return errors.New("withdrawals present in block body")
}
// Blob transactions may be present after the Cancun fork.
var blobs int
for i, tx := range block.Transactions() {

View File

@@ -71,6 +71,8 @@ var (
justifiedBlockGauge = metrics.NewRegisteredGauge("chain/head/justified", nil)
finalizedBlockGauge = metrics.NewRegisteredGauge("chain/head/finalized", nil)
blockInsertMgaspsGauge = metrics.NewRegisteredGauge("chain/insert/mgasps", nil)
chainInfoGauge = metrics.NewRegisteredGaugeInfo("chain/info", nil)
accountReadTimer = metrics.NewRegisteredTimer("chain/account/reads", nil)
@@ -165,6 +167,8 @@ type CacheConfig struct {
StateHistory uint64 // Number of blocks from head whose state histories are reserved.
StateScheme string // Scheme used to store ethereum states and merkle tree nodes on top
PathSyncFlush bool // Whether sync flush the trienodebuffer of pathdb to disk.
JournalFilePath string
JournalFile bool
SnapshotNoBuild bool // Whether the background generation is allowed
SnapshotWait bool // Wait for snapshot construction on startup. TODO(karalabe): This is a dirty hack for testing, nuke it
@@ -184,10 +188,12 @@ func (c *CacheConfig) triedbConfig() *triedb.Config {
}
if c.StateScheme == rawdb.PathScheme {
config.PathDB = &pathdb.Config{
SyncFlush: c.PathSyncFlush,
StateHistory: c.StateHistory,
CleanCacheSize: c.TrieCleanLimit * 1024 * 1024,
DirtyCacheSize: c.TrieDirtyLimit * 1024 * 1024,
SyncFlush: c.PathSyncFlush,
StateHistory: c.StateHistory,
CleanCacheSize: c.TrieCleanLimit * 1024 * 1024,
DirtyCacheSize: c.TrieDirtyLimit * 1024 * 1024,
JournalFilePath: c.JournalFilePath,
JournalFile: c.JournalFile,
}
}
return config
@@ -272,6 +278,7 @@ type BlockChain struct {
highestVerifiedHeader atomic.Pointer[types.Header]
currentBlock atomic.Pointer[types.Header] // Current head of the chain
currentSnapBlock atomic.Pointer[types.Header] // Current head of snap-sync
currentFinalBlock atomic.Pointer[types.Header] // Latest (consensus) finalized block
chasingHead atomic.Pointer[types.Header]
bodyCache *lru.Cache[common.Hash, *types.Body]
@@ -412,7 +419,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
// Make sure the state associated with the block is available, or log out
// if there is no available state, waiting for state sync.
head := bc.CurrentBlock()
if !bc.NoTries() && !bc.HasState(head.Root) {
if !bc.HasState(head.Root) {
if head.Number.Uint64() == 0 {
// The genesis state is missing, which is only possible in the path-based
// scheme. This situation occurs when the initial state sync is not finished
@@ -428,7 +435,7 @@ func NewBlockChain(db ethdb.Database, cacheConfig *CacheConfig, genesis *Genesis
if bc.cacheConfig.SnapshotLimit > 0 {
diskRoot = rawdb.ReadSnapshotRoot(bc.db)
}
if bc.triedb.Scheme() == rawdb.PathScheme {
if bc.triedb.Scheme() == rawdb.PathScheme && !bc.NoTries() {
recoverable, _ := bc.triedb.Recoverable(diskRoot)
if !bc.HasState(diskRoot) && !recoverable {
diskRoot = bc.triedb.Head()
@@ -662,7 +669,7 @@ func (bc *BlockChain) cacheBlock(hash common.Hash, block *types.Block) {
// into node seamlessly.
func (bc *BlockChain) empty() bool {
genesis := bc.genesisBlock.Hash()
for _, hash := range []common.Hash{rawdb.ReadHeadBlockHash(bc.db), rawdb.ReadHeadHeaderHash(bc.db), rawdb.ReadHeadFastBlockHash(bc.db)} {
for _, hash := range []common.Hash{rawdb.ReadHeadBlockHash(bc.db.BlockStore()), rawdb.ReadHeadHeaderHash(bc.db.BlockStore()), rawdb.ReadHeadFastBlockHash(bc.db)} {
if hash != genesis {
return false
}
@@ -698,7 +705,7 @@ func (bc *BlockChain) getFinalizedNumber(header *types.Header) uint64 {
// assumes that the chain manager mutex is held.
func (bc *BlockChain) loadLastState() error {
// Restore the last known head block
head := rawdb.ReadHeadBlockHash(bc.db)
head := rawdb.ReadHeadBlockHash(bc.db.BlockStore())
if head == (common.Hash{}) {
// Corrupt or empty database, init from scratch
log.Warn("Empty database, resetting chain")
@@ -720,7 +727,7 @@ func (bc *BlockChain) loadLastState() error {
// Restore the last known head header
headHeader := headBlock.Header()
if head := rawdb.ReadHeadHeaderHash(bc.db); head != (common.Hash{}) {
if head := rawdb.ReadHeadHeaderHash(bc.db.BlockStore()); head != (common.Hash{}) {
if header := bc.GetHeaderByHash(head); header != nil {
headHeader = header
}
@@ -991,12 +998,23 @@ func (bc *BlockChain) rewindPathHead(head *types.Header, root common.Hash) (*typ
// then block number zero is returned, indicating that snapshot recovery is disabled
// and the whole snapshot should be auto-generated in case of head mismatch.
func (bc *BlockChain) rewindHead(head *types.Header, root common.Hash) (*types.Header, uint64) {
if bc.triedb.Scheme() == rawdb.PathScheme {
if bc.triedb.Scheme() == rawdb.PathScheme && !bc.NoTries() {
return bc.rewindPathHead(head, root)
}
return bc.rewindHashHead(head, root)
}
// SetFinalized sets the finalized block.
// This function differs slightly from Ethereum; we fine-tune it through the outer-layer setting finalizedBlockGauge.
func (bc *BlockChain) SetFinalized(header *types.Header) {
bc.currentFinalBlock.Store(header)
if header != nil {
rawdb.WriteFinalizedBlockHash(bc.db.BlockStore(), header.Hash())
} else {
rawdb.WriteFinalizedBlockHash(bc.db.BlockStore(), common.Hash{})
}
}
// setHeadBeyondRoot rewinds the local chain to a new head with the extra condition
// that the rewind must pass the specified state root. This method is meant to be
// used when rewinding with snapshots enabled to ensure that we go back further than
@@ -1028,6 +1046,19 @@ func (bc *BlockChain) setHeadBeyondRoot(head uint64, time uint64, root common.Ha
// block. Note, depth equality is permitted to allow using SetHead as a
// chain reparation mechanism without deleting any data!
if currentBlock := bc.CurrentBlock(); currentBlock != nil && header.Number.Uint64() <= currentBlock.Number.Uint64() {
// load bc.snaps for the judge `HasState`
if bc.NoTries() {
if bc.cacheConfig.SnapshotLimit > 0 {
snapconfig := snapshot.Config{
CacheSize: bc.cacheConfig.SnapshotLimit,
NoBuild: bc.cacheConfig.SnapshotNoBuild,
AsyncBuild: !bc.cacheConfig.SnapshotWait,
}
bc.snaps, _ = snapshot.New(snapconfig, bc.db, bc.triedb, header.Root, int(bc.cacheConfig.TriesInMemory), bc.NoTries())
}
defer func() { bc.snaps = nil }()
}
var newHeadBlock *types.Header
newHeadBlock, rootNumber = bc.rewindHead(header, root)
rawdb.WriteHeadBlockHash(db, newHeadBlock.Hash())
@@ -1129,6 +1160,11 @@ func (bc *BlockChain) setHeadBeyondRoot(head uint64, time uint64, root common.Ha
bc.txLookupCache.Purge()
bc.futureBlocks.Purge()
if finalized := bc.CurrentFinalBlock(); finalized != nil && head < finalized.Number.Uint64() {
log.Error("SetHead invalidated finalized block")
bc.SetFinalized(nil)
}
return rootNumber, bc.loadLastState()
}
@@ -1197,10 +1233,10 @@ func (bc *BlockChain) ResetWithGenesisBlock(genesis *types.Block) error {
defer bc.chainmu.Unlock()
// Prepare the genesis block and reinitialise the chain
batch := bc.db.NewBatch()
rawdb.WriteTd(batch, genesis.Hash(), genesis.NumberU64(), genesis.Difficulty())
rawdb.WriteBlock(batch, genesis)
if err := batch.Write(); err != nil {
blockBatch := bc.db.BlockStore().NewBatch()
rawdb.WriteTd(blockBatch, genesis.Hash(), genesis.NumberU64(), genesis.Difficulty())
rawdb.WriteBlock(blockBatch, genesis)
if err := blockBatch.Write(); err != nil {
log.Crit("Failed to write genesis block", "err", err)
}
bc.writeHeadBlock(genesis)
@@ -1263,12 +1299,13 @@ func (bc *BlockChain) ExportN(w io.Writer, first uint64, last uint64) error {
// Note, this function assumes that the `mu` mutex is held!
func (bc *BlockChain) writeHeadBlock(block *types.Block) {
// Add the block to the canonical chain number scheme and mark as the head
rawdb.WriteCanonicalHash(bc.db.BlockStore(), block.Hash(), block.NumberU64())
rawdb.WriteHeadHeaderHash(bc.db.BlockStore(), block.Hash())
rawdb.WriteHeadBlockHash(bc.db.BlockStore(), block.Hash())
batch := bc.db.NewBatch()
rawdb.WriteHeadHeaderHash(batch, block.Hash())
rawdb.WriteHeadFastBlockHash(batch, block.Hash())
rawdb.WriteCanonicalHash(batch, block.Hash(), block.NumberU64())
rawdb.WriteTxLookupEntriesByBlock(batch, block)
rawdb.WriteHeadBlockHash(batch, block.Hash())
// Flush the whole batch into the disk, exit the node if failed
if err := batch.Write(); err != nil {
@@ -1354,7 +1391,7 @@ func (bc *BlockChain) Stop() {
} else {
rawdb.WriteSafePointBlockNumber(bc.db, recent.NumberU64())
once.Do(func() {
rawdb.WriteHeadBlockHash(bc.db, recent.Hash())
rawdb.WriteHeadBlockHash(bc.db.BlockStore(), recent.Hash())
})
}
}
@@ -1555,24 +1592,24 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
// Delete block data from the main database.
var (
batch = bc.db.NewBatch()
canonHashes = make(map[common.Hash]struct{})
blockBatch = bc.db.BlockStore().NewBatch()
)
for _, block := range blockChain {
canonHashes[block.Hash()] = struct{}{}
if block.NumberU64() == 0 {
continue
}
rawdb.DeleteCanonicalHash(batch, block.NumberU64())
rawdb.DeleteBlockWithoutNumber(batch, block.Hash(), block.NumberU64())
rawdb.DeleteCanonicalHash(blockBatch, block.NumberU64())
rawdb.DeleteBlockWithoutNumber(blockBatch, block.Hash(), block.NumberU64())
}
// Delete side chain hash-to-number mappings.
for _, nh := range rawdb.ReadAllHashesInRange(bc.db, first.NumberU64(), last.NumberU64()) {
if _, canon := canonHashes[nh.Hash]; !canon {
rawdb.DeleteHeader(batch, nh.Hash, nh.Number)
rawdb.DeleteHeader(blockBatch, nh.Hash, nh.Number)
}
}
if err := batch.Write(); err != nil {
if err := blockBatch.Write(); err != nil {
return 0, err
}
stats.processed += int32(len(blockChain))
@@ -1584,6 +1621,7 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
var (
skipPresenceCheck = false
batch = bc.db.NewBatch()
blockBatch = bc.db.BlockStore().NewBatch()
)
for i, block := range blockChain {
// Short circuit insertion if shutting down or processing failed
@@ -1607,10 +1645,10 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
}
}
// Write all the data out into the database
rawdb.WriteBody(batch, block.Hash(), block.NumberU64(), block.Body())
rawdb.WriteReceipts(batch, block.Hash(), block.NumberU64(), receiptChain[i])
rawdb.WriteBody(blockBatch, block.Hash(), block.NumberU64(), block.Body())
rawdb.WriteReceipts(blockBatch, block.Hash(), block.NumberU64(), receiptChain[i])
if bc.chainConfig.IsCancun(block.Number(), block.Time()) {
rawdb.WriteBlobSidecars(batch, block.Hash(), block.NumberU64(), block.Sidecars())
rawdb.WriteBlobSidecars(blockBatch, block.Hash(), block.NumberU64(), block.Sidecars())
}
// Write everything belongs to the blocks into the database. So that
@@ -1623,6 +1661,13 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
size += int64(batch.ValueSize())
batch.Reset()
}
if blockBatch.ValueSize() >= ethdb.IdealBatchSize {
if err := blockBatch.Write(); err != nil {
return 0, err
}
size += int64(blockBatch.ValueSize())
blockBatch.Reset()
}
stats.processed++
}
// Write everything belongs to the blocks into the database. So that
@@ -1634,6 +1679,12 @@ func (bc *BlockChain) InsertReceiptChain(blockChain types.Blocks, receiptChain [
return 0, err
}
}
if blockBatch.ValueSize() > 0 {
size += int64(blockBatch.ValueSize())
if err := blockBatch.Write(); err != nil {
return 0, err
}
}
updateHead(blockChain[len(blockChain)-1])
return 0, nil
}
@@ -1678,14 +1729,14 @@ func (bc *BlockChain) writeBlockWithoutState(block *types.Block, td *big.Int) (e
if bc.insertStopped() {
return errInsertionInterrupted
}
batch := bc.db.NewBatch()
rawdb.WriteTd(batch, block.Hash(), block.NumberU64(), td)
rawdb.WriteBlock(batch, block)
blockBatch := bc.db.BlockStore().NewBatch()
rawdb.WriteTd(blockBatch, block.Hash(), block.NumberU64(), td)
rawdb.WriteBlock(blockBatch, block)
// if cancun is enabled, here need to write sidecars too
if bc.chainConfig.IsCancun(block.Number(), block.Time()) {
rawdb.WriteBlobSidecars(batch, block.Hash(), block.NumberU64(), block.Sidecars())
rawdb.WriteBlobSidecars(blockBatch, block.Hash(), block.NumberU64(), block.Sidecars())
}
if err := batch.Write(); err != nil {
if err := blockBatch.Write(); err != nil {
log.Crit("Failed to write block into disk", "err", err)
}
return nil
@@ -1723,7 +1774,8 @@ func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts []*types.
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
blockBatch := bc.db.NewBatch()
rawdb.WritePreimages(bc.db, state.Preimages())
blockBatch := bc.db.BlockStore().NewBatch()
rawdb.WriteTd(blockBatch, block.Hash(), block.NumberU64(), externTd)
rawdb.WriteBlock(blockBatch, block)
rawdb.WriteReceipts(blockBatch, block.Hash(), block.NumberU64(), receipts)
@@ -1892,12 +1944,16 @@ func (bc *BlockChain) writeBlockAndSetHead(block *types.Block, receipts []*types
// canonical blocks. Avoid firing too many ChainHeadEvents,
// we will fire an accumulated ChainHeadEvent and disable fire
// event here.
var finalizedHeader *types.Header
if posa, ok := bc.Engine().(consensus.PoSA); ok {
if finalizedHeader = posa.GetFinalizedHeader(bc, block.Header()); finalizedHeader != nil {
bc.SetFinalized(finalizedHeader)
}
}
if emitHeadEvent {
bc.chainHeadFeed.Send(ChainHeadEvent{Block: block})
if posa, ok := bc.Engine().(consensus.PoSA); ok {
if finalizedHeader := posa.GetFinalizedHeader(bc, block.Header()); finalizedHeader != nil {
bc.finalizedHeaderFeed.Send(FinalizedHeaderEvent{finalizedHeader})
}
if finalizedHeader != nil {
bc.finalizedHeaderFeed.Send(FinalizedHeaderEvent{finalizedHeader})
}
}
} else {
@@ -2136,7 +2192,7 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
// state, but if it's this special case here(skip reexecution) we will lose
// the empty receipt entry.
if len(block.Transactions()) == 0 {
rawdb.WriteReceipts(bc.db, block.Hash(), block.NumberU64(), nil)
rawdb.WriteReceipts(bc.db.BlockStore(), block.Hash(), block.NumberU64(), nil)
} else {
log.Error("Please file an issue, skip known block execution without receipt",
"hash", block.Hash(), "number", block.NumberU64())
@@ -2681,6 +2737,7 @@ func (bc *BlockChain) reorg(oldHead *types.Header, newHead *types.Block) error {
var (
indexesBatch = bc.db.NewBatch()
diffs = types.HashDifference(deletedTxs, addedTxs)
blockBatch = bc.db.BlockStore().NewBatch()
)
for _, tx := range diffs {
rawdb.DeleteTxLookupEntry(indexesBatch, tx)
@@ -2697,11 +2754,14 @@ func (bc *BlockChain) reorg(oldHead *types.Header, newHead *types.Block) error {
if hash == (common.Hash{}) {
break
}
rawdb.DeleteCanonicalHash(indexesBatch, i)
rawdb.DeleteCanonicalHash(blockBatch, i)
}
if err := indexesBatch.Write(); err != nil {
log.Crit("Failed to delete useless indexes", "err", err)
}
if err := blockBatch.Write(); err != nil {
log.Crit("Failed to delete useless indexes use block batch", "err", err)
}
// Send out events for logs from the old canon chain, and 'reborn'
// logs from the new canon chain. The number of logs can be very

View File

@@ -48,18 +48,23 @@ func (st *insertStats) report(chain []*types.Block, index int, snapDiffItems, sn
// If we're at the last block of the batch or report period reached, log
if index == len(chain)-1 || elapsed >= statsReportLimit {
// Count the number of transactions in this segment
var txs int
var txs, blobs int
for _, block := range chain[st.lastIndex : index+1] {
txs += len(block.Transactions())
for _, sidecar := range block.Sidecars() {
blobs += len(sidecar.Blobs)
}
}
end := chain[index]
// Assemble the log context and send it to the logger
mgasps := float64(st.usedGas) * 1000 / float64(elapsed)
context := []interface{}{
"number", end.Number(), "hash", end.Hash(), "miner", end.Coinbase(),
"blocks", st.processed, "txs", txs, "mgas", float64(st.usedGas) / 1000000,
"elapsed", common.PrettyDuration(elapsed), "mgasps", float64(st.usedGas) * 1000 / float64(elapsed),
"blocks", st.processed, "txs", txs, "blobs", blobs, "mgas", float64(st.usedGas) / 1000000,
"elapsed", common.PrettyDuration(elapsed), "mgasps", mgasps,
}
blockInsertMgaspsGauge.Update(int64(mgasps))
if timestamp := time.Unix(int64(end.Time()), 0); time.Since(timestamp) > time.Minute {
context = append(context, []interface{}{"age", common.PrettyAge(timestamp)}...)
}

View File

@@ -231,7 +231,7 @@ func (bc *BlockChain) GetReceiptsByHash(hash common.Hash) types.Receipts {
if receipts, ok := bc.receiptsCache.Get(hash); ok {
return receipts
}
number := rawdb.ReadHeaderNumber(bc.db, hash)
number := rawdb.ReadHeaderNumber(bc.db.BlockStore(), hash)
if number == nil {
return nil
}
@@ -396,7 +396,20 @@ func (bc *BlockChain) State() (*state.StateDB, error) {
// StateAt returns a new mutable state based on a particular point in time.
func (bc *BlockChain) StateAt(root common.Hash) (*state.StateDB, error) {
return state.New(root, bc.stateCache, bc.snaps)
stateDb, err := state.New(root, bc.stateCache, bc.snaps)
if err != nil {
return nil, err
}
// If there's no trie and the specified snapshot is not available, getting
// any state will by default return nil.
// Instead of that, it will be more useful to return an error to indicate
// the state is not available.
if stateDb.NoTrie() && stateDb.GetSnap() == nil {
return nil, errors.New("state is not available")
}
return stateDb, err
}
// Config retrieves the chain's fork configuration.
@@ -498,3 +511,12 @@ func (bc *BlockChain) SubscribeBlockProcessingEvent(ch chan<- bool) event.Subscr
func (bc *BlockChain) SubscribeFinalizedHeaderEvent(ch chan<- FinalizedHeaderEvent) event.Subscription {
return bc.scope.Track(bc.finalizedHeaderFeed.Subscribe(ch))
}
// AncientTail retrieves the tail the ancients blocks
func (bc *BlockChain) AncientTail() (uint64, error) {
tail, err := bc.db.Tail()
if err != nil {
return 0, err
}
return tail, nil
}

View File

@@ -1832,10 +1832,14 @@ func testRepairWithScheme(t *testing.T, tt *rewindTest, snapshots bool, scheme s
}
// Force run a freeze cycle
type freezer interface {
Freeze(threshold uint64) error
Freeze() error
Ancients() (uint64, error)
}
db.(freezer).Freeze(tt.freezeThreshold)
if tt.freezeThreshold < uint64(tt.canonicalBlocks) {
final := uint64(tt.canonicalBlocks) - tt.freezeThreshold
chain.SetFinalized(canonblocks[int(final)-1].Header())
}
db.(freezer).Freeze()
// Set the simulated pivot block
if tt.pivotBlock != nil {

View File

@@ -2045,10 +2045,14 @@ func testSetHeadWithScheme(t *testing.T, tt *rewindTest, snapshots bool, scheme
// Force run a freeze cycle
type freezer interface {
Freeze(threshold uint64) error
Freeze() error
Ancients() (uint64, error)
}
db.(freezer).Freeze(tt.freezeThreshold)
if tt.freezeThreshold < uint64(tt.canonicalBlocks) {
final := uint64(tt.canonicalBlocks) - tt.freezeThreshold
chain.SetFinalized(canonblocks[int(final)-1].Header())
}
db.(freezer).Freeze()
// Set the simulated pivot block
if tt.pivotBlock != nil {

View File

@@ -227,8 +227,8 @@ func (c *ChainIndexer) eventLoop(currentHeader *types.Header, events chan ChainH
// Reorg to the common ancestor if needed (might not exist in light sync mode, skip reorg then)
// TODO(karalabe, zsfelfoldi): This seems a bit brittle, can we detect this case explicitly?
if rawdb.ReadCanonicalHash(c.chainDb, prevHeader.Number.Uint64()) != prevHash {
if h := rawdb.FindCommonAncestor(c.chainDb, prevHeader, header); h != nil {
if rawdb.ReadCanonicalHash(c.chainDb.BlockStore(), prevHeader.Number.Uint64()) != prevHash {
if h := rawdb.FindCommonAncestor(c.chainDb.BlockStore(), prevHeader, header); h != nil {
c.newHead(h.Number.Uint64(), true)
}
}

View File

@@ -372,6 +372,9 @@ func GenerateChain(config *params.ChainConfig, parent *types.Block, engine conse
if err != nil {
panic(err)
}
if block.Header().EmptyWithdrawalsHash() {
block = block.WithWithdrawals(make([]*types.Withdrawal, 0))
}
if config.IsCancun(block.Number(), block.Time()) {
for _, s := range b.sidecars {
s.BlockNumber = block.Number()

View File

@@ -5,6 +5,9 @@ import (
"errors"
"fmt"
"sync"
"time"
"github.com/ethereum/go-ethereum/metrics"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/gopool"
@@ -14,6 +17,10 @@ import (
"github.com/ethereum/go-ethereum/params"
)
var (
daCheckTimer = metrics.NewRegisteredTimer("chain/dacheck", nil)
)
// validateBlobSidecar it is same as validateBlobSidecar in core/txpool/validation.go
func validateBlobSidecar(hashes []common.Hash, sidecar *types.BlobSidecar) error {
if len(sidecar.Blobs) != len(hashes) {
@@ -46,6 +53,10 @@ func validateBlobSidecar(hashes []common.Hash, sidecar *types.BlobSidecar) error
// IsDataAvailable it checks that the blobTx block has available blob data
func IsDataAvailable(chain consensus.ChainHeaderReader, block *types.Block) (err error) {
defer func(start time.Time) {
daCheckTimer.Update(time.Since(start))
}(time.Now())
// refer logic in ValidateBody
if !chain.Config().IsCancun(block.Number(), block.Time()) {
if block.Sidecars() != nil {

View File

@@ -1,9 +1,12 @@
package core
import (
"crypto/rand"
"math/big"
"testing"
"github.com/consensys/gnark-crypto/ecc/bls12-381/fr"
gokzg4844 "github.com/crate-crypto/go-kzg-4844"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/crypto/kzg4844"
@@ -247,6 +250,49 @@ func TestCheckDataAvailableInBatch(t *testing.T) {
}
}
func BenchmarkEmptySidecarDAChecking(b *testing.B) {
hr := NewMockDAHeaderReader(params.ParliaTestChainConfig)
block := types.NewBlockWithHeader(&types.Header{
Number: big.NewInt(1),
}).WithBody(types.Transactions{
createMockDATx(hr.Config(), emptySidecar()),
createMockDATx(hr.Config(), emptySidecar()),
createMockDATx(hr.Config(), emptySidecar()),
createMockDATx(hr.Config(), emptySidecar()),
createMockDATx(hr.Config(), emptySidecar()),
createMockDATx(hr.Config(), emptySidecar()),
}, nil)
block = block.WithSidecars(collectBlobsFromTxs(block.Header(), block.Transactions()))
b.ResetTimer()
for i := 0; i < b.N; i++ {
IsDataAvailable(hr, block)
}
}
func BenchmarkRandomSidecarDAChecking(b *testing.B) {
hr := NewMockDAHeaderReader(params.ParliaTestChainConfig)
const count = 10
blocks := make([]*types.Block, count)
for i := 0; i < len(blocks); i++ {
block := types.NewBlockWithHeader(&types.Header{
Number: big.NewInt(1),
}).WithBody(types.Transactions{
createMockDATx(hr.Config(), randomSidecar()),
createMockDATx(hr.Config(), randomSidecar()),
createMockDATx(hr.Config(), randomSidecar()),
createMockDATx(hr.Config(), randomSidecar()),
createMockDATx(hr.Config(), randomSidecar()),
createMockDATx(hr.Config(), randomSidecar()),
}, nil)
block = block.WithSidecars(collectBlobsFromTxs(block.Header(), block.Transactions()))
blocks[i] = block
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
IsDataAvailable(hr, blocks[i%count])
}
}
func collectBlobsFromTxs(header *types.Header, txs types.Transactions) types.BlobSidecars {
sidecars := make(types.BlobSidecars, 0, len(txs))
for i, tx := range txs {
@@ -348,3 +394,43 @@ func createMockDATx(config *params.ChainConfig, sidecar *types.BlobTxSidecar) *t
}
return types.NewTx(tx)
}
func randFieldElement() [32]byte {
bytes := make([]byte, 32)
_, err := rand.Read(bytes)
if err != nil {
panic("failed to get random field element")
}
var r fr.Element
r.SetBytes(bytes)
return gokzg4844.SerializeScalar(r)
}
func randBlob() kzg4844.Blob {
var blob kzg4844.Blob
for i := 0; i < len(blob); i += gokzg4844.SerializedScalarSize {
fieldElementBytes := randFieldElement()
copy(blob[i:i+gokzg4844.SerializedScalarSize], fieldElementBytes[:])
}
return blob
}
func randomSidecar() *types.BlobTxSidecar {
blob := randBlob()
commitment, _ := kzg4844.BlobToCommitment(blob)
proof, _ := kzg4844.ComputeBlobProof(blob, commitment)
return &types.BlobTxSidecar{
Blobs: []kzg4844.Blob{blob},
Commitments: []kzg4844.Commitment{commitment},
Proofs: []kzg4844.Proof{proof},
}
}
func emptySidecar() *types.BlobTxSidecar {
return &types.BlobTxSidecar{
Blobs: []kzg4844.Blob{emptyBlob},
Commitments: []kzg4844.Commitment{emptyBlobCommit},
Proofs: []kzg4844.Proof{emptyBlobProof},
}
}

View File

@@ -216,10 +216,9 @@ func (e *GenesisMismatchError) Error() string {
// ChainOverrides contains the changes to chain config
// Typically, these modifications involve hardforks that are not enabled on the BSC mainnet, intended for testing purposes.
type ChainOverrides struct {
OverrideCancun *uint64
OverrideVerkle *uint64
OverrideFeynman *uint64
OverrideFeynmanFix *uint64
OverrideCancun *uint64
OverrideHaber *uint64
OverrideVerkle *uint64
}
// SetupGenesisBlock writes or updates the genesis block in db.
@@ -248,15 +247,12 @@ func SetupGenesisBlockWithOverride(db ethdb.Database, triedb *triedb.Database, g
if overrides != nil && overrides.OverrideCancun != nil {
config.CancunTime = overrides.OverrideCancun
}
if overrides != nil && overrides.OverrideHaber != nil {
config.HaberTime = overrides.OverrideHaber
}
if overrides != nil && overrides.OverrideVerkle != nil {
config.VerkleTime = overrides.OverrideVerkle
}
if overrides != nil && overrides.OverrideFeynman != nil {
config.FeynmanTime = overrides.OverrideFeynman
}
if overrides != nil && overrides.OverrideFeynmanFix != nil {
config.FeynmanFixTime = overrides.OverrideFeynmanFix
}
}
}
// Just commit the new block if there is no stored genesis block.
@@ -493,13 +489,13 @@ func (g *Genesis) Commit(db ethdb.Database, triedb *triedb.Database) (*types.Blo
if err := flushAlloc(&g.Alloc, db, triedb, block.Hash()); err != nil {
return nil, err
}
rawdb.WriteTd(db, block.Hash(), block.NumberU64(), block.Difficulty())
rawdb.WriteBlock(db, block)
rawdb.WriteReceipts(db, block.Hash(), block.NumberU64(), nil)
rawdb.WriteCanonicalHash(db, block.Hash(), block.NumberU64())
rawdb.WriteHeadBlockHash(db, block.Hash())
rawdb.WriteTd(db.BlockStore(), block.Hash(), block.NumberU64(), block.Difficulty())
rawdb.WriteBlock(db.BlockStore(), block)
rawdb.WriteReceipts(db.BlockStore(), block.Hash(), block.NumberU64(), nil)
rawdb.WriteCanonicalHash(db.BlockStore(), block.Hash(), block.NumberU64())
rawdb.WriteHeadBlockHash(db.BlockStore(), block.Hash())
rawdb.WriteHeadFastBlockHash(db, block.Hash())
rawdb.WriteHeadHeaderHash(db, block.Hash())
rawdb.WriteHeadHeaderHash(db.BlockStore(), block.Hash())
rawdb.WriteChainConfig(db, block.Hash(), config)
return block, nil
}

View File

@@ -97,7 +97,7 @@ func NewHeaderChain(chainDb ethdb.Database, config *params.ChainConfig, engine c
return nil, ErrNoGenesis
}
hc.currentHeader.Store(hc.genesisHeader)
if head := rawdb.ReadHeadBlockHash(chainDb); head != (common.Hash{}) {
if head := rawdb.ReadHeadBlockHash(chainDb.BlockStore()); head != (common.Hash{}) {
if chead := hc.GetHeaderByHash(head); chead != nil {
hc.currentHeader.Store(chead)
}
@@ -144,7 +144,7 @@ func (hc *HeaderChain) GetBlockNumber(hash common.Hash) *uint64 {
if cached, ok := hc.numberCache.Get(hash); ok {
return &cached
}
number := rawdb.ReadHeaderNumber(hc.chainDb, hash)
number := rawdb.ReadHeaderNumber(hc.chainDb.BlockStore(), hash)
if number != nil {
hc.numberCache.Add(hash, *number)
}
@@ -172,9 +172,9 @@ func (hc *HeaderChain) Reorg(headers []*types.Header) error {
// pile them onto the existing chain. Otherwise, do the necessary
// reorgs.
var (
first = headers[0]
last = headers[len(headers)-1]
batch = hc.chainDb.NewBatch()
first = headers[0]
last = headers[len(headers)-1]
blockBatch = hc.chainDb.BlockStore().NewBatch()
)
if first.ParentHash != hc.currentHeaderHash {
// Delete any canonical number assignments above the new head
@@ -183,7 +183,7 @@ func (hc *HeaderChain) Reorg(headers []*types.Header) error {
if hash == (common.Hash{}) {
break
}
rawdb.DeleteCanonicalHash(batch, i)
rawdb.DeleteCanonicalHash(blockBatch, i)
}
// Overwrite any stale canonical number assignments, going
// backwards from the first header in this import until the
@@ -194,7 +194,7 @@ func (hc *HeaderChain) Reorg(headers []*types.Header) error {
headHash = header.Hash()
)
for rawdb.ReadCanonicalHash(hc.chainDb, headNumber) != headHash {
rawdb.WriteCanonicalHash(batch, headHash, headNumber)
rawdb.WriteCanonicalHash(blockBatch, headHash, headNumber)
if headNumber == 0 {
break // It shouldn't be reached
}
@@ -209,16 +209,16 @@ func (hc *HeaderChain) Reorg(headers []*types.Header) error {
for i := 0; i < len(headers)-1; i++ {
hash := headers[i+1].ParentHash // Save some extra hashing
num := headers[i].Number.Uint64()
rawdb.WriteCanonicalHash(batch, hash, num)
rawdb.WriteHeadHeaderHash(batch, hash)
rawdb.WriteCanonicalHash(blockBatch, hash, num)
rawdb.WriteHeadHeaderHash(blockBatch, hash)
}
// Write the last header
hash := headers[len(headers)-1].Hash()
num := headers[len(headers)-1].Number.Uint64()
rawdb.WriteCanonicalHash(batch, hash, num)
rawdb.WriteHeadHeaderHash(batch, hash)
rawdb.WriteCanonicalHash(blockBatch, hash, num)
rawdb.WriteHeadHeaderHash(blockBatch, hash)
if err := batch.Write(); err != nil {
if err := blockBatch.Write(); err != nil {
return err
}
// Last step update all in-memory head header markers
@@ -244,7 +244,7 @@ func (hc *HeaderChain) WriteHeaders(headers []*types.Header) (int, error) {
newTD = new(big.Int).Set(ptd) // Total difficulty of inserted chain
inserted []rawdb.NumberHash // Ephemeral lookup of number/hash for the chain
parentKnown = true // Set to true to force hc.HasHeader check the first iteration
batch = hc.chainDb.NewBatch()
blockBatch = hc.chainDb.BlockStore().NewBatch()
)
for i, header := range headers {
var hash common.Hash
@@ -264,10 +264,10 @@ func (hc *HeaderChain) WriteHeaders(headers []*types.Header) (int, error) {
alreadyKnown := parentKnown && hc.HasHeader(hash, number)
if !alreadyKnown {
// Irrelevant of the canonical status, write the TD and header to the database.
rawdb.WriteTd(batch, hash, number, newTD)
rawdb.WriteTd(blockBatch, hash, number, newTD)
hc.tdCache.Add(hash, new(big.Int).Set(newTD))
rawdb.WriteHeader(batch, header)
rawdb.WriteHeader(blockBatch, header)
inserted = append(inserted, rawdb.NumberHash{Number: number, Hash: hash})
hc.headerCache.Add(hash, header)
hc.numberCache.Add(hash, number)
@@ -280,7 +280,7 @@ func (hc *HeaderChain) WriteHeaders(headers []*types.Header) (int, error) {
return 0, errors.New("aborted")
}
// Commit to disk!
if err := batch.Write(); err != nil {
if err := blockBatch.Write(); err != nil {
log.Crit("Failed to write headers", "error", err)
}
return len(inserted), nil
@@ -642,7 +642,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
}
var (
parentHash common.Hash
batch = hc.chainDb.NewBatch()
blockBatch = hc.chainDb.BlockStore().NewBatch()
origin = true
)
done := func(header *types.Header) bool {
@@ -677,7 +677,7 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
}
}
// Update head header then.
rawdb.WriteHeadHeaderHash(markerBatch, parentHash)
rawdb.WriteHeadHeaderHash(hc.chainDb.BlockStore(), parentHash)
if err := markerBatch.Write(); err != nil {
log.Crit("Failed to update chain markers", "error", err)
}
@@ -708,16 +708,16 @@ func (hc *HeaderChain) setHead(headBlock uint64, headTime uint64, updateFn Updat
}
for _, hash := range hashes {
if delFn != nil {
delFn(batch, hash, num)
delFn(blockBatch, hash, num)
}
rawdb.DeleteHeader(batch, hash, num)
rawdb.DeleteTd(batch, hash, num)
rawdb.DeleteHeader(blockBatch, hash, num)
rawdb.DeleteTd(blockBatch, hash, num)
}
rawdb.DeleteCanonicalHash(batch, num)
rawdb.DeleteCanonicalHash(blockBatch, num)
}
}
// Flush all accumulated deletions.
if err := batch.Write(); err != nil {
if err := blockBatch.Write(); err != nil {
log.Crit("Failed to rewind block", "error", err)
}
// Clear out any stale content from the caches

View File

@@ -37,11 +37,11 @@ import (
// ReadCanonicalHash retrieves the hash assigned to a canonical block number.
func ReadCanonicalHash(db ethdb.Reader, number uint64) common.Hash {
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
data, _ = reader.Ancient(ChainFreezerHashTable, number)
if len(data) == 0 {
// Get it by hash from leveldb
data, _ = db.Get(headerHashKey(number))
data, _ = db.BlockStoreReader().Get(headerHashKey(number))
}
return nil
})
@@ -217,6 +217,22 @@ func WriteHeadFastBlockHash(db ethdb.KeyValueWriter, hash common.Hash) {
}
}
// ReadFinalizedBlockHash retrieves the hash of the finalized block.
func ReadFinalizedBlockHash(db ethdb.KeyValueReader) common.Hash {
data, _ := db.Get(headFinalizedBlockKey)
if len(data) == 0 {
return common.Hash{}
}
return common.BytesToHash(data)
}
// WriteFinalizedBlockHash stores the hash of the finalized block.
func WriteFinalizedBlockHash(db ethdb.KeyValueWriter, hash common.Hash) {
if err := db.Put(headFinalizedBlockKey, hash.Bytes()); err != nil {
log.Crit("Failed to store last finalized block's hash", "err", err)
}
}
// ReadLastPivotNumber retrieves the number of the last pivot block. If the node
// full synced, the last pivot will always be nil.
func ReadLastPivotNumber(db ethdb.KeyValueReader) *uint64 {
@@ -287,7 +303,7 @@ func ReadHeaderRange(db ethdb.Reader, number uint64, count uint64) []rlp.RawValu
// If we need to read live blocks, we need to figure out the hash first
hash := ReadCanonicalHash(db, number)
for ; i >= limit && count > 0; i-- {
if data, _ := db.Get(headerKey(i, hash)); len(data) > 0 {
if data, _ := db.BlockStoreReader().Get(headerKey(i, hash)); len(data) > 0 {
rlpHeaders = append(rlpHeaders, data)
// Get the parent hash for next query
hash = types.HeaderParentHashFromRLP(data)
@@ -300,8 +316,8 @@ func ReadHeaderRange(db ethdb.Reader, number uint64, count uint64) []rlp.RawValu
if count == 0 {
return rlpHeaders
}
// read remaining from ancients
data, err := db.AncientRange(ChainFreezerHeaderTable, i+1-count, count, 0)
// read remaining from ancients, cap at 2M
data, err := db.AncientRange(ChainFreezerHeaderTable, i+1-count, count, 2*1024*1024)
if err != nil {
log.Error("Failed to read headers from freezer", "err", err)
return rlpHeaders
@@ -320,7 +336,7 @@ func ReadHeaderRange(db ethdb.Reader, number uint64, count uint64) []rlp.RawValu
// ReadHeaderRLP retrieves a block header in its raw RLP database encoding.
func ReadHeaderRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue {
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
// First try to look up the data in ancient database. Extra hash
// comparison is necessary since ancient database only maintains
// the canonical data.
@@ -329,7 +345,7 @@ func ReadHeaderRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValu
return nil
}
// If not, try reading from leveldb
data, _ = db.Get(headerKey(number, hash))
data, _ = db.BlockStoreReader().Get(headerKey(number, hash))
return nil
})
return data
@@ -337,10 +353,10 @@ func ReadHeaderRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValu
// HasHeader verifies the existence of a block header corresponding to the hash.
func HasHeader(db ethdb.Reader, hash common.Hash, number uint64) bool {
if isCanon(db, number, hash) {
if isCanon(db.BlockStoreReader(), number, hash) {
return true
}
if has, err := db.Has(headerKey(number, hash)); !has || err != nil {
if has, err := db.BlockStoreReader().Has(headerKey(number, hash)); !has || err != nil {
return false
}
return true
@@ -427,14 +443,14 @@ func ReadBodyRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue
// comparison is necessary since ancient database only maintains
// the canonical data.
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
// Check if the data is in ancients
if isCanon(reader, number, hash) {
data, _ = reader.Ancient(ChainFreezerBodiesTable, number)
return nil
}
// If not, try reading from leveldb
data, _ = db.Get(blockBodyKey(number, hash))
data, _ = db.BlockStoreReader().Get(blockBodyKey(number, hash))
return nil
})
return data
@@ -444,7 +460,7 @@ func ReadBodyRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue
// block at number, in RLP encoding.
func ReadCanonicalBodyRLP(db ethdb.Reader, number uint64) rlp.RawValue {
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
data, _ = reader.Ancient(ChainFreezerBodiesTable, number)
if len(data) > 0 {
return nil
@@ -453,7 +469,7 @@ func ReadCanonicalBodyRLP(db ethdb.Reader, number uint64) rlp.RawValue {
// Note: ReadCanonicalHash cannot be used here because it also
// calls ReadAncients internally.
hash, _ := db.Get(headerHashKey(number))
data, _ = db.Get(blockBodyKey(number, common.BytesToHash(hash)))
data, _ = db.BlockStoreReader().Get(blockBodyKey(number, common.BytesToHash(hash)))
return nil
})
return data
@@ -468,10 +484,10 @@ func WriteBodyRLP(db ethdb.KeyValueWriter, hash common.Hash, number uint64, rlp
// HasBody verifies the existence of a block body corresponding to the hash.
func HasBody(db ethdb.Reader, hash common.Hash, number uint64) bool {
if isCanon(db, number, hash) {
if isCanon(db.BlockStoreReader(), number, hash) {
return true
}
if has, err := db.Has(blockBodyKey(number, hash)); !has || err != nil {
if has, err := db.BlockStoreReader().Has(blockBodyKey(number, hash)); !has || err != nil {
return false
}
return true
@@ -548,14 +564,14 @@ func DeleteBody(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
// ReadTdRLP retrieves a block's total difficulty corresponding to the hash in RLP encoding.
func ReadTdRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue {
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
// Check if the data is in ancients
if isCanon(reader, number, hash) {
data, _ = reader.Ancient(ChainFreezerDifficultyTable, number)
return nil
}
// If not, try reading from leveldb
data, _ = db.Get(headerTDKey(number, hash))
data, _ = db.BlockStoreReader().Get(headerTDKey(number, hash))
return nil
})
return data
@@ -596,10 +612,10 @@ func DeleteTd(db ethdb.KeyValueWriter, hash common.Hash, number uint64) {
// HasReceipts verifies the existence of all the transaction receipts belonging
// to a block.
func HasReceipts(db ethdb.Reader, hash common.Hash, number uint64) bool {
if isCanon(db, number, hash) {
if isCanon(db.BlockStoreReader(), number, hash) {
return true
}
if has, err := db.Has(blockReceiptsKey(number, hash)); !has || err != nil {
if has, err := db.BlockStoreReader().Has(blockReceiptsKey(number, hash)); !has || err != nil {
return false
}
return true
@@ -608,14 +624,14 @@ func HasReceipts(db ethdb.Reader, hash common.Hash, number uint64) bool {
// ReadReceiptsRLP retrieves all the transaction receipts belonging to a block in RLP encoding.
func ReadReceiptsRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.RawValue {
var data []byte
db.ReadAncients(func(reader ethdb.AncientReaderOp) error {
db.BlockStoreReader().ReadAncients(func(reader ethdb.AncientReaderOp) error {
// Check if the data is in ancients
if isCanon(reader, number, hash) {
data, _ = reader.Ancient(ChainFreezerReceiptTable, number)
return nil
}
// If not, try reading from leveldb
data, _ = db.Get(blockReceiptsKey(number, hash))
data, _ = db.BlockStoreReader().Get(blockReceiptsKey(number, hash))
return nil
})
return data
@@ -808,7 +824,7 @@ func WriteAncientBlocksWithBlobs(db ethdb.AncientWriter, blocks []*types.Block,
break
}
}
log.Info("WriteAncientBlocks", "startAt", blocks[0].Number(), "cancunIndex", cancunIndex, "len", len(blocks))
log.Debug("WriteAncientBlocks", "startAt", blocks[0].Number(), "cancunIndex", cancunIndex, "len", len(blocks))
var (
tdSum = new(big.Int).Set(td)
@@ -875,7 +891,7 @@ func ReadBlobSidecarsRLP(db ethdb.Reader, hash common.Hash, number uint64) rlp.R
return nil
}
// If not, try reading from leveldb
data, _ = db.Get(blockBlobSidecarsKey(number, hash))
data, _ = db.BlockStoreReader().Get(blockBlobSidecarsKey(number, hash))
return nil
})
return data
@@ -1077,24 +1093,24 @@ func FindCommonAncestor(db ethdb.Reader, a, b *types.Header) *types.Header {
// ReadHeadHeader returns the current canonical head header.
func ReadHeadHeader(db ethdb.Reader) *types.Header {
headHeaderHash := ReadHeadHeaderHash(db)
headHeaderHash := ReadHeadHeaderHash(db.BlockStoreReader())
if headHeaderHash == (common.Hash{}) {
return nil
}
headHeaderNumber := ReadHeaderNumber(db, headHeaderHash)
headHeaderNumber := ReadHeaderNumber(db.BlockStoreReader(), headHeaderHash)
if headHeaderNumber == nil {
return nil
}
return ReadHeader(db, headHeaderHash, *headHeaderNumber)
return ReadHeader(db.BlockStoreReader(), headHeaderHash, *headHeaderNumber)
}
// ReadHeadBlock returns the current canonical head block.
func ReadHeadBlock(db ethdb.Reader) *types.Block {
headBlockHash := ReadHeadBlockHash(db)
headBlockHash := ReadHeadBlockHash(db.BlockStoreReader())
if headBlockHash == (common.Hash{}) {
return nil
}
headBlockNumber := ReadHeaderNumber(db, headBlockHash)
headBlockNumber := ReadHeaderNumber(db.BlockStoreReader(), headBlockHash)
if headBlockNumber == nil {
return nil
}

View File

@@ -42,7 +42,7 @@ func ReadTxLookupEntry(db ethdb.Reader, hash common.Hash) *uint64 {
}
// Database v4-v5 tx lookup format just stores the hash
if len(data) == common.HashLength {
return ReadHeaderNumber(db, common.BytesToHash(data))
return ReadHeaderNumber(db.BlockStoreReader(), common.BytesToHash(data))
}
// Finally try database v3 tx lookup format
var entry LegacyTxLookupEntry

View File

@@ -121,16 +121,25 @@ func inspectFreezers(db ethdb.Database) ([]freezerInfo, error) {
// ancient indicates the path of root ancient directory where the chain freezer can
// be opened. Start and end specify the range for dumping out indexes.
// Note this function can only be used for debugging purposes.
func InspectFreezerTable(ancient string, freezerName string, tableName string, start, end int64) error {
func InspectFreezerTable(ancient string, freezerName string, tableName string, start, end int64, multiDatabase bool) error {
var (
path string
tables map[string]bool
)
switch freezerName {
case ChainFreezerName:
path, tables = resolveChainFreezerDir(ancient), chainFreezerNoSnappy
if multiDatabase {
path, tables = resolveChainFreezerDir(filepath.Dir(ancient)+"/block/ancient"), chainFreezerNoSnappy
} else {
path, tables = resolveChainFreezerDir(ancient), chainFreezerNoSnappy
}
case StateFreezerName:
path, tables = filepath.Join(ancient, freezerName), stateFreezerNoSnappy
if multiDatabase {
path, tables = filepath.Join(filepath.Dir(ancient)+"/state/ancient", freezerName), stateFreezerNoSnappy
} else {
path, tables = filepath.Join(ancient, freezerName), stateFreezerNoSnappy
}
default:
return fmt.Errorf("unknown freezer, supported ones: %v", freezers)
}

View File

@@ -17,6 +17,7 @@
package rawdb
import (
"errors"
"fmt"
"math/big"
"sync"
@@ -42,12 +43,14 @@ const (
freezerBatchLimit = 30000
)
var (
missFreezerEnvErr = errors.New("missing freezer env error")
)
// chainFreezer is a wrapper of freezer with additional chain freezing feature.
// The background thread will keep moving ancient chain segments from key-value
// database to flat files for saving space on live database.
type chainFreezer struct {
threshold atomic.Uint64 // Number of recent blocks not to freeze (params.FullImmutabilityThreshold apart from tests)
*Freezer
quit chan struct{}
wg sync.WaitGroup
@@ -62,13 +65,11 @@ func newChainFreezer(datadir string, namespace string, readonly bool, offset uin
if err != nil {
return nil, err
}
cf := chainFreezer{
return &chainFreezer{
Freezer: freezer,
quit: make(chan struct{}),
trigger: make(chan chan struct{}),
}
cf.threshold.Store(params.FullImmutabilityThreshold)
return &cf, nil
}, nil
}
// Close closes the chain freezer instance and terminates the background thread.
@@ -82,6 +83,57 @@ func (f *chainFreezer) Close() error {
return f.Freezer.Close()
}
// readHeadNumber returns the number of chain head block. 0 is returned if the
// block is unknown or not available yet.
func (f *chainFreezer) readHeadNumber(db ethdb.KeyValueReader) uint64 {
hash := ReadHeadBlockHash(db)
if hash == (common.Hash{}) {
log.Error("Head block is not reachable")
return 0
}
number := ReadHeaderNumber(db, hash)
if number == nil {
log.Error("Number of head block is missing")
return 0
}
return *number
}
// readFinalizedNumber returns the number of finalized block. 0 is returned
// if the block is unknown or not available yet.
func (f *chainFreezer) readFinalizedNumber(db ethdb.KeyValueReader) uint64 {
hash := ReadFinalizedBlockHash(db)
if hash == (common.Hash{}) {
return 0
}
number := ReadHeaderNumber(db, hash)
if number == nil {
log.Error("Number of finalized block is missing")
return 0
}
return *number
}
// freezeThreshold returns the threshold for chain freezing. It's determined
// by formula: max(finality, HEAD-params.FullImmutabilityThreshold).
func (f *chainFreezer) freezeThreshold(db ethdb.KeyValueReader) (uint64, error) {
var (
head = f.readHeadNumber(db)
final = f.readFinalizedNumber(db)
headLimit uint64
)
if head > params.FullImmutabilityThreshold {
headLimit = head - params.FullImmutabilityThreshold
}
if final == 0 && headLimit == 0 {
return 0, errors.New("freezing threshold is not available")
}
if final > headLimit {
return final, nil
}
return headLimit, nil
}
// freeze is a background thread that periodically checks the blockchain for any
// import progress and moves ancient data from the fast database into the freezer.
//
@@ -119,61 +171,54 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
return
}
}
// Retrieve the freezing threshold.
hash := ReadHeadBlockHash(nfdb)
if hash == (common.Hash{}) {
log.Debug("Current full block hash unavailable") // new chain, empty database
// check freezer env first, it must wait a while when the env is necessary
err := f.checkFreezerEnv()
if err == missFreezerEnvErr {
log.Warn("Freezer need related env, may wait for a while", "err", err)
backoff = true
continue
}
number := ReadHeaderNumber(nfdb, hash)
threshold := f.threshold.Load()
if err != nil {
log.Error("Freezer check FreezerEnv err", "err", err)
backoff = true
continue
}
threshold, err := f.freezeThreshold(nfdb)
if err != nil {
backoff = true
log.Debug("Current full block not old enough to freeze", "err", err)
continue
}
frozen := f.frozen.Load()
switch {
case number == nil:
log.Error("Current full block number unavailable", "hash", hash)
backoff = true
continue
case *number < threshold:
log.Debug("Current full block not old enough to freeze", "number", *number, "hash", hash, "delay", threshold)
backoff = true
continue
case *number-threshold <= frozen:
log.Debug("Ancient blocks frozen already", "number", *number, "hash", hash, "frozen", frozen)
// Short circuit if the blocks below threshold are already frozen.
if frozen != 0 && frozen-1 >= threshold {
backoff = true
log.Debug("Ancient blocks frozen already", "threshold", threshold, "frozen", frozen)
continue
}
head := ReadHeader(nfdb, hash, *number)
if head == nil {
log.Error("Current full block unavailable", "number", *number, "hash", hash)
backoff = true
continue
}
// Seems we have data ready to be frozen, process in usable batches
var (
start = time.Now()
first, _ = f.Ancients()
limit = *number - threshold
start = time.Now()
first = frozen // the first block to freeze
last = threshold // the last block to freeze
)
if limit-first > freezerBatchLimit {
limit = first + freezerBatchLimit
if last-first+1 > freezerBatchLimit {
last = freezerBatchLimit + first - 1
}
ancients, err := f.freezeRangeWithBlobs(nfdb, first, limit)
ancients, err := f.freezeRangeWithBlobs(nfdb, first, last)
if err != nil {
log.Error("Error in block freeze operation", "err", err)
backoff = true
continue
}
// Batch of blocks have been frozen, flush them before wiping from leveldb
if err := f.Sync(); err != nil {
log.Crit("Failed to flush frozen tables", "err", err)
}
// Wipe out all data from the active database
batch := db.NewBatch()
for i := 0; i < len(ancients); i++ {
@@ -250,6 +295,24 @@ func (f *chainFreezer) freeze(db ethdb.KeyValueStore) {
log.Debug("Deep froze chain segment", context...)
env, _ := f.freezeEnv.Load().(*ethdb.FreezerEnv)
hash := ReadHeadBlockHash(nfdb)
if hash == (common.Hash{}) {
log.Debug("Current full block hash unavailable") // new chain, empty database
backoff = true
continue
}
number := ReadHeaderNumber(nfdb, hash)
if number == nil {
log.Error("Current full block number unavailable", "hash", hash)
backoff = true
continue
}
head := ReadHeader(nfdb, hash, *number)
if head == nil {
log.Error("Current full block unavailable", "number", *number, "hash", hash)
backoff = true
continue
}
// try prune blob data after cancun fork
if isCancun(env, head.Number, head.Time) {
f.tryPruneBlobAncientTable(env, *number)
@@ -278,7 +341,7 @@ func (f *chainFreezer) tryPruneBlobAncientTable(env *ethdb.FreezerEnv, num uint6
log.Error("Cannot prune blob ancient", "block", num, "expectTail", expectTail, "err", err)
return
}
log.Info("Chain freezer prune useless blobs, now ancient data is", "from", expectTail, "to", num, "cost", common.PrettyDuration(time.Since(start)))
log.Debug("Chain freezer prune useless blobs, now ancient data is", "from", expectTail, "to", num, "cost", common.PrettyDuration(time.Since(start)))
}
func getBlobExtraReserveFromEnv(env *ethdb.FreezerEnv) uint64 {
@@ -290,7 +353,7 @@ func getBlobExtraReserveFromEnv(env *ethdb.FreezerEnv) uint64 {
func (f *chainFreezer) freezeRangeWithBlobs(nfdb *nofreezedb, number, limit uint64) (hashes []common.Hash, err error) {
defer func() {
log.Info("freezeRangeWithBlobs", "from", number, "to", limit, "err", err)
log.Debug("freezeRangeWithBlobs", "from", number, "to", limit, "err", err)
}()
lastHash := ReadCanonicalHash(nfdb, limit)
if lastHash == (common.Hash{}) {
@@ -339,13 +402,16 @@ func (f *chainFreezer) freezeRangeWithBlobs(nfdb *nofreezedb, number, limit uint
return hashes, err
}
// freezeRange moves a batch of chain segments from the fast database to the freezer.
// The parameters (number, limit) specify the relevant block range, both of which
// are included.
func (f *chainFreezer) freezeRange(nfdb *nofreezedb, number, limit uint64) (hashes []common.Hash, err error) {
if number > limit {
return nil, nil
}
env, _ := f.freezeEnv.Load().(*ethdb.FreezerEnv)
hashes = make([]common.Hash, 0, limit-number)
hashes = make([]common.Hash, 0, limit-number+1)
_, err = f.ModifyAncients(func(op ethdb.AncientWriteOp) error {
for ; number <= limit; number++ {
// Retrieve all the components of the canonical block.
@@ -413,6 +479,21 @@ func (f *chainFreezer) SetupFreezerEnv(env *ethdb.FreezerEnv) error {
return nil
}
func (f *chainFreezer) checkFreezerEnv() error {
_, exist := f.freezeEnv.Load().(*ethdb.FreezerEnv)
if exist {
return nil
}
blobFrozen, err := f.TableAncients(ChainFreezerBlobSidecarTable)
if err != nil {
return err
}
if blobFrozen > 0 {
return missFreezerEnvErr
}
return nil
}
func isCancun(env *ethdb.FreezerEnv, num *big.Int, time uint64) bool {
if env == nil || env.ChainCfg == nil {
return false

View File

@@ -80,7 +80,7 @@ func InitDatabaseFromFreezer(db ethdb.Database) {
}
batch.Reset()
WriteHeadHeaderHash(db, hash)
WriteHeadHeaderHash(db.BlockStore(), hash)
WriteHeadFastBlockHash(db, hash)
log.Info("Initialized database from freezer", "blocks", frozen, "elapsed", common.PrettyDuration(time.Since(start)))
}
@@ -122,7 +122,7 @@ func iterateTransactions(db ethdb.Database, from uint64, to uint64, reverse bool
}
defer close(rlpCh)
for n != end {
data := ReadCanonicalBodyRLP(db, n)
data := ReadCanonicalBodyRLP(db.BlockStore(), n)
// Feed the block to the aggregator, or abort on interrupt
select {
case rlpCh <- &numberRlp{n, data}:

View File

@@ -26,14 +26,13 @@ import (
"strings"
"time"
"github.com/olekukonko/tablewriter"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/ethdb/leveldb"
"github.com/ethereum/go-ethereum/ethdb/memorydb"
"github.com/ethereum/go-ethereum/ethdb/pebble"
"github.com/ethereum/go-ethereum/log"
"github.com/olekukonko/tablewriter"
)
// freezerdb is a database wrapper that enables freezer data retrievals.
@@ -44,6 +43,7 @@ type freezerdb struct {
ethdb.AncientFreezer
diffStore ethdb.KeyValueStore
stateStore ethdb.Database
blockStore ethdb.Database
}
func (frdb *freezerdb) StateStoreReader() ethdb.Reader {
@@ -53,6 +53,18 @@ func (frdb *freezerdb) StateStoreReader() ethdb.Reader {
return frdb.stateStore
}
func (frdb *freezerdb) BlockStoreReader() ethdb.Reader {
if frdb.blockStore == nil {
return frdb
}
return frdb.blockStore
}
func (frdb *freezerdb) BlockStoreWriter() ethdb.Writer {
//TODO implement me
panic("implement me")
}
// AncientDatadir returns the path of root ancient directory.
func (frdb *freezerdb) AncientDatadir() (string, error) {
return frdb.ancientRoot, nil
@@ -78,6 +90,11 @@ func (frdb *freezerdb) Close() error {
errs = append(errs, err)
}
}
if frdb.blockStore != nil {
if err := frdb.blockStore.Close(); err != nil {
errs = append(errs, err)
}
}
if len(errs) != 0 {
return fmt.Errorf("%v", errs)
}
@@ -106,19 +123,28 @@ func (frdb *freezerdb) SetStateStore(state ethdb.Database) {
frdb.stateStore = state
}
func (frdb *freezerdb) BlockStore() ethdb.Database {
if frdb.blockStore != nil {
return frdb.blockStore
} else {
return frdb
}
}
func (frdb *freezerdb) SetBlockStore(block ethdb.Database) {
if frdb.blockStore != nil {
frdb.blockStore.Close()
}
frdb.blockStore = block
}
// Freeze is a helper method used for external testing to trigger and block until
// a freeze cycle completes, without having to sleep for a minute to trigger the
// automatic background run.
func (frdb *freezerdb) Freeze(threshold uint64) error {
func (frdb *freezerdb) Freeze() error {
if frdb.AncientStore.(*chainFreezer).readonly {
return errReadOnly
}
// Set the freezer threshold to a temporary value
defer func(old uint64) {
frdb.AncientStore.(*chainFreezer).threshold.Store(old)
}(frdb.AncientStore.(*chainFreezer).threshold.Load())
frdb.AncientStore.(*chainFreezer).threshold.Store(threshold)
// Trigger a freeze cycle and block until it's done
trigger := make(chan struct{}, 1)
frdb.AncientStore.(*chainFreezer).trigger <- trigger
@@ -135,6 +161,7 @@ type nofreezedb struct {
ethdb.KeyValueStore
diffStore ethdb.KeyValueStore
stateStore ethdb.Database
blockStore ethdb.Database
}
// HasAncient returns an error as we don't have a backing chain freezer.
@@ -225,6 +252,31 @@ func (db *nofreezedb) StateStoreReader() ethdb.Reader {
return db
}
func (db *nofreezedb) BlockStore() ethdb.Database {
if db.blockStore != nil {
return db.blockStore
}
return db
}
func (db *nofreezedb) SetBlockStore(block ethdb.Database) {
db.blockStore = block
}
func (db *nofreezedb) BlockStoreReader() ethdb.Reader {
if db.blockStore != nil {
return db.blockStore
}
return db
}
func (db *nofreezedb) BlockStoreWriter() ethdb.Writer {
if db.blockStore != nil {
return db.blockStore
}
return db
}
func (db *nofreezedb) ReadAncients(fn func(reader ethdb.AncientReaderOp) error) (err error) {
// Unlike other ancient-related methods, this method does not return
// errNotSupported when invoked.
@@ -429,6 +481,7 @@ func NewDatabaseWithFreezer(db ethdb.KeyValueStore, ancient string, namespace st
// freezer.
}
}
// no prune ancient start success
if !readonly {
WriteAncientType(db, EntireFreezerType)
@@ -518,6 +571,7 @@ type OpenOptions struct {
DisableFreeze bool
IsLastOffset bool
PruneAncientData bool
// Ephemeral means that filesystem sync operations should be avoided: data integrity in the face of
// a crash is not important. This option should typically be used in tests.
Ephemeral bool
@@ -613,7 +667,7 @@ func AncientInspect(db ethdb.Database) error {
offset := counter(ReadOffSetOfCurrentAncientFreezer(db))
// Get number of ancient rows inside the freezer.
ancients := counter(0)
if count, err := db.ItemAmountInAncient(); err != nil {
if count, err := db.BlockStore().ItemAmountInAncient(); err != nil {
log.Error("failed to get the items amount in ancientDB", "err", err)
return err
} else {
@@ -661,6 +715,48 @@ func PruneHashTrieNodeInDataBase(db ethdb.Database) error {
return nil
}
type DataType int
const (
StateDataType DataType = iota
BlockDataType
ChainDataType
Unknown
)
func DataTypeByKey(key []byte) DataType {
switch {
// state
case IsLegacyTrieNode(key, key),
bytes.HasPrefix(key, stateIDPrefix) && len(key) == len(stateIDPrefix)+common.HashLength,
IsAccountTrieNode(key),
IsStorageTrieNode(key):
return StateDataType
// block
case bytes.HasPrefix(key, headerPrefix) && len(key) == (len(headerPrefix)+8+common.HashLength),
bytes.HasPrefix(key, blockBodyPrefix) && len(key) == (len(blockBodyPrefix)+8+common.HashLength),
bytes.HasPrefix(key, blockReceiptsPrefix) && len(key) == (len(blockReceiptsPrefix)+8+common.HashLength),
bytes.HasPrefix(key, headerPrefix) && bytes.HasSuffix(key, headerTDSuffix),
bytes.HasPrefix(key, headerPrefix) && bytes.HasSuffix(key, headerHashSuffix),
bytes.HasPrefix(key, headerNumberPrefix) && len(key) == (len(headerNumberPrefix)+common.HashLength):
return BlockDataType
default:
for _, meta := range [][]byte{
fastTrieProgressKey, persistentStateIDKey, trieJournalKey, snapSyncStatusFlagKey} {
if bytes.Equal(key, meta) {
return StateDataType
}
}
for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey} {
if bytes.Equal(key, meta) {
return BlockDataType
}
}
return ChainDataType
}
}
// InspectDatabase traverses the entire database and checks the size
// of all different categories of data.
func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
@@ -668,10 +764,15 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
defer it.Release()
var trieIter ethdb.Iterator
var blockIter ethdb.Iterator
if db.StateStore() != nil {
trieIter = db.StateStore().NewIterator(keyPrefix, nil)
defer trieIter.Release()
}
if db.BlockStore() != db {
blockIter = db.BlockStore().NewIterator(keyPrefix, nil)
defer blockIter.Release()
}
var (
count int64
start = time.Now()
@@ -801,6 +902,7 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
value = trieIter.Value()
size = common.StorageSize(len(key) + len(value))
)
total += size
switch {
case IsLegacyTrieNode(key, value):
@@ -814,9 +916,10 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
default:
var accounted bool
for _, meta := range [][]byte{
fastTrieProgressKey, persistentStateIDKey, trieJournalKey} {
fastTrieProgressKey, persistentStateIDKey, trieJournalKey, snapSyncStatusFlagKey} {
if bytes.Equal(key, meta) {
metadata.Add(size)
accounted = true
break
}
}
@@ -830,6 +933,54 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
logged = time.Now()
}
}
log.Info("Inspecting separate state database", "count", count, "elapsed", common.PrettyDuration(time.Since(start)))
}
// inspect separate block db
if blockIter != nil {
count = 0
logged = time.Now()
for blockIter.Next() {
var (
key = blockIter.Key()
value = blockIter.Value()
size = common.StorageSize(len(key) + len(value))
)
total += size
switch {
case bytes.HasPrefix(key, headerPrefix) && len(key) == (len(headerPrefix)+8+common.HashLength):
headers.Add(size)
case bytes.HasPrefix(key, blockBodyPrefix) && len(key) == (len(blockBodyPrefix)+8+common.HashLength):
bodies.Add(size)
case bytes.HasPrefix(key, blockReceiptsPrefix) && len(key) == (len(blockReceiptsPrefix)+8+common.HashLength):
receipts.Add(size)
case bytes.HasPrefix(key, headerPrefix) && bytes.HasSuffix(key, headerTDSuffix):
tds.Add(size)
case bytes.HasPrefix(key, headerPrefix) && bytes.HasSuffix(key, headerHashSuffix):
numHashPairings.Add(size)
case bytes.HasPrefix(key, headerNumberPrefix) && len(key) == (len(headerNumberPrefix)+common.HashLength):
hashNumPairings.Add(size)
default:
var accounted bool
for _, meta := range [][]byte{headHeaderKey, headFinalizedBlockKey} {
if bytes.Equal(key, meta) {
metadata.Add(size)
accounted = true
break
}
}
if !accounted {
unaccounted.Add(size)
}
}
count++
if count%1000 == 0 && time.Since(logged) > 8*time.Second {
log.Info("Inspecting separate block database", "count", count, "elapsed", common.PrettyDuration(time.Since(start)))
logged = time.Now()
}
}
log.Info("Inspecting separate block database", "count", count, "elapsed", common.PrettyDuration(time.Since(start)))
}
// Display the database statistic of key-value store.
stats := [][]string{
@@ -856,7 +1007,7 @@ func InspectDatabase(db ethdb.Database, keyPrefix, keyStart []byte) error {
{"Light client", "Bloom trie nodes", bloomTrieNodes.Size(), bloomTrieNodes.Count()},
}
// Inspect all registered append-only file store then.
ancients, err := inspectFreezers(db)
ancients, err := inspectFreezers(db.BlockStore())
if err != nil {
return err
}
@@ -995,3 +1146,26 @@ func ReadChainMetadata(db ethdb.KeyValueStore) [][]string {
}
return data
}
func ReadChainMetadataFromMultiDatabase(db ethdb.Database) [][]string {
pp := func(val *uint64) string {
if val == nil {
return "<nil>"
}
return fmt.Sprintf("%d (%#x)", *val, *val)
}
data := [][]string{
{"databaseVersion", pp(ReadDatabaseVersion(db))},
{"headBlockHash", fmt.Sprintf("%v", ReadHeadBlockHash(db.BlockStore()))},
{"headFastBlockHash", fmt.Sprintf("%v", ReadHeadFastBlockHash(db))},
{"headHeaderHash", fmt.Sprintf("%v", ReadHeadHeaderHash(db.BlockStore()))},
{"lastPivotNumber", pp(ReadLastPivotNumber(db))},
{"len(snapshotSyncStatus)", fmt.Sprintf("%d bytes", len(ReadSnapshotSyncStatus(db)))},
{"snapshotDisabled", fmt.Sprintf("%v", ReadSnapshotDisabled(db))},
{"snapshotJournal", fmt.Sprintf("%d bytes", len(ReadSnapshotJournal(db)))},
{"snapshotRecoveryNumber", pp(ReadSnapshotRecoveryNumber(db))},
{"snapshotRoot", fmt.Sprintf("%v", ReadSnapshotRoot(db))},
{"txIndexTail", pp(ReadTxIndexTail(db))},
}
return data
}

View File

@@ -239,7 +239,7 @@ func (f *Freezer) Ancient(kind string, number uint64) ([]byte, error) {
// - if maxBytes is not specified, 'count' items will be returned if they are present.
func (f *Freezer) AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error) {
if table := f.tables[kind]; table != nil {
return table.RetrieveItems(start, count, maxBytes)
return table.RetrieveItems(start-f.offset, count, maxBytes)
}
return nil, errUnknownTable
}
@@ -252,7 +252,7 @@ func (f *Freezer) Ancients() (uint64, error) {
func (f *Freezer) TableAncients(kind string) (uint64, error) {
f.writeLock.RLock()
defer f.writeLock.RUnlock()
return f.tables[kind].items.Load(), nil
return f.tables[kind].items.Load() + f.offset, nil
}
// ItemAmountInAncient returns the actual length of current ancientDB.
@@ -765,7 +765,7 @@ func (f *Freezer) ResetTable(kind string, startAt uint64, onlyEmpty bool) error
f.frozen.Add(f.offset)
f.tail.Add(f.offset)
f.writeBatch = newFreezerBatch(f)
log.Info("Reset Table", "kind", kind, "tail", f.tables[kind].itemHidden.Load(), "frozen", f.tables[kind].items.Load())
log.Debug("Reset Table", "kind", kind, "tail", f.tables[kind].itemHidden.Load(), "frozen", f.tables[kind].items.Load())
return nil
}

View File

@@ -40,6 +40,9 @@ var (
// headFastBlockKey tracks the latest known incomplete block's hash during fast sync.
headFastBlockKey = []byte("LastFast")
// headFinalizedBlockKey tracks the latest known finalized block hash.
headFinalizedBlockKey = []byte("LastFinalized")
// persistentStateIDKey tracks the id of latest stored state(for path-based only).
persistentStateIDKey = []byte("LastStateID")

View File

@@ -27,6 +27,22 @@ type table struct {
prefix string
}
func (t *table) BlockStoreReader() ethdb.Reader {
return t
}
func (t *table) BlockStoreWriter() ethdb.Writer {
return t
}
func (t *table) BlockStore() ethdb.Database {
return t
}
func (t *table) SetBlockStore(block ethdb.Database) {
panic("not implement")
}
// NewTable returns a database object that prefixes all keys with a given string.
func NewTable(db ethdb.Database, prefix string) ethdb.Database {
return &table{

View File

@@ -402,7 +402,7 @@ func (p *BlockPruner) backUpOldDb(name string, cache, handles int, namespace str
var oldOffSet uint64
if interrupt {
// The interrupt scecario within this function is specific for old and new ancientDB exsisted concurrently,
// The interrupt scecario within this function is specific for old and new ancientDB existed concurrently,
// should use last version of offset for oldAncientDB, because current offset is
// actually of the new ancientDB_Backup, but what we want is the offset of ancientDB being backup.
oldOffSet = rawdb.ReadOffSetOfLastAncientFreezer(chainDb)

View File

@@ -1896,6 +1896,10 @@ func (s *StateDB) convertAccountSet(set map[common.Address]*types.StateAccount)
return ret
}
func (s *StateDB) GetSnap() snapshot.Snapshot {
return s.snap
}
// copySet returns a deep-copied set.
func copySet[k comparable](set map[k][]byte) map[k][]byte {
copied := make(map[k][]byte, len(set))

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,15 @@
package bruno
import _ "embed"
// contract codes for Mainnet upgrade
var (
//go:embed mainnet/ValidatorContract
MainnetValidatorContract string
)
// contract codes for Chapel upgrade
var (
//go:embed chapel/ValidatorContract
ChapelValidatorContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,19 @@
package euler
import _ "embed"
// contract codes for Mainnet upgrade
var (
//go:embed mainnet/ValidatorContract
MainnetValidatorContract string
//go:embed mainnet/SlashContract
MainnetSlashContract string
)
// contract codes for Chapel upgrade
var (
//go:embed chapel/ValidatorContract
ChapelValidatorContract string
//go:embed chapel/SlashContract
ChapelSlashContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,59 @@
package feynman
import _ "embed"
// contract codes for Mainnet upgrade
var (
//go:embed mainnet/ValidatorContract
MainnetValidatorContract string
//go:embed mainnet/SlashContract
MainnetSlashContract string
//go:embed mainnet/TokenHubContract
MainnetTokenHubContract string
//go:embed mainnet/GovHubContract
MainnetGovHubContract string
//go:embed mainnet/CrossChainContract
MainnetCrossChainContract string
//go:embed mainnet/StakingContract
MainnetStakingContract string
//go:embed mainnet/StakeHubContract
MainnetStakeHubContract string
//go:embed mainnet/StakeCreditContract
MainnetStakeCreditContract string
//go:embed mainnet/GovernorContract
MainnetGovernorContract string
//go:embed mainnet/GovTokenContract
MainnetGovTokenContract string
//go:embed mainnet/TimelockContract
MainnetTimelockContract string
//go:embed mainnet/TokenRecoverPortalContract
MainnetTokenRecoverPortalContract string
)
// contract codes for Chapel upgrade
var (
//go:embed chapel/ValidatorContract
ChapelValidatorContract string
//go:embed chapel/SlashContract
ChapelSlashContract string
//go:embed chapel/TokenHubContract
ChapelTokenHubContract string
//go:embed chapel/GovHubContract
ChapelGovHubContract string
//go:embed chapel/CrossChainContract
ChapelCrossChainContract string
//go:embed chapel/StakingContract
ChapelStakingContract string
//go:embed chapel/StakeHubContract
ChapelStakeHubContract string
//go:embed chapel/StakeCreditContract
ChapelStakeCreditContract string
//go:embed chapel/GovernorContract
ChapelGovernorContract string
//go:embed chapel/GovTokenContract
ChapelGovTokenContract string
//go:embed chapel/TimelockContract
ChapelTimelockContract string
//go:embed chapel/TokenRecoverPortalContract
ChapelTokenRecoverPortalContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,11 @@
package feynman_fix
import _ "embed"
// contract codes for Chapel upgrade
var (
//go:embed chapel/ValidatorContract
ChapelValidatorContract string
//go:embed chapel/StakeHubContract
ChapelStakeHubContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,19 @@
package gibbs
import _ "embed"
// contract codes for Chapel upgrade
var (
//go:embed chapel/TokenHubContract
ChapelTokenHubContract string
//go:embed chapel/StakingContract
ChapelStakingContract string
)
// contract codes for Mainnet upgrade
var (
//go:embed mainnet/TokenHubContract
MainnetTokenHubContract string
//go:embed mainnet/StakingContract
MainnetStakingContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,23 @@
package kepler
import _ "embed"
// contract codes for Mainnet upgrade
var (
//go:embed mainnet/ValidatorContract
MainnetValidatorContract string
//go:embed mainnet/SlashContract
MainnetSlashContract string
//go:embed mainnet/SystemRewardContract
MainnetSystemRewardContract string
)
// contract codes for Chapel upgrade
var (
//go:embed chapel/ValidatorContract
ChapelValidatorContract string
//go:embed chapel/SlashContract
ChapelSlashContract string
//go:embed chapel/SystemRewardContract
ChapelSystemRewardContract string
)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More