Commit Graph

249 Commits

Author SHA1 Message Date
buddho
93d652bad3
upstream: more code review fix (#33)
* add GenesisHeader to ChainHeaderReader

* fix check cancun header for parlia

* misc
2024-03-01 19:19:18 +08:00
buddh0
a501442a89 Merge branch 'develop' into cancun_code_merge_v1.13.0_v1.13.11 2024-02-21 19:25:44 +08:00
buddho
9ec1223f4d
Cancun code merge v1.13.0 v1.13.11 fix ci (#29)
* remove i386 linux tests

* test: fix building of tests

* return empty logs instead of nil in receipts

* keep InitialBaseFee same with geth, so not break a ton of cases

* fix eth dir

* fix subdir of core

* fix subdir of eth

* fix cmd/geth dir

* fix ethtest by adding UpgradeStatusMsg when handshake

* fix ethclient_test.go

* fix ethclient/simulated

* fix internal

* fix graphql

* fix consensus

* fix accounts

* fix log

* fix p2p

* fix metrics

* fix tests dir

* fix golangci-lint

---------

Co-authored-by: Matus Kysel <matus.kysel@bnbchain.org>
2024-02-21 19:19:20 +08:00
VM
5378df3702
cmd: optimize parse state scheme in cli and config (#2220) 2024-02-20 17:22:27 +08:00
rjl493456442
36a283ef98
core/rawdb: fsync the index file after each freezer write (#28483)
* core/rawdb: fsync the index and data file after each freezer write

* core/rawdb: fsync the data file in freezer after write
2024-02-14 08:22:43 +01:00
buddh0
a1c2491aab Merge tag 'v1.13.11' from go-ethereum 2024-02-02 15:55:18 +08:00
rjl493456442
78a3c32ef4
core, core/rawdb, eth/sync: no tx indexing during snap sync (#28703)
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.

Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
2024-01-22 21:05:18 +01:00
VM
a17fef4f15 test: add UT for resolveChainFreezerDir function 2024-01-15 11:09:41 +08:00
VM
fee8a25957 fix: optimize resolveChainFreezerDir func 2024-01-12 15:36:26 +08:00
rjl493456442
ccb4d55a7c cmd, core: resolve scheme from a read-write database (#28313)
* cmd, core: resolve scheme from a read-write database

* cmd, core, eth: move the scheme check in the ethereum constructor

* cmd/geth: dump should in ro mode

* cmd: reverts
2024-01-12 14:47:01 +08:00
VM
124939aaa4
cmd/geth: fix parse state scheme (#2107)
* cmd/geth: fix parse state scheme

* cmd/geth: enrich logs about state scheme

* cmd/geth: rename functions

---------

Co-authored-by: VM <arimas@foxmail.com>
2024-01-03 15:38:39 +08:00
rjl493456442
f469470aff
core/rawdb: improve state scheme checking (#28724)
This pull request improves the condition to check if path state scheme is in use. 

Originally, root node presence was used as the indicator if path scheme is used or not. However due to fact that root node will be deleted during the initial snap sync, this condition is no longer useful.

If PersistentStateID is present, it shows that we've already configured for path scheme.
2023-12-22 07:50:41 +01:00
rjl493456442
cca94792a4
core, cmd, trie: fix the condition of pathdb initialization (#28718)
Original problem was caused by #28595, where we made it so that as soon as we start to sync, the root of the disk layer is deleted. That is not wrong per se, but another part of the code uses the "presence of the root" as an init-check for the pathdb. And, since the init-check now failed, the code tried to re-initialize it which failed since a sync was already ongoing.

The total impact being: after a state-sync has begun, if the node for some reason is is shut down, it will refuse to start up again, with the error message: `Fatal: Failed to register the Ethereum service: waiting for sync.`. 

This change also modifies how `geth removedb` works, so that the user is prompted for two things: `state data` and `ancient chain`. The former includes both the chaindb aswell as any state history stored in ancients. 

---------

Co-authored-by: Martin HS <martin@swende.se>
2023-12-21 20:28:32 +01:00
lightclient
577be37e0e
cmd/devp2p: update eth/snap protocol test suites for PoS (#28340)
Here we update the eth and snap protocol test suites with a new test chain,
created by the hivechain tool. The new test chain uses proof-of-stake. As such,
tests using PoW block propagation in the eth protocol are removed. The test suite
now connects to the node under test using the engine API in order to make it
accept transactions. 

The snap protocol test suite has been rewritten to output test descriptions and
log requests more verbosely.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-12-20 17:23:48 +01:00
zzzckck
474860ef77
Merge pull request #2035 from weiihann/v1.3.4-snapsync
all: pull snap sync PRs from upstream v1.13.5
2023-12-19 11:25:56 +08:00
wangyifan
cd58897f18
core/rawdb: implement size reporting for live items in freezer_table (#28525)
This is the fix to issue #27483. A new hiddenBytes() is introduced to calculate the byte size of hidden items in the freezer table. When reporting the size of the freezer table, size of the hidden items will be subtracted from the total size.

---------

Co-authored-by: Yifan <Yifan Wang>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-12-18 20:10:54 +01:00
Péter Szilágyi
0b632d97f3 cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD 2023-12-14 11:10:45 +08:00
Felföldi Zsolt
fff843cfaf
beacon/light: add CommitteeChain (#27766)
This change implements CommitteeChain which is a key component of the beacon light client. It is a passive data structure that can validate, hold and update a chain of beacon light sync committees and updates, starting from a checkpoint that proves the starting committee through a beacon block hash, header and corresponding state. Once synced to the current sync period, CommitteeChain can also validate signed beacon headers.
2023-12-08 13:38:00 +01:00
rjl493456442
7891b210e0 trie/triedb/pathdb, core/rawdb: enhance error message in freezer (#28198)
This PR adds more error message for debugging purpose.
2023-12-05 15:46:43 +08:00
rjl493456442
326fa00759
core/rawdb: fsync the index file after each freezer write (#28483)
* core/rawdb: fsync the index and data file after each freezer write

* core/rawdb: fsync the data file in freezer after write
2023-11-10 12:56:39 +02:00
Jakub Freebit
447945e438
core/rawdb: add logging and fix comments around AncientRange function. (#28379)
This adds warning logs when the read does not match the expected count.
We can also remove the size limit since the function documentation explicitly states
that callers should limit the count.
2023-10-31 12:04:45 +01:00
rjl493456442
3853f50082
trie/triedb/pathdb, core/rawdb: enhance error message in freezer (#28198)
This PR adds more error message for debugging purpose.
2023-10-23 15:46:39 +02:00
rjl493456442
5a3109b1bf trie: remove internal nodes between shortNode and child in path mode (#28163)
* trie: remove internal nodes between shortNode and child in path mode

* trie: address comments

* core/rawdb, trie: address comments

* core/rawdb: delete unused func

* trie: change comments

* trie: add missing tests

* trie: fix lint
2023-10-16 15:27:17 +08:00
Delweng
9baffb33b6 core/rawdb: no need to run truncateFile for readonly mode (#28145)
Avoid truncating files, if ancients are opened in readonly mode. With this change, we return error instead of trying (and failing)  to repair
2023-10-16 15:27:17 +08:00
Marius van der Wijden
959b7332bf core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-10-16 15:27:17 +08:00
Péter Szilágyi
509a64ffb9
cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too (#28335)
* cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too

* ethdb/pebble: use Pebble's internal constant calculation
2023-10-13 22:50:20 +03:00
rjl493456442
eeb5dc3ccf
cmd, core: resolve scheme from a read-write database (#28313)
* cmd, core: resolve scheme from a read-write database

* cmd, core, eth: move the scheme check in the ethereum constructor

* cmd/geth: dump should in ro mode

* cmd: reverts
2023-10-11 11:27:44 +03:00
Fynn
4259f4c1f8 fix: state history hasn't write 2023-10-07 14:42:25 +08:00
0xbstn
1f6e63900d
core: fix typos (#28218)
* fix(core/txpool): fix typos

* core/asm: fix typos

* core/bloombits: fix typos

* core/rawdb: fix typos
2023-09-29 10:52:22 +03:00
rjl493456442
73f5bcb75b
core, accounts, eth, trie: handle genesis state missing (#28171)
* core, accounts, eth, trie: handle genesis state missing

* core, eth, trie: polish

* core: manage txpool subscription in mainpool

* eth/backend: fix test

* cmd, eth: fix test

* core/rawdb, trie/triedb/pathdb: address comments

* eth, trie: address comments

* eth: inline the function

* eth: use synced flag

* core/txpool: revert changes in txpool

* core, eth, trie: rename functions
2023-09-28 10:00:53 +03:00
Fynn
b8bad314ed cmd/geth: add hash2path & trie get tools 2023-09-26 20:45:05 +08:00
joeycli
528d97b541 feat: active pbss on bsc
fix: lint error

fix: ut error

fix: code review comments
2023-09-26 16:14:32 +08:00
rjl493456442
720ff1fe57 all: activate pbss as experimental feature from eth (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-09-26 16:14:32 +08:00
Delweng
f1b2ec0833
core/rawdb: use readonly file lock in readonly mode (#28180)
This allows using the freezer from multiple processes at once
in read-only mode.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-09-22 12:10:50 +02:00
rjl493456442
4773dcbc81
trie: remove internal nodes between shortNode and child in path mode (#28163)
* trie: remove internal nodes between shortNode and child in path mode

* trie: address comments

* core/rawdb, trie: address comments

* core/rawdb: delete unused func

* trie: change comments

* trie: add missing tests

* trie: fix lint
2023-09-22 09:31:10 +03:00
Delweng
545f4c5547
core/rawdb: no need to run truncateFile for readonly mode (#28145)
Avoid truncating files, if ancients are opened in readonly mode. With this change, we return error instead of trying (and failing)  to repair
2023-09-21 04:05:55 -04:00
Nathan
41f0667ce1
Revert "core/rawdb: open meta file in read only mode (#26009)" (#1879)
This reverts commit b9ba6f6e4d86d0ee86c63e8f4552e233fe0450aa.
2023-09-20 07:05:18 +08:00
Delweng
ef76afad35
core/rawdb: fix typo in comment (#28140) 2023-09-19 13:43:37 +02:00
Péter Szilágyi
8d38b1fe62
core/rawdb: skip pathdb state inspection in hashdb mode (#28108) 2023-09-13 15:13:10 +03:00
NathanBSC
0f622f3829
lint: fix all golang lint after big merge (#1861) 2023-09-08 16:36:16 +08:00
NathanBSC
202ab70922
Big merge v1.10.16 v1.12.2 fix ci (#1850)
* fix: crash of highestVerifiedHeader

* fix: panic of blobpool

* fix: genesis set up

* 1. modify NewDatabaseWithNodeDB to upstream
2. fix race use of hasher in statedb
3. fix use wrong value when updateTrie

* fix dir legacypool

* fix dir blobpool

* fix dir vote

* remove diffsync related code

* fix core/state/snapshot

* disable pipeCommit for now

* fix applyTransaction for bloom setting

* CI: fast finality in gasprice test

* CI: diffFetcher was removed

* CI: downloader, remove beaconsync test

* CI: no beaconsync in downloader, remove a failed case

TestCheckpointChallenge was removed in:
https://github.com/ethereum/go-ethereum/pull/27147
since after merge, it is useless for ethereum, but might be useful for BSC.
disable the case right now, as it is not a big issue.

* CI: bsc protocol decHandlers

* CI: receipt Bloom process

* 1. skip CheckConfigForkOrder for non-parlia engine
2. all test cases in core work well now
	cd core && go test ./... -v

* fix test cases in trie dir

* CI: no beaconsync in downloader, remove a failed case(redo)

* fix dir miner

* fix dir cmd/geth

* CI: filter test, BaseFee & Finality

* fix dir graphql

* remove diffStore

* fix ethclient

* fix TestRPCGetTransactionReceipt

* fix dir internal

* ut add dir ethstats and signer

* disable pipeCommit thoroughly; fix concurrent map iteration and map write in statedb

* CI: fix snap sync

it could be changed by mistake

* fix tests/Run to generate snapshot

* prepare for merge

* remove useless

* use common hasher in getDeletedStateObject, no race here

* an critical comment for state.Prepare

* do not copy nil accessList

* add omitempty tag for unused new fields of core.Genesis

* remove totalFees

* calculate fees before FinalizeAndAssemble

* revert interface Finalize of consensus

* do not double gas limit upon london block

* use Leveldb as default

* Revert "remove diffStore"

This reverts commit df343b137412b0beb25298a6ba9c3c19e47f20b1.

* Revert "remove diffsync related code"

This reverts commit 8d84b81feae5d794cb5d7fcfdb7f5f7da751941b.

* compile pass after revert

* remove diffsync

* fix dir eth/protocols/trust

* fix TestFastNode

* decHandlers for trust protocol

* keep persist diff in test
2023-09-07 16:39:29 +08:00
Delweng
2e02c1ffd9
core/rawdb: don't warn for missing "unclean shutdown markers" (#28014)
This changes removes the warn-printout about not finding unclean shutdown markers, which always happens on fresh installs / wiped databases.
2023-09-06 06:29:51 -04:00
Felix Lange
9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
Felix Lange
d1f6735171
core/rawdb: fix 32bit build (#27995) 2023-08-23 23:14:36 +02:00
Marius van der Wijden
2f4dbb4f90
core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-08-23 21:41:56 +02:00
Martin Holst Swende
f0f8703bf2
core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-08-23 20:43:55 +02:00
Péter Szilágyi
00fead91c4
cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-08-23 16:42:37 +03:00
NathanBSC
2290201ff1 Merge branch 'develop' into big_merge_v1.10.16_v1.12.2 2023-08-23 19:37:39 +08:00
NathanBSC
872d22ed2d upstream: Merge tag 'v1.12.2' into develop 2023-08-23 17:46:08 +08:00
lx
e2bf0f3fbd
fix: lagging nodes failed to sync (#1829)
FastFinality puts more infor into the header.extra field to keep vote information.
For mainnet, on epoch height, it could be 1526 bytes, which was 517 bytes before.
So the hardcoded 700 bytes for header could be no longer enough, increase it by
2 times would be enough.

this bug could cause P2P sync failure for nodes that are lagging behind, since they
would request access of ancient db, and GetBlockHeaders could be failed.
2023-08-21 10:31:18 +08:00