Commit Graph

2789 Commits

Author SHA1 Message Date
Martin HS
a5a4fa7032
all: use uint256 in state (#28598)
This change makes use of uin256 to represent balance in state. It touches primarily upon statedb, stateobject and state processing, trying to avoid changes in transaction pools, core types, rpc and tracers.
2024-01-23 14:51:58 +01:00
Darioush Jalali
819a4977e8
core: fix genesis setup in benchReadChain (#28856) 2024-01-23 21:46:34 +08:00
rjl493456442
78a3c32ef4
core, core/rawdb, eth/sync: no tx indexing during snap sync (#28703)
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.

Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
2024-01-22 21:05:18 +01:00
Delweng
a6a956263e
core/state: no need to prune block if the same (#1332) 2024-01-22 16:41:48 +08:00
zzzckck
ef462c2b47
Merge pull request #2155 from sysvm/fix-prunestate 2024-01-22 14:36:50 +08:00
zzzckck
bea8f8ecfc
improve: remove sharedpool from miner (#2172) 2024-01-22 11:04:55 +08:00
Marius van der Wijden
f55a10b64d
params, core/forkid: enable cancun on sepolia and holesky (#28834)
This change enables Cancun 

- Sepolia at 1706655072 (Jan 31st, 2024)
- Holesky at 1707305664 (Feb 7th, 2024)

Specification: https://github.com/ethereum/execution-specs/pull/860
2024-01-20 16:03:14 +01:00
Felix Lange
0e93da3197
crypto/kzg4844: add helpers for versioned blob hashes (#28827)
The code to compute a versioned hash was duplicated a couple times, and also had a small
issue: if we ever change params.BlobTxHashVersion, it will most likely also cause changes
to the actual hash computation. So it's a bit useless to have this constant in params.
2024-01-19 11:41:17 +01:00
Sina Mahmoodi
e5d5e09faa
internal/ethapi: handle blobs in API methods (#28786)
EIP-4844 adds a new transaction type for blobs. Users can submit such transactions via `eth_sendRawTransaction`. In this PR we refrain from adding support to `eth_sendTransaction` and in fact it will fail if the user passes in a blob hash.

However since the chain can handle such transactions it makes sense to allow simulating them. E.g. an L2 operator should be able to simulate submitting a rollup blob and updating the L2 state. Most methods that take in a transaction object should recognize blobs. The change boils down to adding `blobVersionedHashes` and `maxFeePerBlobGas` to `TransactionArgs`. In summary:

- `eth_sendTransaction`: will fail for blob txes
- `eth_signTransaction`: will fail for blob txes

The methods that sign txes does not, as of this PR, add support the for new EIP-4844 transaction types. Resuming the summary:

- `eth_sendRawTransaction`: can send blob txes
- `eth_fillTransaction`: will fill in a blob tx. Note: here we simply fill in normal transaction fields + possibly `maxFeePerBlobGas` when blobs are present. One can imagine a more elaborate set-up where users can submit blobs themselves and we fill in proofs and commitments and such. Left for future PRs if desired.
- `eth_call`: can simulate blob messages
- `eth_estimateGas`: blobs have no effect here. They have a separate unit of gas which is not tunable in the transaction.
2024-01-17 15:06:14 +01:00
Alfie John
9ee6809ff4
core/txpool/blobpool: fix typos 2024-01-15 21:45:14 +02:00
kevaundray
0d5ecb5b90 crypto/kzg4844: use the new trusted setup file and format (#28383)
Changes the trusted_setup to the one created during the kzg-ceremony. The trusted setup file can be found in the consensus specs: https://github.com/ethereum/consensus-specs/blob/dev/presets/mainnet/trusted_setups/trusted_setup_4096.json
---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2024-01-15 17:09:47 +08:00
VM
a17fef4f15 test: add UT for resolveChainFreezerDir function 2024-01-15 11:09:41 +08:00
Darioush Jalali
29b73555ae
core/state: unexport GetOrNewStateObject (#28804) 2024-01-14 12:32:23 +01:00
VM
fee8a25957 fix: optimize resolveChainFreezerDir func 2024-01-12 15:36:26 +08:00
rjl493456442
ccb4d55a7c cmd, core: resolve scheme from a read-write database (#28313)
* cmd, core: resolve scheme from a read-write database

* cmd, core, eth: move the scheme check in the ethereum constructor

* cmd/geth: dump should in ro mode

* cmd: reverts
2024-01-12 14:47:01 +08:00
buddho
8ed5d24e1d
tests: revive evm test cases (#2116) 2024-01-10 16:58:27 +08:00
Marius van der Wijden
cfff3cbbf1
params, core/forkid: schedule cancun fork on goerli (#28719)
This PR schedules the cancun fork for the goerli testnet as discussed on ACD.
Spec: ethereum/execution-specs#860

We schedule:
  goerli at 1705473120
2024-01-08 20:33:32 +01:00
buddho
73d19c00cd
core: fix systemcontracts.GenesisHash when run bsc firstly without init (#2132) 2024-01-04 18:26:36 +08:00
buddho
de1a126ec5
parlia: fix verifyVoteAttestation when verify a batch of headers (#2121) 2024-01-03 15:58:06 +08:00
VM
124939aaa4
cmd/geth: fix parse state scheme (#2107)
* cmd/geth: fix parse state scheme

* cmd/geth: enrich logs about state scheme

* cmd/geth: rename functions

---------

Co-authored-by: VM <arimas@foxmail.com>
2024-01-03 15:38:39 +08:00
Darioush Jalali
2365d77968
core/vm: update comments to match eip number (#28743) 2024-01-02 11:39:53 +01:00
buddho
a6befb5078
params: use rialto to test builtin network logic (#2106) 2023-12-28 10:42:37 +08:00
cygaar
d2e3cb894b
core/state: logic equivalence for GetCodeHash (#28733) 2023-12-26 16:38:11 +08:00
buddho
eb4ea42196
core: add metrics for bad block (#2101) 2023-12-25 11:07:31 +08:00
buddho
0ba5816cc7
core/genesis: support chapel to run without geth init (#2100) 2023-12-25 10:58:52 +08:00
rjl493456442
f469470aff
core/rawdb: improve state scheme checking (#28724)
This pull request improves the condition to check if path state scheme is in use. 

Originally, root node presence was used as the indicator if path scheme is used or not. However due to fact that root node will be deleted during the initial snap sync, this condition is no longer useful.

If PersistentStateID is present, it shows that we've already configured for path scheme.
2023-12-22 07:50:41 +01:00
buddho
354c0d7180
params/config: remove useless toml tag for hardforks (#2099) 2023-12-22 11:58:36 +08:00
rjl493456442
cca94792a4
core, cmd, trie: fix the condition of pathdb initialization (#28718)
Original problem was caused by #28595, where we made it so that as soon as we start to sync, the root of the disk layer is deleted. That is not wrong per se, but another part of the code uses the "presence of the root" as an init-check for the pathdb. And, since the init-check now failed, the code tried to re-initialize it which failed since a sync was already ongoing.

The total impact being: after a state-sync has begun, if the node for some reason is is shut down, it will refuse to start up again, with the error message: `Fatal: Failed to register the Ethereum service: waiting for sync.`. 

This change also modifies how `geth removedb` works, so that the user is prompted for two things: `state data` and `ancient chain`. The former includes both the chaindb aswell as any state history stored in ancients. 

---------

Co-authored-by: Martin HS <martin@swende.se>
2023-12-21 20:28:32 +01:00
lightclient
577be37e0e
cmd/devp2p: update eth/snap protocol test suites for PoS (#28340)
Here we update the eth and snap protocol test suites with a new test chain,
created by the hivechain tool. The new test chain uses proof-of-stake. As such,
tests using PoW block propagation in the eth protocol are removed. The test suite
now connects to the node under test using the engine API in order to make it
accept transactions. 

The snap protocol test suite has been rewritten to output test descriptions and
log requests more verbosely.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-12-20 17:23:48 +01:00
zzzckck
474860ef77
Merge pull request #2035 from weiihann/v1.3.4-snapsync
all: pull snap sync PRs from upstream v1.13.5
2023-12-19 11:25:56 +08:00
wangyifan
cd58897f18
core/rawdb: implement size reporting for live items in freezer_table (#28525)
This is the fix to issue #27483. A new hiddenBytes() is introduced to calculate the byte size of hidden items in the freezer table. When reporting the size of the freezer table, size of the hidden items will be subtracted from the total size.

---------

Co-authored-by: Yifan <Yifan Wang>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
2023-12-18 20:10:54 +01:00
buddho
e44de3ab27
core: LoadChainConfig return the predefined config for built-in networks firstly (#2078)
* core: LoadChainConfig return predefined config for built-in net firstly

* cmd/geth: add a warn message for chain config in the configuration file

* consensus/parlia: change chain config log level when New parlia

* core: fix code style
2023-12-18 17:07:50 +08:00
alex
edc864f9ba
all: fix typos in comments (#28682)
chore(core,eth):fix a couple of typos
2023-12-18 09:35:12 +01:00
Péter Szilágyi
0b632d97f3 cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD 2023-12-14 11:10:45 +08:00
weiihann
84b268f6fb cmd, core, eth: change default network from ETH to BSC
fix tests

fix tests
2023-12-14 11:06:07 +08:00
Péter Szilágyi
f628e32ba5 cmd, core, params: add support for the Holesky testnet (#28007)
* cmd, core, params: add support for the Holesky testnet

* cmd/devp2p: add support for holesky for the dns crawler
# Conflicts:
#	cmd/devp2p/nodesetcmd.go
#	cmd/geth/main.go
#	cmd/utils/flags.go
#	core/genesis.go
#	params/bootnodes.go
#	params/config.go
2023-12-14 11:06:07 +08:00
buddh0
1ebf2a4376 params: fix comment 2023-12-13 18:28:12 +08:00
buddh0
8ce9cdaae3 params: set default hardfork times 2023-12-13 18:28:12 +08:00
buddh0
b3df096358 core/vote: wait some blocks beforing voting since mining begin 2023-12-13 18:27:52 +08:00
ucwong
81fd1b3cf9
core/txpool : small cleanup refactors (#28654) 2023-12-12 17:23:36 +02:00
Felföldi Zsolt
fff843cfaf
beacon/light: add CommitteeChain (#27766)
This change implements CommitteeChain which is a key component of the beacon light client. It is a passive data structure that can validate, hold and update a chain of beacon light sync committees and updates, starting from a checkpoint that proves the starting committee through a beacon block hash, header and corresponding state. Once synced to the current sync period, CommitteeChain can also validate signed beacon headers.
2023-12-08 13:38:00 +01:00
Martin HS
1048e2d6a3
cmd/evm: fix dump after state-test exec (#28650)
The dump after state-test didn't work, the problem was an error, "Already committed", which was silently ignored. 

This change re-initialises the state, so the dumping works again.
2023-12-08 11:06:01 +01:00
buddh0
fa5d0cf287 core/systemcontracts: update CommitUrl for keplerUpgrade 2023-12-07 15:53:25 +08:00
buddho
a30beeba59
core/txpool/legacypool: respect nolocals-setting (#2037) 2023-12-06 17:05:13 +08:00
rjl493456442
7891b210e0 trie/triedb/pathdb, core/rawdb: enhance error message in freezer (#28198)
This PR adds more error message for debugging purpose.
2023-12-05 15:46:43 +08:00
rjl493456442
3076ad2ab9 eth/protocols/snap: fix snap sync failure on empty storage range (#28306) 2023-12-05 15:46:43 +08:00
rjl493456442
1ddd337d1c core, trie, eth: refactor stacktrie constructor (#28350) 2023-12-05 15:32:11 +08:00
Martin Holst Swende
a182557ac3 trie: remove owner and binary marshaling from stacktrie (#28291) 2023-12-05 15:32:11 +08:00
zzzckck
5b78f5761a
Merge branch 'develop' into develop_lock_v1.3.4_merge 2023-12-05 11:29:13 +08:00
larry.lx
885de2c1ca fix: failed ut 2023-12-04 19:06:48 +08:00
larry.lx
761563155c fork: add hardfork Hertzfix 2023-12-04 19:06:48 +08:00
larry.lx
f7e9adc2c8 fix: remove sharedPool 2023-12-04 19:06:48 +08:00
lx
20dcaabfdc
Merge branch 'develop' into develop_lock_v1.3.3 2023-11-30 10:14:39 +08:00
rjl493456442
ab0eb46a84
core/state: make stateobject.create selfcontain (#28459) 2023-11-29 16:07:51 +08:00
jwasinger
28e7371701
all: replace log15 with slog (#28187)
This PR replaces Geth's logger package (a fork of [log15](https://github.com/inconshreveable/log15)) with an implementation using slog, a logging library included as part of the Go standard library as of Go1.21.

Main changes are as follows:
* removes any log handlers that were unused in the Geth codebase.
* Json, logfmt, and terminal formatters are now slog handlers.
* Verbosity level constants are changed to match slog constant values.  Internal translation is done to make this opaque to the user and backwards compatible with existing `--verbosity` and `--vmodule` options.
* `--log.backtraceat` and `--log.debug` are removed.

The external-facing API is largely the same as the existing Geth logger.  Logger method signatures remain unchanged.

A small semantic difference is that a `Handler` can only be set once per `Logger` and not changed dynamically.  This just means that a new logger must be instantiated every time the handler of the root logger is changed.

----
For users of the `go-ethereum/log` module. If you were using this module for your own project, you will need to change the initialization. If you previously did 
```golang
log.Root().SetHandler(log.LvlFilterHandler(log.LvlInfo, log.StreamHandler(os.Stderr, log.TerminalFormat(true))))
```
You now instead need to do 
```golang
log.SetDefault(log.NewLogger(log.NewTerminalHandlerWithLevel(os.Stderr, log.LevelInfo, true)))
```
See more about reasoning here: https://github.com/ethereum/go-ethereum/issues/28558#issuecomment-1820606613
2023-11-29 08:33:50 +01:00
larry.lx
8e19728ea7 fix: remove sharedPool 2023-11-29 15:02:58 +08:00
Péter Szilágyi
61b844f2b2
eth/gasestimator: allow slight estimation error in favor of less iterations (#28618)
* eth/gasestimator: early exit for plain transfer and error allowance

* core, eth/gasestimator: hard guess at a possible required gas

* internal/ethapi: update estimation tests with the error ratio

* eth/gasestimator: I hate you linter

* graphql: fix gas estimation test

---------

Co-authored-by: Oren <orenyomtov@users.noreply.github.com>
2023-11-28 22:31:47 +02:00
Martin Holst Swende
63979bc9cc
cmd/evm, core/state: fix post-exec dump of state (statetests, blockchaintests) (#28504)
There were several problems related to dumping state. 

- If a preimage was missing, even if we had set the `OnlyWithAddresses` to `false`, to export them anyway, the way the mapping was constructed (using `common.Address` as key) made the entries get lost anyway. Concerns both state- and blockchain tests. 
- Blockchain test execution was not configured to store preimages.

This changes makes it so that the block test executor takes a callback, just like the state test executor already does. This callback can be used to examine the post-execution state, e.g. to aid debugging of test failures.
2023-11-28 13:54:17 +01:00
buddh0
cd0356b106 core/systemcontracts: include BEP-319 on kepler hardfork 2023-11-22 19:09:34 +08:00
jwasinger
3cfcd252db
cmd/geth: add support for --dev flag in dumpgenesis (#28463)
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: lightclient <lightclient@protonmail.com>
2023-11-22 12:08:39 +01:00
jwasinger
104dbf7821
cmd/utils: validate pre-existing genesis in --dev mode (#28468)
geth --dev can be used with an existing data directory and genesis block. Since
dev mode only works with PoS, we need to verify that the merge has happened.

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-11-22 12:01:38 +01:00
Mario Vega
347fecd881
core/types: make 'v' optional for DynamicFeeTx and BlobTx (#28564)
This fixes an issue where transactions would not be accepted when they have only
'yParity' and not 'v'.
2023-11-22 11:00:44 +01:00
Marius van der Wijden
146e8d999c
core, trie, rpc: speed up tests (#28461)
* rpc: make subscription test faster

reduces time for TestClientSubscriptionChannelClose
from 25 sec to < 1 sec.

* trie: cache trie nodes for faster sanity check

This reduces the time spent on TestIncompleteSyncHash
from ~25s to ~16s.

* core/forkid: speed up validation test

This takes the validation test from > 5s to sub 1 sec

* core/state: improve snapshot test run
brings the time for TestSnapshotRandom from 13s down to 6s

* accounts/keystore: improve keyfile test

This removes some unnecessary waits and reduces the
runtime of TestUpdatedKeyfileContents from 5 to 3 seconds

* trie: remove resolver
* trie: only check ~5% of all trie nodes
2023-11-21 12:19:28 +01:00
rjl493456442
661bd45188
core/state/snapshot: print correct error from trie iterator (#28560) 2023-11-21 10:47:37 +08:00
buddho
0224d48df4
core: enable Shanghai EIPs (#1970) 2023-11-20 19:19:26 +08:00
jp-imx
14a1e96b68
core/txpool/legacypool: respect nolocals-setting (#28435)
This change adds a check to ensure that transactions added to the legacy pool are not treated as 'locals' if the global locals-management has been disabled. 

This change makes the pool enforce the --txpool.pricelimit setting.
2023-11-20 09:05:20 +01:00
Martin Holst Swende
2391fbc676
tests/fuzzers: move fuzzers into native packages (#28467)
This PR moves our fuzzers from tests/fuzzers into whatever their respective 'native' package is.

The historical reason why they were placed in an external location, is that when they were based on go-fuzz, they could not be "hidden" via the _test.go prefix. So in order to shove them away from the go-ethereum "production code", they were put aside.

But now we've rewritten them to be based on golang testing, and thus can be brought back. I've left (in tests/) the ones that are not production (bls128381), require non-standard imports (secp requires btcec, bn256 requires gnark/google/cloudflare deps).

This PR also adds a fuzzer for precompiled contracts, because why not.

This PR utilizes a newly rewritten replacement for go-118-fuzz-build, namely gofuzz-shim, which utilises the inputs from the fuzzing engine better.
2023-11-14 14:34:29 +01:00
Sina Mahmoodi
e803ef09ad
eth/tracers/js: fix isPush for push0 (#28520)
Fixes so that `push0` opcode is correctly reported as `true` by the `IsPush` function

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-11-14 13:14:38 +01:00
Guillaume Ballet
fa8d39807d
cmd, core, trie: verkle-capable geth init (#28270)
This change allows the creation of a genesis block for verkle testnets. This makes for a chunk of code that is easier to review and still touches many discussion points.
2023-11-14 13:09:40 +01:00
buddho
4be9481558
internal/ethapi: fix null effectiveGasPrice in GetTransactionReceipt (#1980) 2023-11-14 16:40:25 +08:00
rjl493456442
326fa00759
core/rawdb: fsync the index file after each freezer write (#28483)
* core/rawdb: fsync the index and data file after each freezer write

* core/rawdb: fsync the data file in freezer after write
2023-11-10 12:56:39 +02:00
Péter Szilágyi
470dba8fc1
core/vm: set basefee to 0 internally on eth_call (#28470)
* core/vm: set basefee to 0 internally on eth_call

* core: nicer 0-basefee, make it work for blob fees too

* internal/ethapi: make tests a bit more complex

* core: fix blob fee checker

* core: make code a bit more readable

* core: fix some test error strings

* core/vm: Get rid of weird comment

* core: dict wrong typo
2023-11-08 11:22:08 +02:00
joey
53559fc4d7
eth, trie/triedb/pathdb: pbss patches (#1955)
* fix: use the top root hash for rewinding under path schema

* feat: add async flush nodebuffer in path schema

* chore: add prun-block param suffix check

* fix: code review comments
2023-11-06 14:11:17 +08:00
lmittmann
a3be38127c
core/vm: performance tweak of OpCode.String() (#28453)
make `opCodeToString` a `[256]string` array

Co-authored-by: lmittmann <lmittmann@users.noreply.github.com>
2023-11-02 08:54:28 +02:00
Felix Lange
bc42e88415
core: add basic chain history support in GenerateChain (#28428)
This change improves GenerateChain to support internal chain history access (ChainReader)
for the consensus engine and EVM.

GenerateChain takes a `parent` block and the number of blocks to create. With my changes,
the consensus engine and EVM can now access blocks from `parent` up to the block currently
being generated. This is required to make the BLOCKHASH instruction work, and also needed
to create real clique chains.  Clique uses chain history to figure out if the current signer is in-turn,
for example.

I've also added some more accessors to BlockGen. These are helpful when creating transactions:

- g.Signer returns a signer instance for the current block
- g.Difficulty returns the current block difficulty
- g.Gas returns the remaining gas amount

Another fix in this commit concerns the receipts returned by GenerateChain. The receipts now
have properly derived fields (BlockHash, etc.) and should generally match what would be
returned by the RPC API.
2023-10-31 12:39:25 +01:00
Jakub Freebit
447945e438
core/rawdb: add logging and fix comments around AncientRange function. (#28379)
This adds warning logs when the read does not match the expected count.
We can also remove the size limit since the function documentation explicitly states
that callers should limit the count.
2023-10-31 12:04:45 +01:00
lx
56424d390f
trie: keep trie prefetch during validation phase (#1954) 2023-10-31 11:32:31 +08:00
GalaIO
0d9151eb8f
txpool: fix a potential crash issue in shutdown; (#1951) 2023-10-30 15:56:37 +08:00
lx
3bf998f55c
code: remove accountTrieCache and storageTrieCache (#1949)
accountTrieCache and storageTrieCache were introduced in this PR:
https://github.com/bnb-chain/bsc/pull/257, which is to improve performance.
Actually the performance gain is quite limited, as there is already dirty
and clean cache for trie node.
And after big merge, these 2 cache can not be used when PBSS is enabled.
So remove these code to simplify the logic.
2023-10-30 10:29:54 +08:00
Guillaume Ballet
4cbca5178a
core, cmd/geth: add --override.* flags to geth init (#28407)
* core, cmd/geth: add --override.* flags to geth init

* also apply overrides before genesis commit with new block

* review feedback
2023-10-27 10:15:14 +08:00
lx
fd6e7bb3b2
performance: commitTire concurrently (#1948) 2023-10-27 08:55:51 +08:00
Martin Holst Swende
96b75033c0
trie: use explicit errors in stacktrie (instead of panic) (#28361)
This PR removes panics from stacktrie (mostly), and makes the Update return errors instead. While adding tests for this, I also found that one case of possible corruption was not caught, which is now fixed.
2023-10-25 14:53:50 +02:00
lx
01d75a9d21
dependency: go version to 1.20 and some dependencies in go.mod (#1939)
* go.mod: upgrade prysm and the indrect dependency
prysm from v4.0.2 to v4.0.8, and run go mod tidy

* ci: upgrade go version from 1.19 to 1.20
* go-version: upgrade from v1.19 to v1.20
there is some dependency on go v1.20, such as go-libp2p v0.27.8
and also run go mod tidy

* dependency: upgrade docker version for security
it is not a big issue, since docker is only used for test purpose.

* rand: update the usage of math/rand after golang v1.20

2 APIs of math/rand module were deprecated since golang v1.20.
that is: rand.Seed() and rand.Read(), refer: ettps://pkg.go.dev/math/rand

"rand.Seed(seed int64)" has been replaced by: "r := rand.New(rand.NewSource(seed int64))",
need to initialize it with an instance before use

"rand.Read()" has been replaced by "crypto/rand.Read()"

* readme: need golang v1.20+ to build bsc
2023-10-24 21:51:30 +08:00
moomin
a8617c6d4d
txpool/legacypool: remove already known error in legacypool (#28400) 2023-10-23 18:36:11 +03:00
rjl493456442
3853f50082
trie/triedb/pathdb, core/rawdb: enhance error message in freezer (#28198)
This PR adds more error message for debugging purpose.
2023-10-23 15:46:39 +02:00
hyunchel
062598bb40
core/blockchain: fix typos and grammar (#28295) 2023-10-23 15:23:41 +02:00
aaronbuchwald
6c6982163b
core/state/pruner: track number of skipped items during state pruning (#28368) 2023-10-23 13:24:32 +08:00
kevaundray
a6a0ae45b6
crypto/kzg4844: use the new trusted setup file and format (#28383)
Changes the trusted_setup to the one created during the kzg-ceremony. The trusted setup file can be found in the consensus specs: https://github.com/ethereum/consensus-specs/blob/dev/presets/mainnet/trusted_setups/trusted_setup_4096.json
---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2023-10-22 16:05:04 +02:00
Martin Holst Swende
c1d5a012ea
core/state, tests: fix memory leak via fastcache (#28387)
This change fixes a memory leak, when running either state-tests or blockchain-tests, we allocate a `1MB` fastcache during snapshot generation. `fastcache` is a bit special, and requires a `Reset()` (it has it's own memory allocator). 

The `1MB` was hidden [here](https://github.com/ethereum/go-ethereum/blob/master/tests/state_test_util.go#L333) and [here](https://github.com/ethereum/go-ethereum/blob/master/tests/block_test_util.go#L146) respectively.
2023-10-20 13:35:49 +02:00
lightclient
20d5256e40
core/chain_makers: add SetParentBeaconRoot(..) to chain makers (#28252) 2023-10-17 14:43:46 +02:00
rjl493456442
1b1611b8d0
core, trie, eth: refactor stacktrie constructor (#28350)
This change enhances the stacktrie constructor by introducing an option struct. It also simplifies the `Hash` and `Commit` operations, getting rid of the special handling round root node.
2023-10-17 14:09:25 +02:00
Nathan
a6cfcfe2b3
consensus/parlia: fix nextForkHash in Extra filed of block header (#1923) 2023-10-17 19:09:11 +08:00
joeycli
9f5842e0ec fix: recover TestStateChanges 2023-10-16 15:27:17 +08:00
rjl493456442
5a3109b1bf trie: remove internal nodes between shortNode and child in path mode (#28163)
* trie: remove internal nodes between shortNode and child in path mode

* trie: address comments

* core/rawdb, trie: address comments

* core/rawdb: delete unused func

* trie: change comments

* trie: add missing tests

* trie: fix lint
2023-10-16 15:27:17 +08:00
joeycli
c3199ab5fc fix: ut error 2023-10-16 15:27:17 +08:00
Delweng
9baffb33b6 core/rawdb: no need to run truncateFile for readonly mode (#28145)
Avoid truncating files, if ancients are opened in readonly mode. With this change, we return error instead of trying (and failing)  to repair
2023-10-16 15:27:17 +08:00
Darioush Jalali
d3a6f9a19e core/state: check err for iter.Error in fastDeleteStorage (#28122)
core/state: check err for iter.Error
2023-10-16 15:27:17 +08:00
rjl493456442
713d8d66a9 core/state: implement fast storage deletion (#27955)
This changes implements faster post-selfdestruct iteration of storage slots for deletion, by using snapshot-storage+stacktrie to recover the trienodes to be deleted. This mechanism is only implemented for path-based schema.

For hash-based schema, the entire post-selfdestruct storage iteration is skipped, with this change, since hash-based does not actually perform deletion anyway.

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-10-16 15:27:17 +08:00
Marius van der Wijden
959b7332bf core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-10-16 15:27:17 +08:00
Péter Szilágyi
509a64ffb9
cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too (#28335)
* cmd, core, ethdb: enable Pebble on 32 bits and OpenBSD too

* ethdb/pebble: use Pebble's internal constant calculation
2023-10-13 22:50:20 +03:00
Martin Holst Swende
f62c58f8de
trie: make rhs-proof align with last key in range proofs (#28311)
During snap-sync, we request ranges of values: either a range of accounts or a range of storage values. For any large trie, e.g. the main account trie or a large storage trie, we cannot fetch everything at once.

Short version; we split it up and request in multiple stages. To do so, we use an origin field, to say "Give me all storage key/values where key > 0x20000000000000000". When the server fulfils this, the server provides the first key after origin, let's say 0x2e030000000000000 -- never providing the exact origin. However, the client-side needs to be able to verify that the 0x2e03.. indeed is the first one after 0x2000.., and therefore the attached proof concerns the origin, not the first key.

So, short-short version: the left-hand side of the proof relates to the origin, and is free-standing from the first leaf.

On the other hand, (pun intended), the right-hand side, there's no such 'gap' between "along what path does the proof walk" and the last provided leaf. The proof must prove the last element (unless there are no elements).

Therefore, we can simplify the semantics for trie.VerifyRangeProof by removing an argument. This doesn't make much difference in practice, but makes it so that we can remove some tests. The reason I am raising this is that the upcoming stacktrie-based verifier does not support such fancy features as standalone right-hand borders.
2023-10-13 16:05:29 +02:00
rjl493456442
1cb3b6aee4
eth/protocols/snap: fix snap sync failure on empty storage range (#28306)
This change addresses an issue in snap sync, specifically when the entire sync process can be halted due to an encountered empty storage range.

Currently, on the snap sync client side, the response to an empty (partial) storage range is discarded as a non-delivery. However, this response can be a valid response, when the particular range requested does not contain any slots.

For instance, consider a large contract where the entire key space is divided into 16 chunks, and there are no available slots in the last chunk [0xf] -> [end]. When the node receives a request for this particular range, the response includes:

    The proof with origin [0xf]
    A nil storage slot set

If we simply discard this response, the finalization of the last range will be skipped, halting the entire sync process indefinitely. The test case TestSyncWithUnevenStorage can reproduce the scenario described above.

In addition, this change also defines the common variables MaxAddress and MaxHash.
2023-10-13 09:08:26 +02:00
lx
181218128b
Merge branch 'develop' into master 2023-10-12 22:17:11 +08:00
Nathan
b86459a722
core/txpool: ignore nil sub when subpool have been shut down (#1915)
Co-authored-by: buddh0 <galaxystroller@gmail.com>
2023-10-12 10:20:48 +08:00
rjl493456442
eeb5dc3ccf
cmd, core: resolve scheme from a read-write database (#28313)
* cmd, core: resolve scheme from a read-write database

* cmd, core, eth: move the scheme check in the ethereum constructor

* cmd/geth: dump should in ro mode

* cmd: reverts
2023-10-11 11:27:44 +03:00
Nathan
73cfed0ea1
core/vm: clean up evm clearly when get it from pool (#1914) 2023-10-11 16:20:47 +08:00
Nathan
35d85e250b
core: write head block hash when shutdown (#1912) 2023-10-11 16:17:17 +08:00
lx
43e2c779b4
Revert "fix: skip a patch that could fork the chain (#1902)" (#1913)
This reverts commit 0d4721319988e36baf67d95f209c7abaa95d2d65.
2023-10-11 14:37:34 +08:00
Martin Holst Swende
8976a0c97a
trie: remove owner and binary marshaling from stacktrie (#28291)
This change
  - Removes the owner-notion from a stacktrie; the owner is only ever needed for comitting to the database, but the commit-function, the `writeFn` is provided by the caller, so the caller can just set the owner into the `writeFn` instead of having it passed through the stacktrie.
  - Removes the `encoding.BinaryMarshaler`/`encoding.BinaryUnmarshaler` interface from stacktrie. We're not using it, and it is doubtful whether anyone downstream is either.
2023-10-11 06:12:45 +02:00
Nathan
17ffdf1a9d
core/state: skip deleting storages for EmptyTrie (#1911) 2023-10-10 18:15:24 +08:00
0xbstn
fa6107c85e
core: fix typos (#28255)
fixes various typos in core
2023-10-10 10:22:03 +02:00
Nathan
f8439514e3
core/state: skip handleDestruction in hash based mode (#1908) 2023-10-08 18:55:30 +08:00
lx
0d47213199
fix: skip a patch that could fork the chain (#1902) 2023-10-07 16:02:33 +08:00
Fynn
4259f4c1f8 fix: state history hasn't write 2023-10-07 14:42:25 +08:00
hyunchel
4e1e37323d
core/txpool/legacypool: fix typo (#28258) 2023-10-07 10:57:12 +08:00
Péter Szilágyi
a8a9c8e4b0
core, eth, miner: start propagating and consuming blob txs (#28243)
* core, eth, miner: start propagating and consuming blob txs

* eth/protocols/eth: disable eth/67 if Cancun is enabled

* core/txpool, eth, miner: pass gas limit infos in lazy tx for mienr filtering

* core/txpool, miner: add lazy resolver for pending txs too

* core, eth: fix review noticed bugs

* eth, miner: minor polishes in the mining and announcing logs

* core/expool: unsubscribe the event scope
2023-10-04 12:36:36 +03:00
0xbstn
339a4cf056
core: fix typos (#28238) 2023-10-03 07:44:01 -04:00
Marius van der Wijden
c39cbc1a78
core: implement BLOBBASEFEE opcode (0x4a) (#28098)
Implements "EIP-7516: BLOBBASEFEE opcode" for cancun, as per spec: https://eips.ethereum.org/EIPS/eip-7516
2023-10-02 05:49:29 -04:00
aaronbuchwald
c5ff839fb2
core/state: small trie prefetcher nits (#28183)
Small trie prefetcher nits
2023-09-29 17:46:23 +03:00
Sina Mahmoodi
0ded110b80
core: infer blobGasUsed in chain maker (#28212)
Same way that the gasUsed in header is updated when a tx 
is added we should update blob gas used instead of requiring caller
to set it manually.
2023-09-29 10:44:28 +02:00
0xbstn
1f6e63900d
core: fix typos (#28218)
* fix(core/txpool): fix typos

* core/asm: fix typos

* core/bloombits: fix typos

* core/rawdb: fix typos
2023-09-29 10:52:22 +03:00
phenix3443
b9450bfcca
core, eth: typos and some code formatting (#28201)
* fix: typo

* feat: revert symbol name
2023-09-28 13:15:50 +03:00
rjl493456442
73f5bcb75b
core, accounts, eth, trie: handle genesis state missing (#28171)
* core, accounts, eth, trie: handle genesis state missing

* core, eth, trie: polish

* core: manage txpool subscription in mainpool

* eth/backend: fix test

* cmd, eth: fix test

* core/rawdb, trie/triedb/pathdb: address comments

* eth, trie: address comments

* eth: inline the function

* eth: use synced flag

* core/txpool: revert changes in txpool

* core, eth, trie: rename functions
2023-09-28 10:00:53 +03:00
lx
4b45c5993c
fix: skip HasState check for fast node (#1901) 2023-09-28 10:07:21 +08:00
0xbstn
a081130081
core/txpool: fix typos (#28213)
fix(core/txpool): fix typos
2023-09-28 09:48:14 +08:00
tokikuch
35b21cac14
core/bloombits: fix deadlock when matcher session hits an error (#1895)
When MatcherSession encounters an error, it attempts to close the session.
Closing waits for all goroutines to finish, including the 'distributor'.
However, the distributor will not exit until all requests have returned.

This patch fixes the issue by delivering the (empty) result to the distributor
before calling Close().
2023-09-27 15:00:27 +08:00
bnovil
614804b33c
core/txpool: fix typos (#28208)
core/txpool:fix typos
2023-09-27 11:08:53 +08:00
Nathan
f8bc2b76ac
core/state: ensure triedb Commit after Update (#1900) 2023-09-27 11:07:45 +08:00
Fynn
b8bad314ed cmd/geth: add hash2path & trie get tools 2023-09-26 20:45:05 +08:00
hzysvilla
4de89e92e4
core/vm: minor code formatting (#28199)
Adding a space beween function opOrigin() and opcCaller() in instruciton.go.
Adding a space beween function opkeccak256()  and opAddress() in instruciton.go.
2023-09-26 11:58:01 +03:00
joeycli
528d97b541 feat: active pbss on bsc
fix: lint error

fix: ut error

fix: code review comments
2023-09-26 16:14:32 +08:00
rjl493456442
720ff1fe57 all: activate pbss as experimental feature from eth (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-09-26 16:14:32 +08:00
GalaIO
d179056512 logs: add some logs to trace block miner, justified reorg; (#1894) 2023-09-26 14:21:22 +08:00
Marius van der Wijden
1fa3362ea7
core/forkid: add forkid test for holesky (#28193) 2023-09-25 17:02:19 +03:00
tokikuch
c2cfe35f12
core/bloombits: fix deadlock when matcher session hits an error (#28184)
When MatcherSession encounters an error, it attempts to close the session.
Closing waits for all goroutines to finish, including the 'distributor'. However, the
distributor will not exit until all requests have returned.

This patch fixes the issue by delivering the (empty) result to the distributor
before calling Close().
2023-09-25 15:35:24 +02:00
Péter Szilágyi
323542af50
core, params: update Holesky testnet to relaunched spec (#28191) 2023-09-25 16:10:23 +03:00
GalaIO
2f7d305013
logs: add some logs to trace block miner, justified reorg; (#1894) 2023-09-25 20:11:52 +08:00
Delweng
f1b2ec0833
core/rawdb: use readonly file lock in readonly mode (#28180)
This allows using the freezer from multiple processes at once
in read-only mode.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-09-22 12:10:50 +02:00
Péter Szilágyi
83f3fc2e80
core/state/snapshot: be very noisy if the generator hits a trie error (#28178) 2023-09-22 10:27:58 +03:00
rjl493456442
4773dcbc81
trie: remove internal nodes between shortNode and child in path mode (#28163)
* trie: remove internal nodes between shortNode and child in path mode

* trie: address comments

* core/rawdb, trie: address comments

* core/rawdb: delete unused func

* trie: change comments

* trie: add missing tests

* trie: fix lint
2023-09-22 09:31:10 +03:00
Delweng
545f4c5547
core/rawdb: no need to run truncateFile for readonly mode (#28145)
Avoid truncating files, if ancients are opened in readonly mode. With this change, we return error instead of trying (and failing)  to repair
2023-09-21 04:05:55 -04:00
Nathan
52e2cb86a5
parlia: reject header with non-nil WithdrawalsHash(#1884)
* parlia: reject header with `WithdrawalsHash` before shanghai fork

* log: output chainconfig when start up

* eth: fix TestOptionMaxPeersPerIP failure of goroutine order
2023-09-21 12:02:59 +08:00
Nathan
41f0667ce1
Revert "core/rawdb: open meta file in read only mode (#26009)" (#1879)
This reverts commit b9ba6f6e4d86d0ee86c63e8f4552e233fe0450aa.
2023-09-20 07:05:18 +08:00
Delweng
ef76afad35
core/rawdb: fix typo in comment (#28140) 2023-09-19 13:43:37 +02:00
Felix Lange
e9f78db79d
cmd/evm: fix some issues with the evm run command (#28109)
* cmd/evm: improve flags handling

This fixes some issues with flags in cmd/evm. The supported flags did not
actually show up in help output because they weren't categorized. I'm also
adding the VM-related flags to the run command here so they can be given
after the subcommand name. So it can be run like this now:

   ./evm run --code 6001 --debug

* cmd/evm: enable all forks by default in run command

The default genesis was just empty with no forks at all, which is annoying because
contracts will be relying on opcodes introduced in a fork. So this changes the default to
have all forks enabled.

* core/asm: fix some issues in the assembler

This fixes minor bugs in the old assembler:

- It is now possible to have comments on the same line as an instruction.
- Errors for invalid numbers in the jump instruction are reported better
- Line numbers in errors were off by one
2023-09-19 13:41:16 +02:00
NathanBSC
1bc27f6d98 core/state: handle account destruction after updating account state 2023-09-19 10:58:35 +08:00
rjl493456442
c53b0fef2a
core, eth/downloader: fix genesis state missing due to state sync (#28124)
* core: fix chain repair corner case in path-based scheme

* eth/downloader: disable trie database whenever state sync is launched
2023-09-17 17:35:09 +03:00
Darioush Jalali
48fdb79de5
core/state: check err for iter.Error in fastDeleteStorage (#28122)
core/state: check err for iter.Error
2023-09-15 14:09:07 +08:00
Felix Lange
909dd4a109
rlp/rlpgen: remove build tag (#28106)
* rlp/rlpgen: remove build tag

This tag was supposed to prevent unstable output when types reference each other. Imagine
there are two struct types A and B, where a reference to type B is in A. If I run rlpgen
on type B first, and then on type A, the generator will see the B.EncodeRLP method and
call it. However, if I run rlpgen on type A first, it will inline the encoding of B.

The solution I chose for the initial release of rlpgen was to just ignore methods
generated by rlpgen using a build tag. But there is a problem with this: if any code in
the package calls EncodeRLP explicitly, the package can't be loaded without errors anymore
in rlpgen, because the loader ignores it. Would be nice if there was a way to just make it
ignore invalid functions during type checking (they're not necessary for rlpgen), but
golang.org/x/tools/go/packages does not provide a way of ignoring them.

Luckily, the types we use rlpgen with do not reference each other right now, so we can
just remove the build tags for now.
2023-09-14 12:28:40 +02:00
Martin Holst Swende
8b6cf128af
metrics: refactor metrics (#28035)
This change includes a lot of things, listed below. 

### Split up interfaces, write vs read

The interfaces have been split up into one write-interface and one read-interface, with `Snapshot` being the gateway from write to read. This simplifies the semantics _a lot_. 

Example of splitting up an interface into one readonly 'snapshot' part, and one updatable writeonly part: 

```golang
type MeterSnapshot interface {
	Count() int64
	Rate1() float64
	Rate5() float64
	Rate15() float64
	RateMean() float64
}

// Meters count events to produce exponentially-weighted moving average rates
// at one-, five-, and fifteen-minutes and a mean rate.
type Meter interface {
	Mark(int64)
	Snapshot() MeterSnapshot
	Stop()
}
```

### A note about concurrency

This PR makes the concurrency model clearer. We have actual meters and snapshot of meters. The `meter` is the thing which can be accessed from the registry, and updates can be made to it. 

- For all `meters`, (`Gauge`, `Timer` etc), it is assumed that they are accessed by different threads, making updates. Therefore, all `meters` update-methods (`Inc`, `Add`, `Update`, `Clear` etc) need to be concurrency-safe. 
- All `meters` have a `Snapshot()` method. This method is _usually_ called from one thread, a backend-exporter. But it's fully possible to have several exporters simultaneously: therefore this method should also be concurrency-safe. 

TLDR: `meter`s are accessible via registry, all their methods must be concurrency-safe. 

For all `Snapshot`s, it is assumed that an individual exporter-thread has obtained a `meter` from the registry, and called the `Snapshot` method to obtain a readonly snapshot. This snapshot is _not_ guaranteed to be concurrency-safe. There's no need for a snapshot to be concurrency-safe, since exporters should not share snapshots. 

Note, though: that by happenstance a lot of the snapshots _are_ concurrency-safe, being unmutable minimal representations of a value. Only the more complex ones are _not_ threadsafe, those that lazily calculate things like `Variance()`, `Mean()`.

Example of how a background exporter typically works, obtaining the snapshot and sequentially accessing the non-threadsafe methods in it: 
```golang
		ms := metric.Snapshot()
                ...
		fields := map[string]interface{}{
			"count":    ms.Count(),
			"max":      ms.Max(),
			"mean":     ms.Mean(),
			"min":      ms.Min(),
			"stddev":   ms.StdDev(),
			"variance": ms.Variance(),
```

TLDR: `snapshots` are not guaranteed to be concurrency-safe (but often are).

### Sample changes

I also changed the `Sample` type: previously, it iterated the samples fully every time `Mean()`,`Sum()`, `Min()` or `Max()` was invoked. Since we now have readonly base data, we can just iterate it once, in the constructor, and set all four values at once. 

The same thing has been done for runtimehistogram. 

### ResettingTimer API

Back when ResettingTImer was implemented, as part of https://github.com/ethereum/go-ethereum/pull/15910, Anton implemented a `Percentiles` on the new type. However, the method did not conform to the other existing types which also had a `Percentiles`. 

1. The existing ones, on input, took `0.5` to mean `50%`. Anton used `50` to mean `50%`. 
2. The existing ones returned `float64` outputs, thus interpolating between values. A value-set of `0, 10`, at `50%` would return `5`, whereas Anton's would return either `0` or `10`. 

This PR removes the 'new' version, and uses only the 'legacy' percentiles, also for the ResettingTimer type. 

The resetting timer snapshot was also defined so that it would expose the internal values. This has been removed, and getters for `Max, Min, Mean` have been added instead. 

### Unexport types

A lot of types were exported, but do not need to be. This PR unexports quite a lot of them.
2023-09-13 13:13:47 -04:00
Péter Szilágyi
8d38b1fe62
core/rawdb: skip pathdb state inspection in hashdb mode (#28108) 2023-09-13 15:13:10 +03:00
lightclient
1efd12f695
core: fix calculation of blob gasprice in tx receipt (#28082)
This fixes the derived value BlobGasPrice on the receipt of EIP-4844 transactions, which was previously erroneously set to the price cap.
2023-09-11 02:14:20 -04:00
NathanBSC
0f622f3829
lint: fix all golang lint after big merge (#1861) 2023-09-08 16:36:16 +08:00
NathanBSC
6632d3bf99 Merge branch 'develop' into merge_develop_to_big_merge_v1.10.16_v1.12.2 2023-09-08 11:14:09 +08:00
NathanBSC
6af6162297
vote: check consensus key match vote key before voting (#1858) 2023-09-08 10:20:07 +08:00
rjl493456442
a7842c9cae
core, trie: cleanup trie database (#28062) 2023-09-07 21:17:14 +08:00
NathanBSC
b5e862b9a8 Merge branch 'develop' into merge_develop_to_big_merge_v1.10.16_v1.12.2 2023-09-07 17:50:15 +08:00
Eric
a0cb4d0377
eth/tracers: trace system tx should add intrinsicGas (#1855) 2023-09-07 16:46:54 +08:00
NathanBSC
202ab70922
Big merge v1.10.16 v1.12.2 fix ci (#1850)
* fix: crash of highestVerifiedHeader

* fix: panic of blobpool

* fix: genesis set up

* 1. modify NewDatabaseWithNodeDB to upstream
2. fix race use of hasher in statedb
3. fix use wrong value when updateTrie

* fix dir legacypool

* fix dir blobpool

* fix dir vote

* remove diffsync related code

* fix core/state/snapshot

* disable pipeCommit for now

* fix applyTransaction for bloom setting

* CI: fast finality in gasprice test

* CI: diffFetcher was removed

* CI: downloader, remove beaconsync test

* CI: no beaconsync in downloader, remove a failed case

TestCheckpointChallenge was removed in:
https://github.com/ethereum/go-ethereum/pull/27147
since after merge, it is useless for ethereum, but might be useful for BSC.
disable the case right now, as it is not a big issue.

* CI: bsc protocol decHandlers

* CI: receipt Bloom process

* 1. skip CheckConfigForkOrder for non-parlia engine
2. all test cases in core work well now
	cd core && go test ./... -v

* fix test cases in trie dir

* CI: no beaconsync in downloader, remove a failed case(redo)

* fix dir miner

* fix dir cmd/geth

* CI: filter test, BaseFee & Finality

* fix dir graphql

* remove diffStore

* fix ethclient

* fix TestRPCGetTransactionReceipt

* fix dir internal

* ut add dir ethstats and signer

* disable pipeCommit thoroughly; fix concurrent map iteration and map write in statedb

* CI: fix snap sync

it could be changed by mistake

* fix tests/Run to generate snapshot

* prepare for merge

* remove useless

* use common hasher in getDeletedStateObject, no race here

* an critical comment for state.Prepare

* do not copy nil accessList

* add omitempty tag for unused new fields of core.Genesis

* remove totalFees

* calculate fees before FinalizeAndAssemble

* revert interface Finalize of consensus

* do not double gas limit upon london block

* use Leveldb as default

* Revert "remove diffStore"

This reverts commit df343b137412b0beb25298a6ba9c3c19e47f20b1.

* Revert "remove diffsync related code"

This reverts commit 8d84b81feae5d794cb5d7fcfdb7f5f7da751941b.

* compile pass after revert

* remove diffsync

* fix dir eth/protocols/trust

* fix TestFastNode

* decHandlers for trust protocol

* keep persist diff in test
2023-09-07 16:39:29 +08:00
Delweng
2e02c1ffd9
core/rawdb: don't warn for missing "unclean shutdown markers" (#28014)
This changes removes the warn-printout about not finding unclean shutdown markers, which always happens on fresh installs / wiped databases.
2023-09-06 06:29:51 -04:00
lightclient
25733a4aad
params: update 4844 parameters (#28026)
On ACD 163, it was agreed to bump the target and max blob values from `2/4` to `3/6` for future devnets until we could decide on final mainnet number. This change contains said update, making master pass all the hive tests. The final decision for mainnet cancun is still to be made.
---------

Co-authored-by: Felix Lange <fjl@twurst.com>
2023-09-05 04:49:51 -04:00
lightclient
eff7c3bda0
core/forkid: skip genesis forks by time (#28034)
* core/forkid: skip genesis forks by time

* core/forkid: add comment about skipping non-zero fork times

* core/forkid: skip all time based forks in genesis using loop

* core/forkid: simplify logic for dropping time-based forks
2023-09-04 09:32:14 -04:00
rjl493456442
0acc0a1f86
core/state: simplify storage trie update and commit (#28030)
This change improves function description and simplifies logic in statedb update and commit operations.
2023-08-31 14:33:18 -04:00
Jorge
53f3c2ae65
metrics, cmd/geth: informational metrics (prometheus, influxdb, opentsb) (#24877)
This chang creates a GaugeInfo metrics type for registering informational (textual) metrics, e.g. geth version number. It also improves the testing for backend-exporters, and uses a shared subpackage in 'internal' to provide sample datasets and ordered registry. 

Implements #21783

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-31 13:37:17 -04:00
lightclient
41ee96fdfe
core/txpool/blobpool: fix rlp decoding flaw during offload (#28027) 2023-08-30 03:28:03 -04:00
Martin Holst Swende
d4e345c7d4
core/state: fix missing import (#28010) 2023-08-26 04:43:36 -04:00
rjl493456442
3ff6b3c31e
core/state: implement fast storage deletion (#27955)
This changes implements faster post-selfdestruct iteration of storage slots for deletion, by using snapshot-storage+stacktrie to recover the trienodes to be deleted. This mechanism is only implemented for path-based schema. 

For hash-based schema, the entire post-selfdestruct storage iteration is skipped, with this change, since hash-based does not actually perform deletion anyway. 

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-26 04:13:22 -04:00
Martin Holst Swende
6aa88ccdd2
beacon/engine, eth/catalyst, miner: EIP-4788 CL/EL protocol updates (#27872)
This PR makes EIP-4788 work in the engine API and miner. It also fixes some bugs related to 
EIP-4844 block processing and mining. Changes in detail:

- Header.BeaconRoot has been renamed to ParentBeaconRoot.
- The engine API now implements forkchoiceUpdatedV3
- newPayloadV3 method has been updated with the parentBeaconBlockRoot parameter
- beacon root is now applied to new blocks in miner
- For EIP-4844, block creation now updates the blobGasUsed field of the header
2023-08-26 04:52:12 +02:00
Felix Lange
9bbb9df185
core/types: transaction and receipt encoding/decoding optimizations (#27976)
Just some minor optimizations I figured out a while ago. By using ReadBytes instead of
Bytes on the rlp stream, we can save the allocation of a temporary buffer for the typed tx
payload.

If kind == rlp.Byte, the size reported by Stream.Kind will be zero, but we need a buffer
of size 1 for ReadBytes. Since typed txs always have to be longer than 1 byte, we can just
return an error for kind == rlp.Byte.

There is a also a small change for Log: since the first three fields of Log are the ones that 
should appear in the canon encoding, we can simply ignore the remaining fields via 
struct tag. Doing this removes an indirection through the rlpLog type.

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-25 20:03:41 +02:00
Péter Szilágyi
6b98d18789
cmd, core, params: add support for the Holesky testnet (#28007)
* cmd, core, params: add support for the Holesky testnet

* cmd/devp2p: add support for holesky for the dns crawler
2023-08-25 18:11:40 +03:00
Felix Lange
6dc9cdf15b
core: support null balance in genesis (#28005) 2023-08-25 15:59:40 +02:00
aaronbuchwald
56d2366699
core/state/snapshot: replace diffToDisk ideal batch size with 64MB (#27977) 2023-08-25 15:48:10 +03:00
Guido Vranken
0ba2d3cfa4
core/vm/runtime: Add Random field to config (#28001) 2023-08-25 04:05:33 -04:00
Felix Lange
9b46986edc
all: use rlp.DecodeBytes instead of rlp.Decode where possible (#27994) 2023-08-24 11:47:42 +03:00
lightclient
feb8f416ac
miner: add to build block with EIP-4844 blobs (#27875)
---------

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 23:16:14 +02:00
Felix Lange
d1f6735171
core/rawdb: fix 32bit build (#27995) 2023-08-23 23:14:36 +02:00
Martin Holst Swende
eb6cbe37e1
core/state: remove public method ForEachStorage (#27986)
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-23 21:53:31 +02:00
Marius van der Wijden
2f4dbb4f90
core/rawdb: allocate database keys with explicit size to avoid slice growth (#27772) 2023-08-23 21:41:56 +02:00
Martin Holst Swende
f0f8703bf2
core/rawdb, ethdb/pebble: avoid fsync db in tests (#27836)
Adds an option to disable fsync for database operations.
This is to make tests faster.
2023-08-23 20:43:55 +02:00
Péter Szilágyi
00fead91c4
cmd/utils: fix a startup issue on deleted chaindata but dangling ancients (#27989) 2023-08-23 16:42:37 +03:00
NathanBSC
2290201ff1 Merge branch 'develop' into big_merge_v1.10.16_v1.12.2 2023-08-23 19:37:39 +08:00
Péter Szilágyi
0c6bbeb423
core, eth, trie: expose more detailed dirty ram tracking for diff layers (#27971) 2023-08-23 14:08:39 +03:00
NathanBSC
872d22ed2d upstream: Merge tag 'v1.12.2' into develop 2023-08-23 17:46:08 +08:00
Felix Lange
4c7053baf1
core: add BeaconRoot to header in GenerateChain (#27974)
Fixes an error in tests of internal/ethapi.
2023-08-22 18:29:17 +02:00
Martin Holst Swende
b8d38e76ef
core, params, beacon/engine: implement EIP 4788 BeaconRoot (#27849)
This change implements "EIP 4788 : Beacon block root in the EVM". It implements version-2 of EPI-4788, main difference being that the contract is an actual contract rather than a precompile, as in #27289.
2023-08-22 08:33:39 -04:00
Delweng
7c0d90c8c9
core: ensure txindex will be triggered at least once (#27847)
Currently, we trigger the logic to (un)index transactions when the node receives a new
block. However, in some cases the node may not receive new blocks (eg, when the Geth node
is configured without peer discovery, or when it acts as an RPC node for historical-only
data).

In these situations, the Geth node user may not have previously configured txlookuplimit
(i.e. the default of around one year), but later realizes they need to index all
historical blocks. However, adding txlookuplimit=0 and restarting geth has no effect. This
change makes it check for required indexing work once, on startup, to fix the issue.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-22 02:58:20 +02:00
Delweng
9f4a528793
internal/ethapi: add testcases for blobTx (#27818)
Co-authored-by: Martin Holst Swende <martin@swende.se>
2023-08-22 02:53:29 +02:00
lx
e2bf0f3fbd
fix: lagging nodes failed to sync (#1829)
FastFinality puts more infor into the header.extra field to keep vote information.
For mainnet, on epoch height, it could be 1526 bytes, which was 517 bytes before.
So the hardcoded 700 bytes for header could be no longer enough, increase it by
2 times would be enough.

this bug could cause P2P sync failure for nodes that are lagging behind, since they
would request access of ancient db, and GetBlockHeaders could be failed.
2023-08-21 10:31:18 +08:00
Martin Holst Swende
1aa5520d75
core/txpool/legacypool: protect cache with mutex (#27898)
This change fixes the a potential race by using mutexes when the m.cache is read or modified.
2023-08-17 05:22:18 -04:00
lightclient
32fde3f838
core/forkid: correctly compute forkid when timestamp fork is activated in genesis (#27895)
This changes the forkID calculation to ignore time-based forks that occurred before the
genesis block. It's supposed to be done this way because the spec says:

> If a chain is configured to start with a non-Frontier ruleset already in its genesis, that is NOT considered a fork.
2023-08-16 23:31:02 +02:00
lightclient
a3e35414b7
core/types: fix unmarshalling of BlobTx values (#27939)
FromBig returns true *when overflow occurs*
2023-08-16 23:22:15 +02:00
Felix Lange
2a6beb6a39
core/types: support for optional blob sidecar in BlobTx (#27841)
This PR removes the newly added txpool.Transaction wrapper type, and instead adds a way
of keeping the blob sidecar within types.Transaction. It's better this way because most
code in go-ethereum does not care about blob transactions, and probably never will. This
will start mattering especially on the client side of RPC, where all APIs are based on
types.Transaction. Users need to be able to use the same signing flows they already
have.

However, since blobs are only allowed in some places but not others, we will now need to
add checks to avoid creating invalid blocks. I'm still trying to figure out the best place
to do some of these. The way I have it currently is as follows:

- In block validation (import), txs are verified not to have a blob sidecar.
- In miner, we strip off the sidecar when committing the transaction into the block.
- In TxPool validation, txs must have a sidecar to be added into the blobpool.
  - Note there is a special case here: when transactions are re-added because of a chain
    reorg, we cannot use the transactions gathered from the old chain blocks as-is,
    because they will be missing their blobs. This was previously handled by storing the
    blobs into the 'blobpool limbo'. The code has now changed to store the full
    transaction in the limbo instead, but it might be confusing for code readers why we're
    not simply adding the types.Transaction we already have.

Code changes summary:

- txpool.Transaction removed and all uses replaced by types.Transaction again
- blobpool now stores types.Transaction instead of defining its own blobTx format for storage
- the blobpool limbo now stores types.Transaction instead of storing only the blobs
- checks to validate the presence/absence of the blob sidecar added in certain critical places
2023-08-14 10:13:34 +02:00
Péter Szilágyi
811a674059 all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:19:12 +02:00
imulmat4
55863ce0d8 core/txpool/blobpool: fix metrics name for prometheus export (#27901) 2023-08-12 00:18:16 +02:00
Péter Szilágyi
be65b47645
all: update golang/x/ext and fix slice sorting fallout (#27909)
The Go authors updated golang/x/ext to change the function signature of the slices sort method. 
It's an entire shitshow now because x/ext is not tagged, so everyone's codebase just 
picked a new version that some other dep depends on, causing our code to fail building.

This PR updates the dep on our code too and does all the refactorings to follow upstream...
2023-08-12 00:04:12 +02:00
DongXi Huang
80b76a9527
core/types: remove duplication in eip2930 signer (#27860)
Remove duplication in signer
---------

Co-authored-by: GDdark <huangdongxi@minijoy.work>
Co-authored-by: Felix Lange <fjl@twurst.com>
2023-08-11 09:50:57 -04:00
imulmat4
6ddb92cac3
core/txpool/blobpool: fix metrics name for prometheus export (#27901) 2023-08-11 10:12:26 +03:00
rjl493456442
503f1f7ada
all: activate pbss as experimental feature (#26274)
* all: activate pbss

* core/rawdb: fix compilation error

* cma, core, eth, les, trie: address comments

* cmd, core, eth, trie: polish code

* core, cmd, eth: address comments

* cmd, core, eth, les, light, tests: address comment

* cmd/utils: shorten log message

* trie/triedb/pathdb: limit node buffer size to 1gb

* cmd/utils: fix opening non-existing db

* cmd/utils: rename flag name

* cmd, core: group chain history flags and fix tests

* core, eth, trie: fix memory leak in snapshot generation

* cmd, eth, internal: deprecate flags

* all: enable state tests for pathdb, fixes

* cmd, core: polish code

* trie/triedb/pathdb: limit the node buffer size to 256mb

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2023-08-10 22:21:36 +03:00
Martin Holst Swende
8d1db1601d
eth/filters, core/rawdb: remove unused param, refactor filtering-loop (#27891)
This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed.
It also modifies the filter loop slightly, avoiding a labeled break and instead using a method.

This change does not modify any behaviour.
2023-08-10 06:49:05 -04:00
Guillaume Ballet
67979022aa
core/state: move UpdateContractCode before the trie hash is computed (#27853)
Context: The UpdateContractCode method was introduced for the state storage commitment
schemes that include the whole code for their commitment computation. It must therefore be called
before the root hash is computed at the end of IntermediateRoot.

This should have no impact on the MPT since, in this context, the method is a no-op.
2023-08-09 18:02:45 +02:00