Our discovery crawler spits out a huge amount of logs, most of which is pretty non-interesting. This change moves the very verbose output to Debug, and adds a 8-second status log message giving the general idea about what's going on.
This change ports some changes from the main PBSS PR:
- get rid of callback function in `trie.Database.Commit` which is not required anymore
- rework the `nodeResolver` in `trie.Iterator` to make it compatible with multiple state scheme
- some other shallow changes in tests and typo-fixes
This PR moves some trie-related db accessor methods to a different file, and also removes the schema type. Instead of the schema type, a string is used to distinguish between hashbased/pathbased db accessors.
This also moves some code from trie package to rawdb package.
This PR is intended to be a no-functionality-change prep PR for #25963 .
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
* eth: cmd: deprecate personal namespace
* eth: cmd: move deprecation to node
* node: disable toml of enablepersonal
* node: disable personal on ipc as well
* Update node/node.go
Co-authored-by: Martin Holst Swende <martin@swende.se>
* console: error -> warn
* node: less roulette
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
Adds blocktest subcommand to the evm command, which is very similar to statetest, but instead of loading a StateTest static test it loads a BlockchainTest from a json file and runs it.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
This change implements withdrawals as specified in EIP-4895.
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: marioevz <marioevz@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
This PR changes the API so that uint64 is used for fork timestamps.
It's a good choice because types.Header also uses uint64 for time.
Co-authored-by: Felix Lange <fjl@twurst.com>
This fixes a regression where the flag did not accept values without
the 0x prefix anymore. What's worse, if an invalid value was passed,
the client would just log an INFO level message and continue.
This change introduces a breaking change to miner.etherbase is configured.
Previously, users did not need to explicitly set the etherbase address via flag, since 'first' local account was used as etherbase automatically. This change removes the "default first account" feature.
In Proof-of-stake world, the fee recipient address is provided by CL, and not configured in Geth any more - meaning that miner.etherbase is mostly for legacy networks(pow, clique networks etc).
This PR does a few things.
It fixes a shutdown-order flaw in the chainfreezer. Previously, the chain-freezer would shutdown the freezer backend first, and then signal for the loop to exit. This can lead to a scenario where the freezer tries to fsync closed files, which is an error-conditon that could lead to exit via log.Crit.
It also makes the printout more detailed when truncating 'dangling' items, by showing the exact number instead of approximate MB.
This PR also adds calls to fsync files before closing them, and also makes the `db inspect` command slightly more robust.
Implementation of https://eips.ethereum.org/EIPS/eip-3860, limit and meter initcode. This PR enables EIP-3860 as part of the Shanghai fork.
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
* p2p/discover: add more packet information in logs
This adds more fields to discv5 packet logs. These can be useful when
debugging multi-packet interactions.
The FINDNODE message also gets an additional field, OpID for debugging
purposes. This field is not encoded onto the wire.
I'm also removing topic system related message types in this change.
These will come back in the future, where support for them will be
guarded by a config flag.
* p2p/discover/v5wire: rename 'Total' to 'RespCount'
The new name captures the meaning of this field better.
This PR builds on #26299, but also updates the tests to the most recent version, which includes tests regarding TheMerge.
This change adds checks to the beacon consensus engine, making it more strict in validating the pre- and post-headers, and not relying on the caller to have already correctly sanitized the headers/blocks.
--syncTarget is a feature for development purpose in post-merge world. Previously
it's added into eth.Config. But it turns out that's a stupid idea.
- syncTarget is a block object, which is hard to be put in config file(large)
- syncTarget is just a dev feature, doesn't make too much sense to add it in config file
So I remove it from the eth config object. And it also fixes the #26328
The new flag allows configuring an explicit endpoint which is to be
announced in the DHT. This feature was originally developed for the
discv5 wormhole experiment (#25798), but it's useful in other contexts
as well.
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt #22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted #23247. Since then we added a command to migrate legacy dbs #24028.
As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
This PR introduces a node scheme abstraction. The interface is only implemented by `hashScheme` at the moment, but will be extended by `pathScheme` very soon.
Apart from that, a few changes are also included which is worth mentioning:
- port the changes in the stacktrie, tracking the path prefix of nodes during commit
- use ethdb.Database for constructing trie.Database. This is not necessary right now, but it is required for path-based used to open reverse diff freezer
This commit adds support for two new commands to clef, making it possible to list accounts / wallets from the command-line-interface.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR adds a parameter to startup, --synctarget. The synctarget flag is a developer-flag, that can be useful in some scenarios as a replacement for a CL node. It defines a fixed block sync target:
geth --syncmode=full --synctarget=./block_15816882.hex_rlp
The --synctarget is only made available during syncmode=full
Currently, in order to chain together sequential valid t8n transitions the caller must manually calculate the block base fee. This PR adds support for the necessary parent fee market data to calculate the base fee for the current transition.
Concretely, env is extended to accept the following:
parentBaseFee
parentGasUsed
parentGasLimit
Example usage can be found in ./cmd/evm/testdata/25.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR fixes a regression causing snapshots not to be generated in "geth --import" mode. It also fixes the geth export command to be truly readonly, and adds a new test for geth export.
This update resolves an issue where StringSliceFlag would not be
rendered correctly in help output + mention that -H can be used multiple times
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR makes it so that the snap server responds to trie heal requests when possible, even if the snapshot does not exist. The idea being that it might prolong the lifetime of a state root, so we don't have to pivot quite as often.
This PR makes it possible to set custom headers, in particular for two scenarios:
- geth attach
- geth commands which can use --remotedb, e..g geth db inspect
The ability to use custom headers is typically useful for connecting to cloud-apis, e.g. providing an infura- or alchemy key, or for that matter access-keys for environments behind cloudflare.
Co-authored-by: Felix Lange <fjl@twurst.com>
`geth dumpgenesis` currently does not respect the content of the data directory. Instead, it outputs the genesis block created by command-line flags. This PR fixes it to read the genesis from the database, if the database already exists.
Co-authored-by: Martin Holst Swende <martin@swende.se>
This PR cleans up the configurations for pruner and snapshotter by passing a config struct.
And also, this PR disables the snapshot background generation if the chain is opened in "read-only" mode. The read-only mode is necessary in some cases. For example, we have a list of commands to open the etheruem node in "read-only" mode, like export-chain. In these cases, the snapshot background generation is non expected and should be banned explicitly.
The abigen exclusion pattern, previously on the form "path:type", now supports wildcards. Examples "*:type" to exclude a named type in all files, or "/path/to/foo.sol:*" all types in foo.sol.
This changes the CI build to store the git commit and date into package
internal/version instead of package main. Doing this essentially merges our
two ways of tracking the go-ethereum version into a single place, achieving
two objectives:
- Bad block reports, which use version.Info(), will now have the git commit
information even when geth is built in an environment such as
launchpad.net where git access is unavailable.
- For geth builds created by `go build ./cmd/geth` (i.e. not using `go run
build/ci.go install`), git information stored by the go tool is now used
in the p2p node name as well as in `geth version` and `geth
version-check`.
* cmd/geth: add a verkle subcommand
* fix copyright year
* remove unused command parameters
* check that the output file was successfully written to
Co-authored-by: Martin Holst Swende <martin@swende.se>
* cmd/geth: goimports fix
Co-authored-by: Martin Holst Swende <martin@swende.se>
This changes the CI / release builds to use the latest Go version. It also
upgrades golangci-lint to a newer version compatible with Go 1.19.
In Go 1.19, godoc has gained official support for links and lists. The
syntax for code blocks in doc comments has changed and now requires a
leading tab character. gofmt adapts comments to the new syntax
automatically, so there are a lot of comment re-formatting changes in this
PR. We need to apply the new format in order to pass the CI lint stage with
Go 1.19.
With the linter upgrade, I have decided to disable 'gosec' - it produces
too many false-positive warnings. The 'deadcode' and 'varcheck' linters
have also been removed because golangci-lint warns about them being
unmaintained. 'unused' provides similar coverage and we already have it
enabled, so we don't lose much with this change.
* eth/fetcher: introduce some lag in tx fetching
* eth/fetcher: change conditions a bit
* eth/fetcher: use per-batch quota check
* eth/fetcher: fix some comments
* eth/fetcher: address review concerns
* eth/fetcher: fix panic + add warn log
* eth/fetcher: fix log
* eth/fetcher: fix log
* cmd/devp2p/internal/ethtest: fix ignorign tx announcements from prev. tests
* cmd/devp2p/internal/ethtest: fix TestLargeTxRequest
This increases the number of tx relay messages the test waits for. Since
go-ethereum now processes incoming txs in smaller batches, the
announcement messages it sends are also smaller.
Co-authored-by: Felix Lange <fjl@twurst.com>
This adds a cache for block logs which is shared by all filters. The cache
size of is configurable using the `--cache.blocklogs` flag.
Co-authored-by: Felix Lange <fjl@twurst.com>
* core: use TryGetAccount to read where TryUpdateAccount has been used to write
* Gary's review feedback
* implement Gary's suggestion
* fix bug + rename NewSecure into NewStateTrie
* trie: add backwards-compatibility aliases for SecureTrie
* Update database.go
* make the linter happy
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
This change updates our urfave/cli dependency to the v2 branch of the library.
There are some Go API changes in cli v2:
- Flag values can now be accessed using the methods ctx.Bool,
ctx.Int, ctx.String, ... regardless of whether the flag is 'local' or
'global'.
- v2 has built-in support for flag categories. Our home-grown category
system is removed and the categories of flags are assigned as part of
the flag definition.
For users, there is only one observable difference with cli v2: flags must now
strictly appear before regular arguments. For example, the following command is
now invalid:
geth account import mykey.json --password file.txt
Instead, the command must be invoked as follows:
geth account import --password file.txt mykey.json
This enables the following linters
- typecheck
- unused
- staticcheck
- bidichk
- durationcheck
- exportloopref
- gosec
WIth a few exceptions.
- We use a deprecated protobuf in trezor. I didn't want to mess with that, since I cannot meaningfully test any changes there.
- The deprecated TypeMux is used in a few places still, so the warning for it is silenced for now.
- Using string type in context.WithValue is apparently wrong, one should use a custom type, to prevent collisions between different places in the hierarchy of callers. That should be fixed at some point, but may require some attention.
- The warnings for using weak random generator are squashed, since we use a lot of random without need for cryptographic guarantees.
* accounts/abi/bind: fix duplicate field names in the generated go struct #24627
* accounts, cmd/abigen: resolve name conflicts
* ci lint, accounts/abi: remove unused function overloadedArgName
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
In #24028 we flagged a warning when finding legacy receipts in the freezer. This PR nudges users a bit more strongly by preventing geth from starting in this case until receipts have been migrated.
It also adds a flag --ignore-legacy-receipts which when present allows geth to start normally.
* ethdb/remotedb, cmd: add support for remote (readonly) databases
* ethdb/remotedb: minor changes
* ethdb/remotedb: close the conn
* cmd, ethdb: add rpc accessor for ancient data
* internal/ethapi: license
* ethdb/remotedb: linter fixes
This PR adds db tooling (geth db check-state-content) to verify the integrity of trie nodes. It iterates through the 32-byte key space in the database, which is expected to contain RLP-encoded trie nodes, addressed by hash.
Previously freezer has only been used for storing ancient chain data, while obviously it can be used more. This PR unties the chain data and freezer, keep the minimal freezer structure and move all other logic (like incrementally freezing block data) into a separate structure called ChainFreezer.
This PR also extends the database interface by adding a new ancient store function AncientDatadir which can return the root directory of ancient store. The ancient root directory can be used when we want to open some other ancient-stores (e.g. reverse diff freezer).
This PR groups all built-in network flags together and list them in the command as a whole.
And all database path flags(datadir, ancient) are also grouped, since usually these two are
used together.
This adds the ability to run --state.fork=Merged, and have post-merge rules apply. When doing so, it also requires the input env to contain currentRandom, and enforces the currentDifficulty to be omitted or zero.
This PR fixes up the example python clef wrapper. The poc is intended to demonstrate how to wite a UI for clef, and had severely bitrotted.
With these changes, it "works" in the sense that all the built-in tests triggers the intended python callbacks (no errors about method not found). It does not "work" in the sense that the wrapper can be used as an actual UI. It will auto-reject any signing requests, for example.
This PR fixes the flaw that @rjl493456442 found in https://github.com/ethereum/go-ethereum/pull/#issuecomment-1093817551 , namely, that the snapshot iterator uses the combined (disk + difflayers) 'view', wheres the raw iterator uses only the disk 'view'.
This PR instead splits up the work: one phase is iterating the disk layer data, another phase is loading the journalled difflayers and performing the same check there.
This adds a tools.go file to import all command packages used for
go:generate. Doing so makes it possible to execute go-based code
generators using 'go run', locking in the tool version using go.mod.
Co-authored-by: Felix Lange <fjl@twurst.com>
This commit replaces ioutil.TempDir with t.TempDir in tests. The
directory created by t.TempDir is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using ioutil.TempDir
had to be removed manually by calling os.RemoveAll, which is omitted in
some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but t.TempDir handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* eth/catalyst: only apply block if we actually have the state
* add header to payload queue
* Update cmd/geth/dbcmd.go
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Martin Holst Swende <martin@swende.se>
* cmd/geth: only check for presence of legacy receipts if developer mode is not enabled
* cmd/geth: degrade log level
* cmd/geth: fix format
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
