This is a follow-up to #29520, and a preparatory PR to a more thorough
change in the journalling system.
### API methods instead of `append` operations
This PR hides the journal-implementation details away, so that the
statedb invokes methods like `JournalCreate`, instead of explicitly
appending journal-events in a list. This means that it's up to the
journal whether to implement it as a sequence of events or
aggregate/merge events.
### Snapshot-management inside the journal
This PR also makes it so that management of valid snapshots is moved
inside the journal, exposed via the methods `Snapshot() int` and
`RevertToSnapshot(revid int, s *StateDB)`.
### SetCode
JournalSetCode journals the setting of code: it is implicit that the
previous values were "no code" and emptyCodeHash. Therefore, we can
simplify the setCode journal.
### Selfdestruct
The self-destruct journalling is a bit strange: we allow the
selfdestruct operation to be journalled several times. This makes it so
that we also are forced to store whether the account was already
destructed.
What we can do instead, is to only journal the first destruction, and
after that only journal balance-changes, but not journal the
selfdestruct itself.
This simplifies the journalling, so that internals about state
management does not leak into the journal-API.
### Preimages
Preimages were, for some reason, integrated into the journal management,
despite not being a consensus-critical data structure. This PR undoes
that.
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This pull request adds a few more performance metrics, specifically:
- The average time cost of an account read
- The average time cost of a storage read
- The rate of account reads
- The rate of storage reads
This pull request fixes the broken feature where the entire storage set is overridden.
Originally, the storage set override was achieved by marking the associated account
as deleted, preventing access to the storage slot on disk. However, since #29520, this
flag is also checked when accessing the account, rendering the account unreachable.
A fix has been applied in this pull request, which re-creates a new state object with all
account metadata inherited.
This pull request adds an additional error check after statedb.IntermediateRoot,
ensuring that no errors occur during this call. This step is essential, as the call might
encounter database errors.
Originally, these metrics were added to track the largest storage wiping.
Since account self-destruction was deprecated with the Cancun fork,
these metrics have become meaningless.
* all: add stateless verifications
* all: simplify witness and integrate it into live geth
---------
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Always prefetch the account trie while starting the prefetcher.
Co-authored-by: steven <steven@stevendeMacBook-Pro.local>
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
* core/state: trie prefetcher change: calling trie() doesn't stop the associated subfetcher
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
* core/state: improve prefetcher
* core/state: restore async prefetcher stask scheduling
* core/state: finish prefetching async and process storage updates async
* core/state: don't use the prefetcher for missing snapshot items
* core/state: remove update concurrency for Verkle tries
* core/state: add some termination checks to prefetcher async shutdowns
* core/state: differentiate db tries and prefetched tries
* core/state: teh teh teh
---------
Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Co-authored-by: Martin HS <martin@swende.se>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
* core/state, internal/workerpool: parallelize parts of state commit
* core, internal: move workerpool into syncx
* core/state: use errgroups, commit accounts concurrently
* core: resurrect detailed commit timers to almost-accuracy
This PR fixes some flaws with the existing tests.
The randomized testing (TestSnapshotRandom) executes a series of steps which modify the state and create journal-events. Later on, we compare the forward-going-states against the backwards-unrolling-journal-states, and check that they are identical.
The "identical" check is performed using various accessors. It turned out that we failed to check some things:
- the accesslist contents
- the transient storage contents
- the 'newContract' flag
- the dirty storage map
This change adds these new checks
Currently our state journal tracks each storage update to a contract, having the ability to revert those changes to the previously set value.
For the very first modification however, it behaves a bit wonky. Reverting the update doesn't actually remove the dirty-ness of the slot, rather leaves it as "change this slot to it's original value". This can cause issues down the line with for example write witnesses needing to gather an unneeded proof.
This PR modifies the storageChange journal entry to not only track the previous value of a slot, but also whether there was any previous value at all set in the current execution context. In essence, the PR changes the semantic of storageChange so it does not simply track storage changes, rather it tracks dirty storage changes, an important distinction for being able to cleanly revert the journal item.
This pull request defines a gentrie for snap sync purpose.
The stackTrie is used to generate the merkle tree nodes upon receiving a state batch. Several additional options have been added into stackTrie to handle incomplete states (either missing states before or after).
In this pull request, these options have been relocated from stackTrie to genTrie, which serves as a wrapper for stackTrie specifically for snap sync purposes.
Further, the logic for managing incomplete state has been enhanced in this change. Originally, there are two cases handled:
- boundary node filtering
- internal (covered by extension node) node clearing
This changes adds one more:
- Clearing leftover nodes on the boundaries.
This feature is necessary if there are leftover trie nodes in database, otherwise node inconsistency may break the state healing.
This addresses an edge-case (detailed in the code comment) where the computation of the intermediate trie root would force the unnecessary resolution of a hash node. The change makes it so that when we process changes from a block, we first process trie-updates and afterwards process trie-deletions.
Here we add a Go API for running tracing plugins within the main block import process.
As an advanced user of geth, you can now create a Go file in eth/tracers/live/, and within
that file register your custom tracer implementation. Then recompile geth and select your tracer
on the command line. Hooks defined in the tracer will run whenever a block is processed.
The hook system is defined in package core/tracing. It uses a struct with callbacks, instead of
requiring an interface, for several reasons:
- We plan to keep this API stable long-term. The core/tracing hook API does not depend on
on deep geth internals.
- There are a lot of hooks, and tracers will only need some of them. Using a struct allows you
to implement only the hooks you want to actually use.
All existing tracers in eth/tracers/native have been rewritten to use the new hook system.
This change breaks compatibility with the vm.EVMLogger interface that we used to have.
If you are a user of vm.EVMLogger, please migrate to core/tracing, and sorry for breaking
your stuff. But we just couldn't have both the old and new tracing APIs coexist in the EVM.
---------
Co-authored-by: Matthieu Vachon <matthieu.o.vachon@gmail.com>
Co-authored-by: Delweng <delweng@gmail.com>
Co-authored-by: Martin HS <martin@swende.se>
As SELF-DESTRUCT opcode is disabled in the cancun fork(unless the
account is created within the same transaction, nothing to delete
in this case). The account will only be deleted in the following
cases:
- The account is created within the same transaction. In this case
the original storage was empty.
- The account is empty(zero nonce, zero balance, zero code) and
is touched within the transaction. Fortunately this kind of accounts
are not-existent on ethereum-mainnet.
All in all, after cancun, we are pretty sure there is no large contract
deletion and we don't need this mechanism for oom protection.
