* offline block prune
* update
* update
* update and add unit test
* addressed comments from walt
* Addressed comments from walt and Igor
* ensure MPT and snapshot matched
* add one more parameter to indicate blockprune
* update the logic of creating freezerDb
* update flag command description
* expose the function for db inspect the offset/startBlockNumber
* add flags to inspect prune info
* rename flag of reserved-recent-blocks to block-amount-reserved
* addressed comments from walt
* handle the case of command interruption
* refined goimports
* addressed comments from walt
* change the logic as restarting prune after interruption
* addressed comments
* reclaimed freezer logic
* introduce flag to enable/disable check between MPT and snapshot
* update the logic of frozen field in freezerDB
* update the code in all places related to freezer change
* addressed comments from dylan
* update the logic for backup block difficulty
* addressed comments from dylan
* eth/tracers: implement debug.intermediateRoots (#23594)
This PR implements a new debug method, which I've talked briefly about to some other client developers. It allows the caller to obtain the intermediate state roots for a block (which might be either a canon block or a 'bad' block).
Signed-off-by: wenbiao <delweng@gmail.com>
* core, rpc: disable memory output by default in traces (#23558)
* core: cmd: invert disableMemory
* core: fix missed inversion
* cmd/evm: preserve Flags but change default value
* Apply suggestions from code review
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: abort evm execution when trace is aborted (#23580)
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: avoid unsyncronized mutations on trie database (#23632)
This PR fixes an issue in traceChain, where the statedb Commit operation was performed asynchronously with dereference-operations agains the underlying trie.Database instance. Due to how the reference counting works within the trie database (where parent count is recursively updated when new parents are added), doing dereferencing in the middle of Commit can cause the refcount to become wrong, leading to an inconsistent state.
This was fixed by doing Commit/Deref from the same routine.
Signed-off-by: wenbiao <delweng@gmail.com>
* core,eth: call frame tracing (#23087)
This change introduces 2 new optional methods; `enter()` and `exit()` for js tracers, and makes `step()` optiona. The two new methods are invoked when entering and exiting a call frame (but not invoked for the outermost scope, which has it's own methods). Currently these are the data fields passed to each of them:
enter: type (opcode), from, to, input, gas, value
exit: output, gasUsed, error
The PR also comes with a re-write of the callTracer. As a backup we keep the previous tracing script under the name `callTracerLegacy`. Behaviour of both tracers are equivalent for the most part, although there are some small differences (improvements), where the new tracer is more correct / has more information.
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: re-write of 4byte tracer using enter/exit (#23622)
* eth/tracers: add re-write of 4byte tracer using enter/exit
* eth/tracers: fix 4byte indent
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: tx.BaseFee not implemented
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: do the JSON serialization via .js to capture C faults
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: fix callTracer fault handling (#23667)
* eth/tracers: fix calltracer fault handling
* eth/tracers: fix calltracer indentation
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: invoke enter/exit on 0-value calls to inex accounts (#23828)
Signed-off-by: wenbiao <delweng@gmail.com>
* eth: make traceChain avoid OOM on long-running tracing (#23736)
This PR changes long-running chain tracing, so that it at some points releases the memory trie db, and switch over to a fresh disk-backed trie.
Signed-off-by: wenbiao <delweng@gmail.com>
* eth/tracers: expose contextual infos (block hash, tx hash, tx index)
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: redefine Context
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: support for golang tracers + add golang callTracer (#23708)
* eth/tracers: add basic native loader
* eth/tracers: add GetResult to tracer interface
* eth/tracers: add native call tracer
* eth/tracers: fix call tracer json result
* eth/tracers: minor fix
* eth/tracers: fix
* eth/tracers: fix benchTracer
* eth/tracers: test native call tracer
* eth/tracers: fix
* eth/tracers: rm extra make
Co-authored-by: Martin Holst Swende <martin@swende.se>
* eth/tracers: rm extra make
* eth/tracers: make callFrame private
* eth/tracers: clean-up and comments
* eth/tracers: add license
* eth/tracers: rework the model a bit
* eth/tracers: move tracecall tests to subpackage
* cmd/geth: load native tracers
* eth/tracers: minor fix
* eth/tracers: impl stop
* eth/tracers: add native noop tracer
* renamings
Co-authored-by: Martin Holst Swende <martin@swende.se>
* eth/tracers: more renamings
* eth/tracers: make jstracer non-exported, avoid cast
* eth/tracers, core/vm: rename vm.Tracer to vm.EVMLogger for clarity
* eth/tracers: minor comment fix
* eth/tracers/testing: lint nitpicks
* core,eth: cancel evm on nativecalltracer stop
* Revert "core,eth: cancel evm on nativecalltracer stop"
This reverts commit 01bb908790a369c1bb9d3937df9325c6857bf855.
* eth/tracers: linter nits
* eth/tracers: fix output on err
Co-authored-by: Martin Holst Swende <martin@swende.se>
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: make native calltracer default (#23867)
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: package restructuring (#23857)
* eth/tracers: restructure tracer package
* core/vm/runtime: load js tracers
* eth/tracers: mv bigint js code to own file
* eth/tracers: add method docs for native tracers
* eth/tracers: minor doc fix
* core,eth: cancel evm on nativecalltracer stop
* core/vm: fix failing test
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: ethapi.TransactionArgs was not merged
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: fix the api_test with ErrInsufficientFunds to ErrInsufficientFundsForTransfer
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: check posa before statedb.Prepare in IntermiateRoots api
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: make js calltracer default, compatible with old version
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: fix the default callTrace name of callTracerJs
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* Revert "eth/tracers: fix the default callTrace name of callTracerJs"
This reverts commit 62a3bc215d9f07e422a4c659289bb3ba4f9ed2fa.
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* Revert "eth/tracers: make js calltracer default, compatible with old version"
This reverts commit 85ef42c0ea651f0b228d4209b1b2598b24e12f1f.
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
* eth/tracers: fix the variable race condition
Signed-off-by: wenbiao <wenbiao.zheng@ambergroup.io>
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
* reannouce local pending transactions
* add tests for tx_pool reannouce local pending transactions
* add tests for handler reannounce local pending transactions
* focus on performance improvement in many aspects.
1. Do BlockBody verification concurrently;
2. Do calculation of intermediate root concurrently;
3. Preload accounts before processing blocks;
4. Make the snapshot layers configurable.
5. Reuse some object to reduce GC.
add
* rlp: improve decoder stream implementation (#22858)
This commit makes various cleanup changes to rlp.Stream.
* rlp: shrink Stream struct
This removes a lot of unused padding space in Stream by reordering the
fields. The size of Stream changes from 120 bytes to 88 bytes. Stream
instances are internally cached and reused using sync.Pool, so this does
not improve performance.
* rlp: simplify list stack
The list stack kept track of the size of the current list context as
well as the current offset into it. The size had to be stored in the
stack in order to subtract it from the remaining bytes of any enclosing
list in ListEnd. It seems that this can be implemented in a simpler
way: just subtract the size from the enclosing list context in List instead.
* rlp: use atomic.Value for type cache (#22902)
All encoding/decoding operations read the type cache to find the
writer/decoder function responsible for a type. When analyzing CPU
profiles of geth during sync, I found that the use of sync.RWMutex in
cache lookups appears in the profiles. It seems we are running into
CPU cache contention problems when package rlp is heavily used
on all CPU cores during sync.
This change makes it use atomic.Value + a writer lock instead of
sync.RWMutex. In the common case where the typeinfo entry is present in
the cache, we simply fetch the map and lookup the type.
* rlp: optimize byte array handling (#22924)
This change improves the performance of encoding/decoding [N]byte.
name old time/op new time/op delta
DecodeByteArrayStruct-8 336ns ± 0% 246ns ± 0% -26.98% (p=0.000 n=9+10)
EncodeByteArrayStruct-8 225ns ± 1% 148ns ± 1% -34.12% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
DecodeByteArrayStruct-8 120B ± 0% 48B ± 0% -60.00% (p=0.000 n=10+10)
EncodeByteArrayStruct-8 0.00B 0.00B ~ (all equal)
* rlp: optimize big.Int decoding for size <= 32 bytes (#22927)
This change grows the static integer buffer in Stream to 32 bytes,
making it possible to decode 256bit integers without allocating a
temporary buffer.
In the recent commit 088da24, Stream struct size decreased from 120
bytes down to 88 bytes. This commit grows the struct to 112 bytes again,
but the size change will not degrade performance because Stream
instances are internally cached in sync.Pool.
name old time/op new time/op delta
DecodeBigInts-8 12.2µs ± 0% 8.6µs ± 4% -29.58% (p=0.000 n=9+10)
name old speed new speed delta
DecodeBigInts-8 230MB/s ± 0% 326MB/s ± 4% +42.04% (p=0.000 n=9+10)
* eth/protocols/eth, les: avoid Raw() when decoding HashOrNumber (#22841)
Getting the raw value is not necessary to decode this type, and
decoding it directly from the stream is faster.
* fix testcase
* debug no lazy
* fix can not repair
* address comments
Co-authored-by: Felix Lange <fjl@twurst.com>
This change adds the --catalyst flag, enabling an RPC API for eth2 integration.
In this initial version, catalyst mode also disables all peer-to-peer networking.
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
The PR implements the --miner.notify.full flag that enables full pending block
notifications. When this flag is used, the block notifications sent to mining
endpoints contain the complete block header JSON instead of a work package
array.
Co-authored-by: AlexSSD7 <alexandersadovskyi7@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
This adds support for EIP-2718 typed transactions as well as EIP-2930
access list transactions (tx type 1). These EIPs are scheduled for the
Berlin fork.
There very few changes to existing APIs in core/types, and several new APIs
to deal with access list transactions. In particular, there are two new
constructor functions for transactions: types.NewTx and types.SignNewTx.
Since the canonical encoding of typed transactions is not RLP-compatible,
Transaction now has new methods for encoding and decoding: MarshalBinary
and UnmarshalBinary.
The existing EIP-155 signer does not support the new transaction types.
All code dealing with transaction signatures should be updated to use the
newer EIP-2930 signer. To make this easier for future updates, we have
added new constructor functions for types.Signer: types.LatestSigner and
types.LatestSignerForChainID.
This change also adds support for the YoloV3 testnet.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com>
This PR adds a more CLI flag, so that the les-server can serve light clients even the local node is not synced yet.
This functionality is needed in some testing environments(e.g. hive). After launching the les server, no more blocks will be imported so the node is always marked as "non-synced".
This PR prevents users from submitting transactions without EIP-155 enabled. This behaviour can be overridden by specifying the flag --rpc.allow-unprotected-txs=true.
This PR introduces:
- db.put to put a value into the database
- db.get to read a value from the database
- db.delete to delete a value from the database
- db.stats to check compaction info from the database
- db.compact to trigger a db compaction
It also moves inspectdb to db.inspect.
This moves the eth config definition into a separate package, eth/ethconfig.
Packages eth and les can now import this common package instead of
importing eth from les, reducing dependencies.
Co-authored-by: Felix Lange <fjl@twurst.com>
This change allows users to set a custom path prefix on which to mount the http-rpc
or ws-rpc handlers via the new flags --http.rpcprefix and --ws.rpcprefix.
Fixes#21826
Co-authored-by: Felix Lange <fjl@twurst.com>
Removes the yolov2 definition, adds yolov3, including EIP-2565. This PR also disables some of the erroneously generated blockchain and statetests, and adds the new genesis hash + alloc for yolov3.
This PR disables the CLI switches for yolo, since it's not complete until we merge support for 2930.
Adding warnings of free disk space left and graceful shutdown when there is not enough space left.
This also adds a flag datadir.minfreedisk which can be used to set the trigger for low disk space, and setting it to zero disables the check.
Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
USB enumeration still occured. Make sure it will only occur if --usb is set.
This also deprecates the 'NoUSB' config file option in favor of a new option 'USB'.
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md) , but does not enable it by default.
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
* cmd/geth: implement vulnerability check
* cmd/geth: use minisign to verify vulnerability feed
* cmd/geth: add the test too
* cmd/geth: more minisig/signify testing
* cmd/geth: support multiple pubfiles for signing
* cmd/geth: add @holiman minisig pubkey
* cmd/geth: polishes on vulnerability check
* cmd/geth: fix ineffassign linter nit
* cmd/geth: add CVE to version check struct
* cmd/geth/testdata: add missing testfile
* cmd/geth: add more keys to versionchecker
* cmd/geth: support file:// URLs in version check
* cmd/geth: improve key ID printing when signature check fails
Co-authored-by: Felix Lange <fjl@twurst.com>
