When we map a file for generating the DAG, we do a simple truncate to e.g. 1Gb. This is fine, even if we have nowhere near 1Gb disk available, as the actual file doesn't take up the full 1Gb, merely a few bytes. When we start generating into it, however, it eventually crashes with a unexpected fault address .
This change fixes it (on linux systems) by using the Fallocate syscall, which preallocates suffcient space on disk to avoid that situation.
Co-authored-by: Felix Lange <fjl@twurst.com>
This removes some code:
- The clique engine calculated the snapshot twice when verifying headers/blocks.
- The method GetBlockHashesFromHash in Header/Block/Lightchain was only used by tests. It
is now removed from the API.
- The method GetTdByHash internally looked up the number before calling GetTd(hash, num).
In many cases, callers already had the number, and used this method just because it has a
shorter name. I have removed the method to make the API surface smaller.
* accounts/abi/bind: fix bounded contracts and sim backend for 1559
* accounts/abi/bind, ethclient: don't rely on chain config for gas prices
* all: enable London for all internal tests
* les: get receipt type info in les tests
* les: fix weird test
Co-authored-by: Martin Holst Swende <martin@swende.se>
This is the initial implementation of EIP-1559 in packages core/types and core.
Mining, RPC, etc. will be added in subsequent commits.
Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: lightclient@protonmail.com <lightclient@protonmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
This change adds the --catalyst flag, enabling an RPC API for eth2 integration.
In this initial version, catalyst mode also disables all peer-to-peer networking.
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
The PR implements the --miner.notify.full flag that enables full pending block
notifications. When this flag is used, the block notifications sent to mining
endpoints contain the complete block header JSON instead of a work package
array.
Co-authored-by: AlexSSD7 <alexandersadovskyi7@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
The PR makes use of the stacktrie, which is is more lenient on resource consumption, than the regular trie, in cases where we only need it for DeriveSha
This PR implements the following modifications
- Don't shortcut check if block is present, thus avoid disk lookup
- Don't check hash ancestry in early-check (it's still done in parallel checker)
- Don't check time.Now for every single header
Charts and background info can be found here: https://github.com/holiman/headerimport/blob/main/README.md
With these changes, writing 1M headers goes down to from 80s to 62s.
This PR adds re-written difficulty calculators, which are based on uint256. It also adds a fuzzer + oss-fuzz integration for the new fuzzer. It does differential fuzzing between the new and old calculators.
Note: this PR does not actually enable the new calculators.
* consensus/ethash: only use *reflect.SliceHeader, not reflect.SliceHeader. See comment here: https://github.com/golang/go/issues/40397\#issuecomment-663748689
* consensus/ethash: pr feedback from @mdempsky, makes a copy of dest such that is not mutated
* consensus/ethash: remove noop assign
* consensus/ethash: apply same fix to another location
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
This change introduces garbage collection for the light client. Historical
chain data is deleted periodically. If you want to disable the GC, use
the --light.nopruning flag.
* cmd, core, eth: init tx lookup in background
* core/rawdb: tiny log fixes to make it clearer what's happening
* core, eth: fix rebase errors
* core/rawdb: make reindexing less generic, but more optimal
* rlp: implement rlp list iterator
* core/rawdb: new implementation of tx indexing/unindex using generic tx iterator and hashing rlp-data
* core/rawdb, cmd/utils: fix review concerns
* cmd/utils: fix merge issue
* core/rawdb: add some log formatting polishes
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
The original idea behind this change was to remove a use of the
deprecated CancelRequest method. Simply removing it would've been an
option, but I couldn't resist and did a bit of a refactoring instead.
All remote sealing code was contained in a single giant function. Remote
sealing is now extracted into its own object, remoteSealer.
This PR introduces clique_status which gives info about the health of
the clique network.
It's currently a bit PITA to find out how a clique network is
performing, and it can easily happen that sealers drop off -- and
everything is 'fine' until one more signer drops off, and the network
suddenly halts.
The new method provides the following stats:
- Which signers are currently active, and have signed blocks in the last
N (set to 64) blocks?
- How many blocks has each signer signed?
- What is the difficulty in the last N blocks, compared to the
theoretical maximum?
* travis: Enable ARM support
* Include fixes from 20039
* Add a trace to debug the invalid lookup issue
* Try increasing the timeout to see if the arm test passes
* Investigate the resolver issue
* Increase arm64 timeout for clique test
* increase timeout in tests for arm64
* Only test the failing tests
* Review feedback: don't export epsilon
* Remove investigation tricks+include fjl's feeback
* Revert the retry ahead of using the mock resolver
* Fix rebase errors
* core: fix import errors on clique crashes + empty blocks
* cosensus/clique, core: add test for the mirrored state issue
* core: address todo question wrt log count
* core: raise a louder warning for non-clique known blocks
* Named functions and defined a basic EIP191 content type list
* Written basic content type functions
* Added ecRecover method in the clef api
* Updated the extapi changelog and addded indications in the README
* Changed the version of the external API
* Added tests for 0x45
* Implementing UnmarshalJSON() for TypedData
* Working on TypedData
* Solved the auditlog issue
* Changed method to signTypedData
* Changed mimes and implemented the 'encodeType' function for EIP-712
* Polished docstrings, ran goimports and swapped fmt.Errorf with errors.New where possible
* Drafted recursive encodeData
* Ran goimports and gofmt
* Drafted first version of EIP-712, including tests
* Temporarily switched to using common.Address in tests
* Drafted text/validator and and rewritten []byte as hexutil.Bytes
* Solved stringified address encoding issue
* Changed the property type required by signData from bytes to interface{}
* Fixed bugs in 'data/typed' signs
* Brought legal warning back after temporarily disabling it for development
* Added example RPC calls for account_signData and account_signTypedData
* Named functions and defined a basic EIP191 content type list
* Written basic content type functions
* Added ecRecover method in the clef api
* Updated the extapi changelog and addded indications in the README
* Added tests for 0x45
* Implementing UnmarshalJSON() for TypedData
* Working on TypedData
* Solved the auditlog issue
* Changed method to signTypedData
* Changed mimes and implemented the 'encodeType' function for EIP-712
* Polished docstrings, ran goimports and swapped fmt.Errorf with errors.New where possible
* Drafted recursive encodeData
* Ran goimports and gofmt
* Drafted first version of EIP-712, including tests
* Temporarily switched to using common.Address in tests
* Drafted text/validator and and rewritten []byte as hexutil.Bytes
* Solved stringified address encoding issue
* Changed the property type required by signData from bytes to interface{}
* Fixed bugs in 'data/typed' signs
* Brought legal warning back after temporarily disabling it for development
* Added example RPC calls for account_signData and account_signTypedData
* Polished and fixed PR
* Polished and fixed PR
* Solved malformed data panics and also wrote tests
* Solved malformed data panics and also wrote tests
* Added alphabetical sorting to type dependencies
* Added alphabetical sorting to type dependencies
* Added pretty print to data/typed UI
* Added pretty print to data/typed UI
* signer: more tests for typed data
* signer: more tests for typed data
* Fixed TestMalformedData4 errors and renamed IsValid to Validate
* Fixed TestMalformedData4 errors and renamed IsValid to Validate
* Fixed more new failing tests and deanonymised some functions
* Fixed more new failing tests and deanonymised some functions
* Added types to EIP712 output in cliui
* Added types to EIP712 output in cliui
* Fixed regexp issues
* Fixed regexp issues
* Added pseudo-failing test
* Added pseudo-failing test
* Fixed false positive test
* Fixed false positive test
* Added PrettyPrint method
* Added PrettyPrint method
* signer: refactor formatting and UI
* signer: make ui use new message format for signing
* Fixed breaking changes
* Fixed rules_test failing test
* Added extra regexp for reference types
* signer: more hard types
* Fixed failing test, formatted files
* signer: use golang/x keccak
* Fixed goimports error
* clef, signer: address some review concerns
* Implemented latest recommendations
* Fixed comments and uintint256 issue
* accounts, signer: fix mimetypes, add interface to sign data with passphrase
* signer, accounts: remove duplicated code, pass hash preimages to signing
* signer: prevent panic in type assertions, make cliui print rawdata as quotable-safe
* signer: linter fixes, remove deprecated crypto dependency
* accounts: fix goimport
* miner: commit state which is relative with sealing result
* consensus, core, miner, mobile: introduce sealHash interface
* miner: evict pending task with threshold
* miner: go fmt
* consensus/ethash: start remote ggoroutine to handle remote mining
* consensus/ethash: expose remote miner api
* consensus/ethash: expose submitHashrate api
* miner, ethash: push empty block to sealer without waiting execution
* consensus, internal: add getHashrate API for ethash
* consensus: add three method for consensus interface
* miner: expose consensus engine running status to miner
* eth, miner: specify etherbase when miner created
* miner: commit new work when consensus engine is started
* consensus, miner: fix some logics
* all: delete useless interfaces
* consensus: polish a bit
* consensus/ethash: add maxEpoch constant
* consensus/ethash: improve cache/dataset handling
There are two fixes in this commit:
Unmap the memory through a finalizer like the libethash wrapper did. The
release logic was incorrect and freed the memory while it was being
used, leading to crashes like in #14495 or #14943.
Track caches and datasets using simplelru instead of reinventing LRU
logic. This should make it easier to see whether it's correct.
* consensus/ethash: restore 'future item' logic in lru
* consensus/ethash: use mmap even in test mode
This makes it possible to shorten the time taken for TestCacheFileEvict.
* consensus/ethash: shuffle func calc*Size comments around
* consensus/ethash: ensure future cache/dataset is in the lru cache
* consensus/ethash: add issue link to the new test
* consensus/ethash: fix vet
* consensus/ethash: fix test
* consensus: tiny issue + nitpick fixes
* cmd, consensus, eth: split ethash related config to it own
* eth, consensus: minor polish
* eth, consenus, console: compress pow testing config field to single one
* consensus, eth: document pow mode
* cmd, consensus, core, miner: instatx clique for --dev
* cmd, consensus, clique: support configurable --dev block times
* cmd, core: allow --dev to use persistent storage too
Tests are now included as a submodule. This should make updating easier
and removes ~60MB of JSON data from the working copy.
State tests are replaced by General State Tests, which run the same test
with multiple fork configurations.
With the new test runner, consensus tests are run as subtests by walking
json files. Many hex issues have been fixed upstream since the last
update and most custom parsing code is replaced by existing JSON hex
types. Tests can now be marked as 'expected failures', ensuring that
fixes for those tests will trigger an update to test configuration. The
new test runner also supports parallel execution and the -short flag.
This PR polishes the EIP 100 difficulty adjustment algorithm
to match the same mechanisms as the Homestead was implemented
to keep the code uniform. It also avoids a few memory allocs
by reusing big1 and big2, pulling it out of the common package
and into ethash.
The commit also fixes chain maker to forward the uncle hash
when creating a simulated chain (it wasn't needed until now
so we just skipped a copy there).
`geth makedag <blocknumber> <path>` was creating DAGs for
`<blocknumber>/<epoch_length> + 1`, hence
it was impossible to create an epoch 0 DAG.
This fixes the calculations in `consensus/ethash/ethash.go` for
`MakeDataset` and `MakeCache`, and applies `gofmt`.
This commit is a preparation for the upcoming metropolis hardfork. It
prepares the state, core and vm packages such that integration with
metropolis becomes less of a hassle.
* Difficulty calculation requires header instead of individual
parameters
* statedb.StartRecord renamed to statedb.Prepare and added Finalise
method required by metropolis, which removes unwanted accounts from
the state (i.e. selfdestruct)
* State keeps record of destructed objects (in addition to dirty
objects)
* core/vm pre-compiles may now return errors
* core/vm pre-compiles gas check now take the full byte slice as argument
instead of just the size
* core/vm now keeps several hard-fork instruction tables instead of a
single instruction table and removes the need for hard-fork checks in
the instructions
* core/vm contains a empty restruction function which is added in
preparation of metropolis write-only mode operations
* Adds the bn256 curve
* Adds and sets the metropolis chain config block parameters (2^64-1)
This commit adds pluggable consensus engines to go-ethereum. In short, it
introduces a generic consensus interface, and refactors the entire codebase to
use this interface.
