Commit Graph

13006 Commits

Author SHA1 Message Date
Martin Holst Swende
1e1865b73f
core: implement background trie prefetcher
Squashed from the following commits:

core/state: lazily init snapshot storage map
core/state: fix flawed meter on storage reads
core/state: make statedb/stateobjects reuse a hasher
core/blockchain, core/state: implement new trie prefetcher
core: make trie prefetcher deliver tries to statedb
core/state: refactor trie_prefetcher, export storage tries
blockchain: re-enable the next-block-prefetcher
state: remove panics in trie prefetcher
core/state/trie_prefetcher: address some review concerns

sq
2021-01-21 01:46:38 +02:00
meowsbits
81bf9f97c9
downloader: extract findAncestor search functions (#21744)
This is a simple refactoring, extracting common ancestor
negotiation logic to named function
2021-01-20 22:45:01 +01:00
ucwong
7da8f75d5b
go.mod: upgrade golang-lru (#22134) 2021-01-20 19:34:21 +01:00
Marius van der Wijden
d1301eb0df
oss-fuzz: fix abi fuzzer (#22199) 2021-01-20 18:21:13 +01:00
gary rong
45cb1a580a
eth, les: add new config field SyncFromCheckpoint (#22123)
This PR introduces a new config field SyncFromCheckpoint for light client.

In some special scenarios, it's required to start synchronization from some
arbitrary checkpoint or even from the scratch. So this PR offers this
flexibility to users so that the synchronization start point can be configured.

There are two relevant configs: SyncFromCheckpoint and Checkpoint.

- If the SyncFromCheckpoint is true, the light client will try to sync from the
  specified checkpoint.

- If the Checkpoint is not configured, then the light client will sync from the
  scratch(from the latest header if the database is not empty)

Additional notes: these two configs are not visible in the CLI flags but only
accessable in the config file.

Example Usage:

[Eth]
SyncFromCheckpoint = true

[Eth.Checkpoint]
SectionIndex = 100
SectionHead = "0xabc"
CHTRoot = "0xabc"
BloomRoot = "0xabc"

PS. Historical checkpoint can be retrieved from the synced full node or light
client via les_getCheckpoint API.
2021-01-19 10:52:45 +01:00
Alex Mazalov
24c1e3053b
cmd/geth: graceful shutdown if disk is full (#22103)
Adding warnings of free disk space left and graceful shutdown when there is not enough space left.
This also adds a flag datadir.minfreedisk which can be used to set the trigger for low disk space, and setting it to zero disables the check. 

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Felix Lange <fjl@twurst.com>
2021-01-19 09:26:42 +01:00
gary rong
5e9f5ca5d3
core/state/snapshot: write snapshot generator in batch (#22163)
* core/state/snapshot: write snapshot generator in batch

* core: refactor the tests

* core: update tests

* core: update tests
2021-01-18 14:39:43 +01:00
Marius van der Wijden
10555d4684
cmd/geth: dump config for metrics (#22083)
* cmd/geth: dump config

* cmd/geth: dump config

* cmd/geth: properly read config again

* cmd/geth: override metrics if flags are set

* cmd/geth: write metrics regardless if enabled

* cmd/geth: renamed to metricsfromcliargs

* metrics: add default configuration
2021-01-18 14:36:05 +01:00
Marius van der Wijden
398182284c
tests/fuzzers/abi: better test generation (#22158)
* tests/fuzzers/abi: better test generation

* tests/fuzzers/abi: fixed packing issue

* oss-fuzz: enable abi fuzzer
2021-01-18 14:33:15 +01:00
gary rong
034ecc3210
les: remove useless protocol defines (#22115)
This PR has two changes in the les protocol:

- the auxRoot is not supported. See ethereum/devp2p#171 for more information
- the empty response will be returned in GetHelperTrieProofsMsg request if the merkle
   proving is failed. note, for backward compatibility, the empty merkle proof as well as
   the request auxiliary data will still be returned in  les2/3 protocol no matter the proving
   is successful or not. the proving failure can happen e.g. request the proving for a
   non-included entry in helper trie (unstable header).
2021-01-16 19:06:18 +01:00
Dan DeGreef
c76573a97b
eth/protocols/eth: fix slice resize flaw (#22181) 2021-01-16 18:15:18 +01:00
gary rong
8d62ee65b2
les: don't drop sentTo for normal cases (#22048) 2021-01-15 23:04:38 +01:00
Péter Szilágyi
3944976a9a
Merge pull request #22177 from karalabe/snapshot-storage-logs
core/state/snapshot: add generation logs to storage too
2021-01-15 12:54:34 +02:00
Péter Szilágyi
c4deebbf1e
core/state/snapshot: add generation logs to storage too 2021-01-15 12:26:46 +02:00
Péter Szilágyi
d13c59fef0
Merge pull request #22169 from karalabe/faucet-regen
cmd/faucet: update the embedded website asset
2021-01-14 12:19:57 +02:00
Péter Szilágyi
12969084d1
cmd/faucet: update the embedded website asset 2021-01-14 12:10:52 +02:00
Martin Holst Swende
96157a897b
graphql: fix spurious travis failure (#22166)
The tests sometimes failed with certain go versions because
the behavior of http.Server.Shutdown changed over time. A bug
that was fixed in Go 1.15 could cause active connections on unrelated
servers to close unexpectedly. This is fixed by avoiding use of the
same port number in all tests.
2021-01-13 22:43:07 +01:00
Marius van der Wijden
2aaff0ad76
consensus/ethash: increase seal timeout for tests (#22162)
It seems that the 2 second timeout is not enough for Travis CI:

   --- FAIL: TestTestMode (2.00s)
       ethash_test.go:53: sealing result timeout
2021-01-13 11:44:20 +01:00
Martin Holst Swende
6296211a3e
graphql: fix spurious error in test (#22164)
This solves an issue in graphql tests:

    graphql_test.go:38: could not create new node: datadir already used by another process
2021-01-13 11:42:26 +01:00
Martin Holst Swende
c94081774f
tests: update the reference tests (#22009) 2021-01-13 11:29:28 +01:00
Guillaume Ballet
c7a6be163f
cmd/utils: don't enumerate USB unless --usb is set (#22130)
USB enumeration still occured. Make sure it will only occur if --usb is set.
This also deprecates the 'NoUSB' config file option in favor of a new option 'USB'.
2021-01-13 11:14:36 +01:00
Martin Holst Swende
93a89b2681
go.mod: use github.com/holiman/bloomfilter/v2 (#22044)
* deps: use improved bloom filter implementation

* eth/handler, trie: use 4 keys for syncbloom + minor fixes

* eth/protocols, trie: revert change on syncbloom method signature
2021-01-12 17:39:31 +01:00
meowsbits
23f837c388
cmd/utils: avoid making console preloads absolute (#22109)
Resolves https://github.com/etclabscore/core-geth/issues/273

jsre.JSRE already handles establishing preload
file paths relative to the 'assets' path (aka docroot),
where it joins the assets dir and the file path if relative,
or uses the file path only if absolute.

The duplication of this logic by MakeConsolePreloads
caused preloaded files to have paths which contained
duplicate references to the assets dir path.

Date: 2020-12-30 08:25:01-06:00
Signed-off-by: meows <b5c6@protonmail.com>
2021-01-12 15:50:11 +01:00
rene
984e752ce5
eth: return error from eth_chainID during sync before EIP-155 activates (#21686)
This changes the chainID RPC method to return an error when EIP-155 is not yet
active at the current block height. It used to simply return zero in this case, but
that's confusing.
2021-01-12 10:52:13 +01:00
rene
39b3b8ffb4
graphql: fix issue with unmarshalling int32 into Long type #22153 2021-01-11 14:55:42 +01:00
Marius van der Wijden
49c2816d54
eth: improve log message (#22146)
* eth: fixed typos

* eth: fixed log message
2021-01-11 12:53:13 +01:00
Péter Szilágyi
79e2174e4d
Merge pull request #22157 from karalabe/prque-tests
common/prque: pull in tests and benchmarks from upstream
2021-01-11 10:52:46 +02:00
Péter Szilágyi
ab5e3f400f
common/prque: pull in tests and benchmarks from upstream 2021-01-11 10:31:03 +02:00
gary rong
5a1b384352
core: persist bad blocks (#21827)
* core: persist bad blocks

* core, eth, internal: address comments

* core/rawdb: add badblocks to inspector

* core, eth: update

* internal: revert

* core, eth: only save 10 bad blocks

* core/rawdb: address comments

* core/rawdb: fix

* core: address comments
2021-01-10 12:54:15 +01:00
Chris Ziogas
89030ec0b4
eth/downloader: fix race condition in tests (#22140)
* downloader: fix race condition in tests

* eth/downloader: fix race condition in tests

* Revert "downloader: fix race condition in tests"

This reverts commit 108033ebc6985de83791d375b6e6647a77d28d5a.
2021-01-09 17:29:19 +01:00
Marius van der Wijden
889f5645b5
ethclient: better test suite for ethclient package (#22127)
This commit extends the ethclient test suite and increases code coverage of the ethclient
package from ~15% to >55%. These tests act as early smoke tests to signal issues in the
RPC-interface. E.g. if a functionality like eth_chainId or eth_call breaks, the test
will break.
2021-01-08 21:29:25 +01:00
Martin Holst Swende
6b88ab75bc
cmd/faucet: fix nonce-gap problem (#22145)
* cmd/faucet: avoid encoding for each client

* cmd/faucet: fix flaw in clearing of txs, avoid sending more than necessary

* cmd/faucet: fix flaw in tx cropping

* cmd/faucet: revert change to not always send tx info

* cmd/faucet: review fixes

* cmd/faucet: revert #22018, fix order in UI

* cmd/faucet: fix lock error

* cmd/faucet: revert json changes

* squashme
2021-01-08 12:17:15 +02:00
gary rong
165f53fc6e
les: remove transaction propagation limits (#22125) 2021-01-07 23:39:35 +01:00
Péter Szilágyi
d3952898c3
Merge pull request #22137 from karalabe/faucet-fb-fix
cmd/faucet: switch Facebook auth over to mobile site
2021-01-07 18:45:27 +02:00
Péter Szilágyi
3c6665e7d6
cmd/faucet: switch Facebook auth over to mobile site 2021-01-07 18:14:44 +02:00
Martin Holst Swende
4bb5c6ca7a
eth/protocols/snap: speed up hash checks (#22023)
* eth/protocols/snap: speed up hash checks

* eth/protocols/snap: nit fix

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-01-07 18:12:41 +02:00
Péter Szilágyi
38310f9022
Merge pull request #22136 from karalabe/faucet-websocket-fix
cmd/faucet: fix websocket race regression after switching to gorilla
2021-01-07 12:58:42 +02:00
Martin Holst Swende
58b9db5f7c
eth/protocols/snap: track reverts when peer rejects request (#22016)
* eth/protocols/snap: reschedule missed deliveries

* eth/protocols/snap: clarify log message

* eth/protocols/snap: revert failures async and update runloop

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
2021-01-07 12:58:07 +02:00
Péter Szilágyi
44208d9258
cmd/faucet: fix websocket race regression after switching to gorilla 2021-01-07 10:23:50 +02:00
Péter Szilágyi
8bd8e1b24a
Merge pull request #22122 from karalabe/snapshot-polishes
cmd/utils, eth/downloader: minor snap nitpicks
2021-01-07 09:12:20 +02:00
Melvin Junhee Woo
d2e1b17f18
snapshot, trie: fixed typos, mostly in snapshot pkg (#22133) 2021-01-07 08:36:21 +02:00
rene
072fd96254
graphql: return decimal for estimateGas and cumulativeGas queries (#22126)
* estimateGas, cumulativeGas
* linted
* add test for estimateGas
2021-01-06 17:19:16 +01:00
Marius van der Wijden
d667ee2d10
crypto: fix ineffectual assignments (#22124)
* crypto/bls12381: fixed ineffectual assignment

* crypto/signify: fix ineffectual assignment
2021-01-06 13:06:44 +02:00
Péter Szilágyi
83d317cff9
cmd/utils, eth/downloader: minor snap nitpicks 2021-01-06 08:37:45 +02:00
jk-jeongkyun
618454214b
eth/downloader: enhanced test cases for downloader queue (#22114) 2021-01-05 14:56:01 +01:00
Marius van der Wijden
9ba306d47e
common/compiler: fix parsing of solc output with solidity v.0.8.0 (#22092)
Solidity 0.8.0 changes the way that output is marshalled. This patch allows to parse both
the legacy format used previously and the new format.

See also https://docs.soliditylang.org/en/breaking/080-breaking-changes.html#interface-changes
2021-01-05 14:48:22 +01:00
Marius van der Wijden
4714ce9430
cmd/geth: added --mainnet flag (#21932)
* cmd/geth: added --mainnet flag

* cmd/utils: set default genesis if --mainnet is specified

* cmd/utils: addressed comments
2021-01-05 14:31:23 +01:00
Antoine Toulme
eb2a1dfdd2
graphql: use a decimal representation for gas limit and gas used (#21883)
This changes the JSON encoding of blocks returned by the API
to have decimal instead of hexadecimal numbers. The spec wants
it this way.

Co-authored-by: Martin Holst Swende <martin@swende.se>
2021-01-05 11:22:32 +01:00
Guillaume Ballet
664903dc88
cmd/geth: usb is off by default (#21984) 2021-01-05 11:18:22 +01:00
Martin Holst Swende
9584f56b9d
miner: avoid sleeping in miner (#22108)
This PR removes a logic in the miner, which was originally intended to help temporary testnets based on ethash from "running off into the future". If the difficulty was low, and a few computers started mining several blocks per second, the ethash rules (which demand 1s delay between blocks) would push the blocktimes further and further away.
The solution was to make the miner sleep while this happened.

Nowadays, this problem is solved instead by PoA chains, and it's recommended to let testnets and devnets be based on clique instead. The existing logic is problematic, since it can cause stalls within the miner making it difficult for remote workers to submit work if the channel is blocked on a sleep.

Credits to Saar Tochner for reporting this via the bug bounty
2021-01-05 10:44:33 +01:00