tornadocontrib/bsc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,549 Commits 35 Branches 91 Tags 289 MiB
8c90dc946a
Commit Graph

84 Commits

Author SHA1 Message Date
aaronbuchwald
8c90dc946a core/rawdb: simplify TestDiskSeek to use memorydb (#25182) 2022-10-21 14:23:12 +08:00
Marius van der Wijden
e8b93382da core/state/snapshot: fix race condition (#24685) 
Fixes three race conditions found through fuzzing by David Theodore
 2022-09-21 17:02:02 +08:00
Péter Szilágyi
f310921b41 core/state/snapshot: remove noop map item assignment 2022-09-21 17:02:02 +08:00
Leon
76e3c9071a [R4R] Pipecommit enable trie prefetcher (#992) 2022-08-01 17:41:50 +08:00
j75689
79bd42836c fix: code compatibility fixes 2022-07-05 11:14:21 +08:00
zjubfd
a50a05d624 merge with go-ethereum 1.1.5 2022-02-10 18:48:16 +08:00
Martin Holst Swende
51eb5f8ca8
cmd/geth: add db cmd to show metadata (#23900) 
* cmd/geth: add db cmd to show metadata

* cmd/geth: better output generator status

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>

* cmd: minor

Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
 2022-01-18 12:30:41 +02:00
Taeik Lim
85064ed09b
all: fix 'the the' in comments (#24036) 2021-12-02 15:42:09 +01:00
Martin Holst Swende
9055cc14ec
core/vm, core/state/snapshot: remove unused code (#23956) 
* core/state/snapshot: remove wiper functionality

* core/vm: remove unused 'unofficial' opcodes
 2021-11-25 10:37:47 +02:00
Martin Holst Swende
526c3f6b9e
core/state/snapshot: fix benchmarks (#23804) 2021-10-26 11:01:01 +02:00
Ziyuan Zhong
312e02bca9
core/state/snapshot: fix BAD BLOCK error when snapshot is generating (#23635) 
* core/state/snapshot: fix BAD BLOCK error when snapshot is generating

* core/state/snapshot: alternative fix for the snapshot generator

* add comments and minor update

Co-authored-by: Martin Holst Swende <martin@swende.se>
 2021-10-20 10:05:27 +02:00
rjl493456442
f915f6873f
core/state/snapshot: fix data race in layer flattening (#23628) 
* core/state/snapshot: fix data race in layer flattening

* core/state/snapshot: fix typo
 2021-10-15 10:52:40 +03:00
Ikko Ashimine
31be5d41d9
core: fix typo in iterator.go (#23502) 
strorage -> storage
 2021-08-31 10:21:42 +02:00
gary rong
a5a5237178
core, light, tests, trie: add state metrics (#23433) 2021-08-24 22:00:42 +03:00
Péter Szilágyi
9d6480c3cd
core/state/snapshot: gofmt 2021-08-10 16:58:38 +03:00
Tyler Chambers
66948316f7
core/state/snapshot: clarify comment about snapshot repair (#23305) 
Co-authored-by: Tyler Chambers <me@tylerchambers.net>
Co-authored-by: Felix Lange <fjl@twurst.com>
 2021-08-10 11:16:53 +02:00
Ziyuan Zhong
57d9e0ac75
core/state/snapshot: fix typo in comment (#23219) 2021-08-10 11:04:29 +02:00
gary rong
c73652da0b
core/state/snapshot: fix flaky tests (#22944) 
* core/state/snapshot: fix flaky tests

* core/state/snapshot: fix tests
 2021-05-26 10:58:09 +03:00
fudongbai
3ab97c2a83 disable log of snap update 2021-05-10 17:10:11 +08:00
zjubfd
031607200a
Merge pull request #186 from binance-chain/upgrade_tmp 
[R4R] make prune tool available
 2021-05-06 12:00:34 +08:00
fudongbai
3d42dd1105 tolerate unindex tx light client 2021-05-04 21:23:46 +08:00
Péter Szilágyi
745757ac6b
core, eth: abort snapshot generation on snap sync and resume later 2021-04-30 17:03:10 +03:00
Péter Szilágyi
fae165a5de
core, eth, ethdb, trie: simplify range proofs 2021-04-29 10:59:08 +03:00
fudongbai
c92a53d2bf add more logs 2021-04-29 13:24:22 +08:00
Martin Holst Swende
a81cf0d2b3
trie: remove redundant returns + use stacktrie where applicable (#22760) 
* trie: add benchmark for proofless range

* trie: remove unused returns + use stacktrie
 2021-04-28 22:47:48 +03:00
Martin Holst Swende
49281ab84f
core/state/snapshot, true: reuse dirty data instead of hitting disk when generating (#22667) 
* core/state/snapshot: reuse memory data instead of hitting disk when generating

* trie: minor nitpicks wrt the resolver optimization

* core/state/snapshot, trie: use key/value store for resolver

* trie: fix linter

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 2021-04-23 14:39:18 +03:00
gary rong
d6ffa14035
core: nuke legacy snapshot supporting (#22663) 2021-04-20 08:27:46 +03:00
Martin Holst Swende
09d44e9925
core/state/snapshot: avoid copybytes for stacktrie 2021-04-16 14:58:23 +02:00
gary rong
7088f1e814
core, eth: faster snapshot generation (#22504) 
* eth/protocols: persist received state segments

* core: initial implementation

* core/state/snapshot: add tests

* core, eth: updates

* eth/protocols/snapshot: count flat state size

* core/state: add metrics

* core/state/snapshot: skip unnecessary deletion

* core/state/snapshot: rename

* core/state/snapshot: use the global batch

* core/state/snapshot: add logs and fix wiping

* core/state/snapshot: fix

* core/state/snapshot: save generation progress even if the batch is empty

* core/state/snapshot: fixes

* core/state/snapshot: fix initial account range length

* core/state/snapshot: fix initial account range

* eth/protocols/snap: store flat states during the healing

* eth/protocols/snap: print logs

* core/state/snapshot: refactor (#4)

* core/state/snapshot: refactor

* core/state/snapshot: tiny fix and polish

Co-authored-by: rjl493456442 <garyrong0905@gmail.com>

* core, eth: fixes

* core, eth: fix healing writer

* core, trie, eth: fix paths

* eth/protocols/snap: fix encoding

* eth, core: add debug log

* core/state/generate: release iterator asap (#5)

core/state/snapshot: less copy

core/state/snapshot: revert split loop

core/state/snapshot: handle storage becoming empty, improve test robustness

core/state: test modified codehash

core/state/snapshot: polish

* core/state/snapshot: optimize stats counter

* core, eth: add metric

* core/state/snapshot: update comments

* core/state/snapshot: improve tests

* core/state/snapshot: replace secure trie with standard trie

* core/state/snapshot: wrap return as the struct

* core/state/snapshot: skip wiping correct states

* core/state/snapshot: updates

* core/state/snapshot: fixes

* core/state/snapshot: fix panic due to reference flaw in closure

* core/state/snapshot: fix errors in state generation logic + fix log output

* core/state/snapshot: remove an error case

* core/state/snapshot: fix condition-check for exhausted snap state

* core/state/snapshot: use stackTrie for small tries

* core/state/snapshot: don't resolve small storage tries in vain

* core/state/snapshot: properly clean up storage of deleted accounts

* core/state/snapshot: avoid RLP-encoding in some cases + minor nitpicks

* core/state/snapshot: fix error (+testcase)

* core/state/snapshot: clean up tests a bit

* core/state/snapshot: work in progress on better tests

* core/state/snapshot: polish code

* core/state/snapshot: fix trie iteration abortion trigger

* core/state/snapshot: fixes flaws

* core/state/snapshot: remove panic

* core/state/snapshot: fix abort

* core/state/snapshot: more tests (plus failing testcase)

* core/state/snapshot: more testcases + fix for failing test

* core/state/snapshot: testcase for malformed data

* core/state/snapshot: some test nitpicks

* core/state/snapshot: improvements to logging

* core/state/snapshot: testcase to demo error in abortion

* core/state/snapshot: fix abortion

* cmd/geth: make verify-state report the root

* trie: fix failing test

* core/state/snapshot: add timer metrics

* core/state/snapshot: fix metrics

* core/state/snapshot: udpate tests

* eth/protocols/snap: write snapshot account even if code or state is needed

* core/state/snapshot: fix diskmore check

* core/state/snapshot: review fixes

* core/state/snapshot: improve error message

* cmd/geth: rename 'error' to 'err' in logs

* core/state/snapshot: fix some review concerns

* core/state/snapshot, eth/protocols/snap: clear snapshot marker when starting/resuming snap sync

* core: add error log

* core/state/snapshot: use proper timers for metrics collection

* core/state/snapshot: address some review concerns

* eth/protocols/snap: improved log message

* eth/protocols/snap: fix heal logs to condense infos

* core/state/snapshot: wait for generator termination before restarting

* core/state/snapshot: revert timers to counters to track total time

Co-authored-by: Martin Holst Swende <martin@swende.se>
Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 2021-04-14 23:23:11 +03:00
Edgar Aroutiounian
c79fc209cd
core/state/snapshot: fix data race in diff layer (#22540) 2021-04-06 11:57:00 +03:00
Péter Szilágyi
61ff3e86b2
core/state/snapshot, ethdb: track deletions more accurately (#22582) 
* core/state/snapshot, ethdb: track deletions more accurately

* core/state/snapshot: don't reset the iterator, leveldb's screwy

* ethdb: don't mess with the insert batches for now
 2021-03-30 19:04:22 +03:00
Péter Szilágyi
a31f6d54df
core/state/snapshot: fix panic on missing parent 2021-03-22 22:41:28 +02:00
gary rong
0c70b83e00
all: add read-only option to database (#22407) 
* all: add read-only option to database

* all: fixes tests

* cmd/geth: migrate flags

* cmd/geth: fix the compact

* cmd/geth: fix the format

* cmd/geth: fix log

* cmd: add chain-readonly

* core: add readonly notion to freezer

* core/rawdb: add log

* core/rawdb: fix freezer close

* cmd: fix

* cmd, core: construct db

* core: update tests
 2021-03-22 19:06:30 +01:00
Péter Szilágyi
9ec3329899
core/state/snapshot: ensure Cap retains a min number of layers 2021-02-16 15:25:20 +02:00
gary rong
f566dd305e
all: bloom-filter based pruning mechanism (#21724) 
* cmd, core, tests: initial state pruner

core: fix db inspector

cmd/geth: add verify-state

cmd/geth: add verification tool

core/rawdb: implement flatdb

cmd, core: fix rebase

core/state: use new contract code layout

core/state/pruner: avoid deleting genesis state

cmd/geth: add helper function

core, cmd: fix extract genesis

core: minor fixes

contracts: remove useless

core/state/snapshot: plugin stacktrie

core: polish

core/state/snapshot: iterate storage concurrently

core/state/snapshot: fix iteration

core: add comments

core/state/snapshot: polish code

core/state: polish

core/state/snapshot: rebase

core/rawdb: add comments

core/rawdb: fix tests

core/rawdb: improve tests

core/state/snapshot: fix concurrent iteration

core/state: run pruning during the recovery

core, trie: implement martin's idea

core, eth: delete flatdb and polish pruner

trie: fix import

core/state/pruner: add log

core/state/pruner: fix issues

core/state/pruner: don't read back

core/state/pruner: fix contract code write

core/state/pruner: check root node presence

cmd, core: polish log

core/state: use HEAD-127 as the target

core/state/snapshot: improve tests

cmd/geth: fix verification tool

cmd/geth: use HEAD as the verification default target

all: replace the bloomfilter with martin's fork

cmd, core: polish code

core, cmd: forcibly delete state root

core/state/pruner: add hash64

core/state/pruner: fix blacklist

core/state: remove blacklist

cmd, core: delete trie clean cache before pruning

cmd, core: fix lint

cmd, core: fix rebase

core/state: fix the special case for clique networks

core/state/snapshot: remove useless code

core/state/pruner: capping the snapshot after pruning

cmd, core, eth: fixes

core/rawdb: update db inspector

cmd/geth: polish code

core/state/pruner: fsync bloom filter

cmd, core: print warning log

core/state/pruner: adjust the parameters for bloom filter

cmd, core: create the bloom filter by size

core: polish

core/state/pruner: sanitize invalid bloomfilter size

cmd: address comments

cmd/geth: address comments

cmd/geth: address comment

core/state/pruner: address comments

core/state/pruner: rename homedir to datadir

cmd, core: address comments

core/state/pruner: address comment

core/state: address comments

core, cmd, tests: address comments

core: address comments

core/state/pruner: release the iterator after each commit

core/state/pruner: improve pruner

cmd, core: adjust bloom paramters

core/state/pruner: fix lint

core/state/pruner: fix tests

core: fix rebase

core/state/pruner: remove atomic rename

core/state/pruner: address comments

all: run go mod tidy

core/state/pruner: avoid false-positive for the middle state roots

core/state/pruner: add checks for middle roots

cmd/geth: replace crit with error

* core/state/pruner: fix lint

* core: drop legacy bloom filter

* core/state/snapshot: improve pruner

* core/state/snapshot: polish concurrent logs to report ETA vs. hashes

* core/state/pruner: add progress report for pruning and compaction too

* core: fix snapshot test API

* core/state: fix some pruning logs

* core/state/pruner: support recovering from bloom flush fail

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 2021-02-08 13:16:30 +02:00
Alex Prut
ef84da8481
all: remove unneeded parentheses (#21921) 
* remove uneeded convertion type

* remove redundant type in composite literal

* omit explicit type where implicit

* remove unused redundant parenthesis

* remove redundant import alias duktape
 2021-02-02 11:32:44 +02:00
Melvin Junhee Woo
04a72260c5
snapshot: merge loops for better performance (#22160) 2021-01-25 14:25:55 +01:00
gary rong
5e9f5ca5d3
core/state/snapshot: write snapshot generator in batch (#22163) 
* core/state/snapshot: write snapshot generator in batch

* core: refactor the tests

* core: update tests

* core: update tests
 2021-01-18 14:39:43 +01:00
Péter Szilágyi
c4deebbf1e
core/state/snapshot: add generation logs to storage too 2021-01-15 12:26:46 +02:00
Martin Holst Swende
93a89b2681
go.mod: use github.com/holiman/bloomfilter/v2 (#22044) 
* deps: use improved bloom filter implementation

* eth/handler, trie: use 4 keys for syncbloom + minor fixes

* eth/protocols, trie: revert change on syncbloom method signature
 2021-01-12 17:39:31 +01:00
Melvin Junhee Woo
d2e1b17f18
snapshot, trie: fixed typos, mostly in snapshot pkg (#22133) 2021-01-07 08:36:21 +02:00
Melvin Junhee Woo
167ff563d1
core/state/snapshot: gethring -> gathering typo (#22104) 2021-01-04 10:07:43 +02:00
Péter Szilágyi
017831dd5b
core, eth: split eth package, implement snap protocol (#21482) 
This commit splits the eth package, separating the handling of eth and snap protocols. It also includes the capability to run snap sync (https://github.com/ethereum/devp2p/blob/master/caps/snap.md) , but does not enable it by default. 

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Co-authored-by: Martin Holst Swende <martin@swende.se>
 2020-12-14 10:27:15 +01:00
Péter Szilágyi
7b7b327ff2
core/state/snapshot: update generator marker in sync with flushes 2020-11-09 16:03:58 +02:00
gary rong
e6402677c2
core/state/snapshot: fix journal recovery from generating old journal (#21775) 
* core/state/snapshot: print warning if failed to resolve journal

* core/state/snapshot: fix snapshot recovery

When we meet the snapshot journal consisted with:
- disk layer generator with new-format
- diff layer journal with old-format

The base layer should be returned without error.
The broken diff layer can be reconstructed later
but we definitely don't want to reconstruct the
huge diff layer.

* core: add tests
 2020-11-04 13:41:46 +02:00
gary rong
b63e3c37a6
core: improve snapshot journal recovery (#21594) 
* core/state/snapshot: introduce snapshot journal version

* core: update the disk layer in an atomic way

* core: persist the disk layer generator periodically

* core/state/snapshot: improve logging

* core/state/snapshot: forcibly ensure the legacy snapshot is matched

* core/state/snapshot: add debug logs

* core, tests: fix tests and special recovery case

* core: polish

* core: add more blockchain tests for snapshot recovery

* core/state: fix comment

* core: add recovery flag for snapshot

* core: add restart after start-after-crash tests

* core/rawdb: fix imports

* core: fix tests

* core: remove log

* core/state/snapshot: fix snapshot

* core: avoid callbacks in SetHead

* core: fix setHead cornercase where the threshold root has state

* core: small docs for the test cases

Co-authored-by: Péter Szilágyi <peterke@gmail.com>
 2020-10-29 21:01:58 +02:00
gary rong
43c278cdf9
core/state: disable snapshot iteration if it's not fully constructed (#21682) 
* core/state/snapshot: add diskRoot function

* core/state/snapshot: disable iteration if the snapshot is generating

* core/state/snapshot: simplify the function

* core/state: panic for undefined layer
 2020-10-28 14:27:37 +02:00
Péter Szilágyi
a15d71a255
core/state/snapshot: stop generator if it hits missing trie nodes (#21649) 
* core/state/snapshot: exit Geth if generator hits missing trie nodes

* core/state/snapshot: error instead of hard die on generator fault

* core/state/snapshot: don't enable logging on the tests
 2020-10-05 11:52:36 +03:00
Péter Szilágyi
3ee91b9f2e
core/state/snapshot: reduce disk layer depth during generation 2020-08-24 13:22:36 +03:00
Martin Michlmayr
4f2784b38f
all: fix typos in comments (#21118) 2020-05-25 10:21:28 +02:00