Commit Graph

14 Commits

Author SHA1 Message Date
zjubfd
1ded097733
[R4R]implement diff sync (#376)
* implement block process part of light sync

* add difflayer protocol

* handle difflayer and refine light processor

* add testcase for diff protocol

* make it faster

* allow validator to light sync

* change into diff sync

* ligth sync: download difflayer (#2)

* ligth sync: download difflayer

Signed-off-by: kyrie-yl <lei.y@binance.com>

* download diff layer: fix according to the comments

Signed-off-by: kyrie-yl <lei.y@binance.com>

* download diff layer: update

Signed-off-by: kyrie-yl <lei.y@binance.com>

* download diff layer: fix accroding comments

Signed-off-by: kyrie-yl <lei.y@binance.com>

Co-authored-by: kyrie-yl <lei.y@binance.com>

* update light sync to diff sync

* raise the max diff limit

* add switcher of snap protocol

* fix test case

* make commit concurrently

* remove peer for diff cache when peer closed

* consensus tuning

* add test code

* remove extra message

* fix testcase and lint

make diff block configable

wait code write

fix testcase

resolve comments

resolve comment

* resolve comments

* resolve comments

* resolve comment

* fix mistake

Co-authored-by: kyrie-yl <83150977+kyrie-yl@users.noreply.github.com>
Co-authored-by: kyrie-yl <lei.y@binance.com>
2021-09-28 16:03:38 +08:00
Martin Holst Swende
9b99e3dfe0
core/rawdb: fix datarace in freezer (#22728)
The Append / truncate operations were racy. When a datafile reaches 2Gb, a new file is needed. For this operation, we require a writelock, which is not needed in the 99.99% of all cases where the data does fit in the current head-file.

This transition from readlock to writelock was incorrect, and as the readlock was released, a truncate operation could slip in between, and truncate the data. This would have been fine, however, the Append operation continued writing as if no truncation had occurred, e.g writing item 5 where item 0 should reside.

This PR changes the behaviour, so that if when we run into the situation that a new file is needed, it aborts, and retries, this time with a writelock.

The outcome of the situation described above, running on this PR, would instead be that the Append operation exits with a failure.
2021-04-26 18:19:07 +02:00
Martin Holst Swende
271e5b7fc9
cmd/geth: add db-command to inspect freezer index (#22633)
This PR makes it easier to inspect the freezer index, which could be useful to investigate things like #22111
2021-04-13 15:45:30 +02:00
zhangsoledad
bcf19bc4be
core/rawdb: swap tailId and itemOffset for deleted items in freezer (#21220)
* fix(freezer): tailId filenum offset were misplaced

* core/rawdb: assume first item in freezer always start from zero
2020-06-17 09:41:07 +02:00
meowsbits
20a092fb9f
core/rawdb: fix freezer table test error check
Fixes: Condition is always 'false' because 'err' is always 'nil'
2020-03-18 06:55:30 -05:00
Felix Lange
689486449d build: use golangci-lint (#20295)
* build: use golangci-lint

This changes build/ci.go to download and run golangci-lint instead
of gometalinter.

* core/state: fix unnecessary conversion

* p2p/simulations: fix lock copying (found by go vet)

* signer/core: fix unnecessary conversions

* crypto/ecies: remove unused function cmpPublic

* core/rawdb: remove unused function print

* core/state: remove unused function xTestFuzzCutter

* core/vm: disable TestWriteExpectedValues in a different way

* core/forkid: remove unused function checksum

* les: remove unused type proofsData

* cmd/utils: remove unused functions prefixedNames, prefixFor

* crypto/bn256: run goimports

* p2p/nat: fix goimports lint issue

* cmd/clef: avoid using unkeyed struct fields

* les: cancel context in testRequest

* rlp: delete unreachable code

* core: gofmt

* internal/build: simplify DownloadFile for Go 1.11 compatibility

* build: remove go test --short flag

* .travis.yml: disable build cache

* whisper/whisperv6: fix ineffectual assignment in TestWhisperIdentityManagement

* .golangci.yml: enable goconst and ineffassign linters

* build: print message when there are no lint issues

* internal/build: refactor download a bit
2019-11-18 10:49:17 +02:00
Péter Szilágyi
72d5a27a39
core, metrics, p2p: switch some invalid counters to gauges 2019-09-10 14:39:07 +03:00
Péter Szilágyi
1a83114c74
all: update author list and licenses 2019-07-22 12:17:27 +03:00
Christian Muehlhaeuser
57fc1d21e1 cmd/geth, core/rawdb: add missing error checks (#19871)
* Added missing error checks

Add error handling where we assign err a value, but don't check for it being nil.

* core/rawdb: tiny style nit
2019-07-22 10:51:13 +03:00
大彬
4ac04ae0fe all: replace fmt.Print* calls with t.Log* in tests (#19670) 2019-07-17 13:20:24 +02:00
Péter Szilágyi
b02958b9c5
core, ethdb, metrics, p2p: expose various counter metrics for grafana 2019-06-11 09:49:13 +03:00
Péter Szilágyi
536b3b416c
cosensus, core, eth, params, trie: fixes + clique history cap 2019-05-16 10:39:35 +03:00
Martin Holst Swende
331de17e4d
core/rawdb: support starting offset for future deletion 2019-05-16 10:39:33 +03:00
Martin Holst Swende
b69bdc2a4f
freezer: implement split files for data
* freezer: implement split files for data

* freezer: add tests

* freezer: close old head-file when opening next

* freezer: fix truncation

* freezer: more testing around close/open

* rawdb/freezer: address review concerns

* freezer: fix minor review concerns

* freezer: fix remaining concerns + testcases around truncation

* freezer: docs

* freezer: implement multithreading

* core/rawdb: fix freezer nitpicks + change offsets to uint32

* freezer: preopen files, simplify lock constructs

* freezer: delete files during truncation
2019-05-16 10:39:30 +03:00