* core: use finalized block as the chain freeze indicator (#28683)
* core/rawdb: use max(finality, head-90k) as chain freezing threshold
* core: impl multi database for block data
* core: fix db inspect total size bug
* core: add tips for user who use multi-database
* core: adapt some cmd for multi-database
* core: adapter blockBlobSidecars
* core: fix freezer readHeader bug
---------
Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
* ci: temp enable blobtx branch ci run;
* Switch ON blobpool & ensure Cancun hardfork can occur (#2223)
* feat: support blob storage & miscs; (#2229)
* chainconfig: use cancun fork for BSC;
* feat: fill WithdrawalsHash when BSC enable cancun fork;
* rawdb: support to CRUD blobs;
* freezer: support to freeze block blobs;
* blockchain: add blob cache & blob query helper;
* freezer: refactor addition table logic, add uts;
* blobexpiry: add more extra expiry time, and logs;
* parlia: implement IsDataAvailable function;
* blob: refactor blob transfer logic;
* blob: support config blob extra reserve;
* blockchian: support to import block with blob & blobGasFee; (#2260)
* blob: implement min&max gas price logic;
* blockchian: support import side chain;
* blobpool: reject the banned address;
* blockchain: add chasing head for DA check;
* params: update blob related config;
* blockchain: opt data available checking performance;
* params: modify blob related params;
* gasprice: support BEP-336 blob gas price calculate;
* blobTx: mining + brodcasting (#2253)
* blobtx mining pass (#2282)
* Sidecar fetching changes for 4844 (#2283)
* ci: temp enable blobtx branch ci run;
* Switch ON blobpool & ensure Cancun hardfork can occur (#2223)
* feat: support blob storage & miscs; (#2229)
* chainconfig: use cancun fork for BSC;
feat: fill WithdrawalsHash when BSC enable cancun fork;
* rawdb: support to CRUD blobs;
* freezer: support to freeze block blobs;
* blockchain: add blob cache & blob query helper;
* freezer: refactor addition table logic, add uts;
* blobexpiry: add more extra expiry time, and logs;
* parlia: implement IsDataAvailable function;
* blob: refactor blob transfer logic;
* blob: support config blob extra reserve;
* blockchian: support to import block with blob & blobGasFee; (#2260)
* blob: implement min&max gas price logic;
* blockchian: support import side chain;
* blobpool: reject the banned address;
* blockchain: add chasing head for DA check;
* params: update blob related config;
* blockchain: opt data available checking performance;
* params: modify blob related params;
* gasprice: support BEP-336 blob gas price calculate;
* fix failed check for WithdrawalsHash (#2276)
* eth: include sidecars in fitering of body
* core: refactor sidecars name
* eth: sidecars type refactor
* core: remove extra from bad merge
* eth: fix handlenewblock test after merge
* Implement eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2286)
* execution: add blob gas fee reward to system;
* syncing: support blob syncing & DA checking;
* naming: rename blobs to sidecars;
* fix the semantics of WithXXX (#2293)
* config: reduce sidecar cache to 1024 and rename (#2297)
* fix: Withdrawals turn into empty from nil when BlockBody has Sidecars (#2301)
* internal/api_test: add test case for eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2300)
* consensus/misc: rollback CalcBlobFee (#2306)
* flags: add new flags to override blobs' params;
* freezer: fix blob ancient save error;
* blobsidecar: add new sidecar struct with metadata; (#2315)
* core/rawdb: optimize write block with sidecars (#2318)
* core: more check for validity of sidecars
* mev: add TxIndex for mev bid (#2325)
* remove useless Config() (#2326)
* fix WithSidecars (#2327)
* fix: fix mined block sidecar issue; (#2328)
* fix WithSidecars (#2329)
---------
Co-authored-by: GalaIO <GalaIO@users.noreply.github.com>
Co-authored-by: buddho <galaxystroller@gmail.com>
Co-authored-by: Satyajit Das <emailtovamos@gmail.com>
Co-authored-by: Eric <45141191+zlacfzy@users.noreply.github.com>
Co-authored-by: zzzckck <152148891+zzzckck@users.noreply.github.com>
* all: implement era format, add history importer/export
* internal/era/e2store: refactor e2store to provide ReadAt interface
* internal/era/e2store: export HeaderSize
* internal/era: refactor era to use ReadAt interface
* internal/era: elevate anonymous func to named
* cmd/utils: don't store entire era file in-memory during import / export
* internal/era: better abstraction between era and e2store
* cmd/era: properly close era files
* cmd/era: don't let defers stack
* cmd/geth: add description for import-history
* cmd/utils: better bytes buffer
* internal/era: error if accumulator has more records than max allowed
* internal/era: better doc comment
* internal/era/e2store: rm superfluous reader, rm superfluous testcases, add fuzzer
* internal/era: avoid some repetition
* internal/era: simplify clauses
* internal/era: unexport things
* internal/era,cmd/utils,cmd/era: change to iterator interface for reading era entries
* cmd/utils: better defer handling in history test
* internal/era,cmd: add number method to era iterator to get the current block number
* internal/era/e2store: avoid double allocation during write
* internal/era,cmd/utils: fix lint issues
* internal/era: add ReaderAt func so entry value can be read lazily
Co-authored-by: lightclient <lightclient@protonmail.com>
Co-authored-by: Martin Holst Swende <martin@swende.se>
* internal/era: improve iterator interface
* internal/era: fix rlp decode of header and correctly read total difficulty
* cmd/era: fix rebase errors
* cmd/era: clearer comments
* cmd,internal: fix comment typos
---------
Co-authored-by: Martin Holst Swende <martin@swende.se>
This change simplifies the logic for indexing transactions and enhances the UX when transaction is not found by returning more information to users.
Transaction indexing is now considered as a part of the initial sync, and `eth.syncing` will thus be `true` if transaction indexing is not yet finished. API consumers can use the syncing status to determine if the node is ready to serve users.
* core/state, light, les: make signature of ContractCode hash-independent
* push current state for feedback
* les: fix unit test
* core, les, light: fix les unittests
* core/state, trie, les, light: fix state iterator
* core, les: address comments
* les: fix lint
---------
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This PR introduces a node scheme abstraction. The interface is only implemented by `hashScheme` at the moment, but will be extended by `pathScheme` very soon.
Apart from that, a few changes are also included which is worth mentioning:
- port the changes in the stacktrie, tracking the path prefix of nodes during commit
- use ethdb.Database for constructing trie.Database. This is not necessary right now, but it is required for path-based used to open reverse diff freezer
It seems there is no fully typed library implementation of an LRU cache.
So I wrote one. Method names are the same as github.com/hashicorp/golang-lru,
and the new type can be used as a drop-in replacement.
Two reasons to do this:
- It's much easier to understand what a cache is for when the types are right there.
- Performance: the new implementation is slightly faster and performs zero memory
allocations in Add when the cache is at capacity. Overall, memory usage of the cache
is much reduced because keys are values are no longer wrapped in interface.
This PR reduces the amount of work we do when answering header queries, e.g. when a peer
is syncing from us.
For some items, e.g block bodies, when we read the rlp-data from database, we plug it
directly into the response package. We didn't do that for headers, but instead read
headers-rlp, decode to types.Header, and re-encode to rlp. This PR changes that to keep it
in RLP-form as much as possible. When a node is syncing from us, it typically requests 192
contiguous headers. On master it has the following effect:
- For headers not in ancient: 2 db lookups. One for translating hash->number (even though
the request is by number), and another for reading by hash (this latter one is sometimes
cached).
- For headers in ancient: 1 file lookup/syscall for translating hash->number (even though
the request is by number), and another for reading the header itself. After this, it
also performes a hashing of the header, to ensure that the hash is what it expected. In
this PR, I instead move the logic for "give me a sequence of blocks" into the lower
layers, where the database can determine how and what to read from leveldb and/or
ancients.
There are basically four types of requests; three of them are improved this way. The
fourth, by hash going backwards, is more tricky to optimize. However, since we know that
the gap is 0, we can look up by the parentHash, and stlil shave off all the number->hash
lookups.
The gapped collection can be optimized similarly, as a follow-up, at least in three out of
four cases.
Co-authored-by: Felix Lange <fjl@twurst.com>
