bsc/ethdb/database.go

255 lines
8.3 KiB
Go
Raw Permalink Normal View History

2019-07-22 12:17:27 +03:00
// Copyright 2014 The go-ethereum Authors
// This file is part of the go-ethereum library.
2015-07-07 02:54:22 +02:00
//
// The go-ethereum library is free software: you can redistribute it and/or modify
2015-07-07 02:54:22 +02:00
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// The go-ethereum library is distributed in the hope that it will be useful,
2015-07-07 02:54:22 +02:00
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2015-07-07 02:54:22 +02:00
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
2015-07-07 02:54:22 +02:00
// Package ethdb defines the interfaces for an Ethereum data store.
2014-02-14 23:56:09 +01:00
package ethdb
BlobTx: implement EIP-4844 on BSC (#2279) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; * feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * blobTx: mining + brodcasting (#2253) * blobtx mining pass (#2282) * Sidecar fetching changes for 4844 (#2283) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * fix failed check for WithdrawalsHash (#2276) * eth: include sidecars in fitering of body * core: refactor sidecars name * eth: sidecars type refactor * core: remove extra from bad merge * eth: fix handlenewblock test after merge * Implement eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2286) * execution: add blob gas fee reward to system; * syncing: support blob syncing & DA checking; * naming: rename blobs to sidecars; * fix the semantics of WithXXX (#2293) * config: reduce sidecar cache to 1024 and rename (#2297) * fix: Withdrawals turn into empty from nil when BlockBody has Sidecars (#2301) * internal/api_test: add test case for eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2300) * consensus/misc: rollback CalcBlobFee (#2306) * flags: add new flags to override blobs' params; * freezer: fix blob ancient save error; * blobsidecar: add new sidecar struct with metadata; (#2315) * core/rawdb: optimize write block with sidecars (#2318) * core: more check for validity of sidecars * mev: add TxIndex for mev bid (#2325) * remove useless Config() (#2326) * fix WithSidecars (#2327) * fix: fix mined block sidecar issue; (#2328) * fix WithSidecars (#2329) --------- Co-authored-by: GalaIO <GalaIO@users.noreply.github.com> Co-authored-by: buddho <galaxystroller@gmail.com> Co-authored-by: Satyajit Das <emailtovamos@gmail.com> Co-authored-by: Eric <45141191+zlacfzy@users.noreply.github.com> Co-authored-by: zzzckck <152148891+zzzckck@users.noreply.github.com>
2024-03-22 22:37:47 +08:00
import (
"io"
"github.com/ethereum/go-ethereum/params"
)
// KeyValueReader wraps the Has and Get method of a backing data store.
type KeyValueReader interface {
// Has retrieves if a key is present in the key-value data store.
Has(key []byte) (bool, error)
// Get retrieves the given key if it's present in the key-value data store.
Get(key []byte) ([]byte, error)
}
// KeyValueWriter wraps the Put method of a backing data store.
type KeyValueWriter interface {
// Put inserts the given value into the key-value data store.
Put(key []byte, value []byte) error
// Delete removes the key from the key-value data store.
Delete(key []byte) error
}
// KeyValueStater wraps the Stat method of a backing data store.
type KeyValueStater interface {
// Stat returns a particular internal stat of the database.
Stat(property string) (string, error)
}
// Compacter wraps the Compact method of a backing data store.
type Compacter interface {
// Compact flattens the underlying data store for the given key range. In essence,
// deleted and overwritten versions are discarded, and the data is rearranged to
// reduce the cost of operations needed to access them.
//
// A nil start is treated as a key before all keys in the data store; a nil limit
// is treated as a key after all keys in the data store. If both is nil then it
// will compact entire data store.
Compact(start []byte, limit []byte) error
}
// KeyValueStore contains all the methods required to allow handling different
// key-value data stores backing the high level database.
type KeyValueStore interface {
KeyValueReader
KeyValueWriter
KeyValueStater
Batcher
Iteratee
Compacter
Snapshotter
io.Closer
}
// AncientReaderOp contains the methods required to read from immutable ancient data.
type AncientReaderOp interface {
// HasAncient returns an indicator whether the specified data exists in the
// ancient store.
HasAncient(kind string, number uint64) (bool, error)
// Ancient retrieves an ancient binary blob from the append-only immutable files.
Ancient(kind string, number uint64) ([]byte, error)
// AncientRange retrieves multiple items in sequence, starting from the index 'start'.
// It will return
// - at most 'count' items,
// - if maxBytes is specified: at least 1 item (even if exceeding the maxByteSize),
// but will otherwise return as many items as fit into maxByteSize.
// - if maxBytes is not specified, 'count' items will be returned if they are present
AncientRange(kind string, start, count, maxBytes uint64) ([][]byte, error)
// Ancients returns the ancient item numbers in the ancient store.
Ancients() (uint64, error)
// Tail returns the number of first stored item in the freezer.
// This number can also be interpreted as the total deleted item numbers.
Tail() (uint64, error)
// AncientSize returns the ancient size of the specified category.
AncientSize(kind string) (uint64, error)
2022-07-05 11:14:21 +08:00
// ItemAmountInAncient returns the actual length of current ancientDB.
ItemAmountInAncient() (uint64, error)
// AncientOffSet returns the offset of current ancientDB.
AncientOffSet() uint64
}
// AncientReader is the extended ancient reader interface including 'batched' or 'atomic' reading.
type AncientReader interface {
AncientReaderOp
// ReadAncients runs the given read operation while ensuring that no writes take place
// on the underlying freezer.
ReadAncients(fn func(AncientReaderOp) error) (err error)
}
// AncientWriter contains the methods required to write to immutable ancient data.
type AncientWriter interface {
// ModifyAncients runs a write operation on the ancient store.
// If the function returns an error, any changes to the underlying store are reverted.
// The integer return value is the total size of the written data.
ModifyAncients(func(AncientWriteOp) error) (int64, error)
// TruncateHead discards all but the first n ancient data from the ancient store.
// After the truncation, the latest item can be accessed it item_n-1(start from 0).
TruncateHead(n uint64) (uint64, error)
// TruncateTail discards the first n ancient data from the ancient store. The already
// deleted items are ignored. After the truncation, the earliest item can be accessed
// is item_n(start from 0). The deleted items may not be removed from the ancient store
// immediately, but only when the accumulated deleted data reach the threshold then
// will be removed all together.
TruncateTail(n uint64) (uint64, error)
// Sync flushes all in-memory ancient store data to disk.
Sync() error
// MigrateTable processes and migrates entries of a given table to a new format.
// The second argument is a function that takes a raw entry and returns it
// in the newest format.
MigrateTable(string, func([]byte) ([]byte, error)) error
BlobTx: implement EIP-4844 on BSC (#2279) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; * feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * blobTx: mining + brodcasting (#2253) * blobtx mining pass (#2282) * Sidecar fetching changes for 4844 (#2283) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * fix failed check for WithdrawalsHash (#2276) * eth: include sidecars in fitering of body * core: refactor sidecars name * eth: sidecars type refactor * core: remove extra from bad merge * eth: fix handlenewblock test after merge * Implement eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2286) * execution: add blob gas fee reward to system; * syncing: support blob syncing & DA checking; * naming: rename blobs to sidecars; * fix the semantics of WithXXX (#2293) * config: reduce sidecar cache to 1024 and rename (#2297) * fix: Withdrawals turn into empty from nil when BlockBody has Sidecars (#2301) * internal/api_test: add test case for eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2300) * consensus/misc: rollback CalcBlobFee (#2306) * flags: add new flags to override blobs' params; * freezer: fix blob ancient save error; * blobsidecar: add new sidecar struct with metadata; (#2315) * core/rawdb: optimize write block with sidecars (#2318) * core: more check for validity of sidecars * mev: add TxIndex for mev bid (#2325) * remove useless Config() (#2326) * fix WithSidecars (#2327) * fix: fix mined block sidecar issue; (#2328) * fix WithSidecars (#2329) --------- Co-authored-by: GalaIO <GalaIO@users.noreply.github.com> Co-authored-by: buddho <galaxystroller@gmail.com> Co-authored-by: Satyajit Das <emailtovamos@gmail.com> Co-authored-by: Eric <45141191+zlacfzy@users.noreply.github.com> Co-authored-by: zzzckck <152148891+zzzckck@users.noreply.github.com>
2024-03-22 22:37:47 +08:00
// TruncateTableTail will truncate certain table to new tail
TruncateTableTail(kind string, tail uint64) (uint64, error)
// ResetTable will reset certain table with new start point
ResetTable(kind string, startAt uint64, onlyEmpty bool) error
BlobTx: implement EIP-4844 on BSC (#2279) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; * feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * blobTx: mining + brodcasting (#2253) * blobtx mining pass (#2282) * Sidecar fetching changes for 4844 (#2283) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * fix failed check for WithdrawalsHash (#2276) * eth: include sidecars in fitering of body * core: refactor sidecars name * eth: sidecars type refactor * core: remove extra from bad merge * eth: fix handlenewblock test after merge * Implement eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2286) * execution: add blob gas fee reward to system; * syncing: support blob syncing & DA checking; * naming: rename blobs to sidecars; * fix the semantics of WithXXX (#2293) * config: reduce sidecar cache to 1024 and rename (#2297) * fix: Withdrawals turn into empty from nil when BlockBody has Sidecars (#2301) * internal/api_test: add test case for eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2300) * consensus/misc: rollback CalcBlobFee (#2306) * flags: add new flags to override blobs' params; * freezer: fix blob ancient save error; * blobsidecar: add new sidecar struct with metadata; (#2315) * core/rawdb: optimize write block with sidecars (#2318) * core: more check for validity of sidecars * mev: add TxIndex for mev bid (#2325) * remove useless Config() (#2326) * fix WithSidecars (#2327) * fix: fix mined block sidecar issue; (#2328) * fix WithSidecars (#2329) --------- Co-authored-by: GalaIO <GalaIO@users.noreply.github.com> Co-authored-by: buddho <galaxystroller@gmail.com> Co-authored-by: Satyajit Das <emailtovamos@gmail.com> Co-authored-by: Eric <45141191+zlacfzy@users.noreply.github.com> Co-authored-by: zzzckck <152148891+zzzckck@users.noreply.github.com>
2024-03-22 22:37:47 +08:00
}
type FreezerEnv struct {
ChainCfg *params.ChainConfig
BlobExtraReserve uint64
}
// AncientFreezer defines the help functions for freezing ancient data
type AncientFreezer interface {
// SetupFreezerEnv provides params.ChainConfig for checking hark forks, like isCancun.
SetupFreezerEnv(env *FreezerEnv) error
}
// AncientWriteOp is given to the function argument of ModifyAncients.
type AncientWriteOp interface {
// Append adds an RLP-encoded item.
Append(kind string, number uint64, item interface{}) error
// AppendRaw adds an item without RLP-encoding it.
AppendRaw(kind string, number uint64, item []byte) error
}
// AncientStater wraps the Stat method of a backing data store.
type AncientStater interface {
// AncientDatadir returns the path of root ancient directory. Empty string
// will be returned if ancient store is not enabled at all. The returned
// path can be used to construct the path of other freezers.
AncientDatadir() (string, error)
}
type StateStoreReader interface {
StateStoreReader() Reader
}
type BlockStore interface {
BlockStore() Database
SetBlockStore(block Database)
HasSeparateBlockStore() bool
}
type BlockStoreReader interface {
BlockStoreReader() Reader
}
type BlockStoreWriter interface {
BlockStoreWriter() Writer
}
// Reader contains the methods required to read data from both key-value as well as
// immutable ancient data.
type Reader interface {
KeyValueReader
AncientReader
StateStoreReader
BlockStoreReader
}
// Writer contains the methods required to write data to both key-value as well as
// immutable ancient data.
type Writer interface {
KeyValueWriter
AncientWriter
BlockStoreWriter
}
// Stater contains the methods required to retrieve states from both key-value as well as
// immutable ancient data.
type Stater interface {
KeyValueStater
AncientStater
}
2019-03-14 14:59:47 +08:00
// AncientStore contains all the methods required to allow handling different
// ancient data stores backing immutable chain data store.
type AncientStore interface {
AncientReader
AncientWriter
2019-03-14 14:59:47 +08:00
io.Closer
}
2022-07-05 11:14:21 +08:00
type DiffStore interface {
DiffStore() KeyValueStore
SetDiffStore(diff KeyValueStore)
}
type StateStore interface {
StateStore() Database
SetStateStore(state Database)
}
// Database contains all the methods required by the high level database to not
// only access the key-value data store but also the chain freezer.
type Database interface {
Reader
Writer
2022-07-05 11:14:21 +08:00
DiffStore
StateStore
BlockStore
Batcher
Iteratee
Stater
Compacter
Snapshotter
BlobTx: implement EIP-4844 on BSC (#2279) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; * feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * blobTx: mining + brodcasting (#2253) * blobtx mining pass (#2282) * Sidecar fetching changes for 4844 (#2283) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * fix failed check for WithdrawalsHash (#2276) * eth: include sidecars in fitering of body * core: refactor sidecars name * eth: sidecars type refactor * core: remove extra from bad merge * eth: fix handlenewblock test after merge * Implement eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2286) * execution: add blob gas fee reward to system; * syncing: support blob syncing & DA checking; * naming: rename blobs to sidecars; * fix the semantics of WithXXX (#2293) * config: reduce sidecar cache to 1024 and rename (#2297) * fix: Withdrawals turn into empty from nil when BlockBody has Sidecars (#2301) * internal/api_test: add test case for eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2300) * consensus/misc: rollback CalcBlobFee (#2306) * flags: add new flags to override blobs' params; * freezer: fix blob ancient save error; * blobsidecar: add new sidecar struct with metadata; (#2315) * core/rawdb: optimize write block with sidecars (#2318) * core: more check for validity of sidecars * mev: add TxIndex for mev bid (#2325) * remove useless Config() (#2326) * fix WithSidecars (#2327) * fix: fix mined block sidecar issue; (#2328) * fix WithSidecars (#2329) --------- Co-authored-by: GalaIO <GalaIO@users.noreply.github.com> Co-authored-by: buddho <galaxystroller@gmail.com> Co-authored-by: Satyajit Das <emailtovamos@gmail.com> Co-authored-by: Eric <45141191+zlacfzy@users.noreply.github.com> Co-authored-by: zzzckck <152148891+zzzckck@users.noreply.github.com>
2024-03-22 22:37:47 +08:00
AncientFreezer
io.Closer
}