Commit Graph

66 Commits

Author SHA1 Message Date
zjubfd
9603407407
fix useful difflayer item in cache been prune issue (#527)
add logs

fix useful difflayer item in cache been prune issue

fix too many open files
2021-11-09 19:39:58 +08:00
zjubfd
1ded097733
[R4R]implement diff sync (#376)
* implement block process part of light sync

* add difflayer protocol

* handle difflayer and refine light processor

* add testcase for diff protocol

* make it faster

* allow validator to light sync

* change into diff sync

* ligth sync: download difflayer (#2)

* ligth sync: download difflayer

Signed-off-by: kyrie-yl <lei.y@binance.com>

* download diff layer: fix according to the comments

Signed-off-by: kyrie-yl <lei.y@binance.com>

* download diff layer: update

Signed-off-by: kyrie-yl <lei.y@binance.com>

* download diff layer: fix accroding comments

Signed-off-by: kyrie-yl <lei.y@binance.com>

Co-authored-by: kyrie-yl <lei.y@binance.com>

* update light sync to diff sync

* raise the max diff limit

* add switcher of snap protocol

* fix test case

* make commit concurrently

* remove peer for diff cache when peer closed

* consensus tuning

* add test code

* remove extra message

* fix testcase and lint

make diff block configable

wait code write

fix testcase

resolve comments

resolve comment

* resolve comments

* resolve comments

* resolve comment

* fix mistake

Co-authored-by: kyrie-yl <83150977+kyrie-yl@users.noreply.github.com>
Co-authored-by: kyrie-yl <lei.y@binance.com>
2021-09-28 16:03:38 +08:00
yutianwu
061cd5030c upgrade to 1.10.2 2021-04-16 12:45:26 +08:00
gary rong
0c70b83e00
all: add read-only option to database (#22407)
* all: add read-only option to database

* all: fixes tests

* cmd/geth: migrate flags

* cmd/geth: fix the compact

* cmd/geth: fix the format

* cmd/geth: fix log

* cmd: add chain-readonly

* core: add readonly notion to freezer

* core/rawdb: add log

* core/rawdb: fix freezer close

* cmd: fix

* cmd, core: construct db

* core: update tests
2021-03-22 19:06:30 +01:00
rene
4eae0c6b6f
cmd/geth, node: allow configuring JSON-RPC on custom path prefix (#22184)
This change allows users to set a custom path prefix on which to mount the http-rpc
or ws-rpc handlers via the new flags --http.rpcprefix and --ws.rpcprefix.

Fixes #21826

Co-authored-by: Felix Lange <fjl@twurst.com>
2021-02-02 10:05:46 +01:00
rene
5c2a7ce2cc
node: rename startNetworking to openEndpoints (#22105) 2021-01-04 12:39:25 +01:00
rene
c0c01612e9
node: refactor package node (#21105)
This PR significantly changes the APIs for instantiating Ethereum nodes in
a Go program. The new APIs are not backwards-compatible, but we feel that
this is made up for by the much simpler way of registering services on
node.Node. You can find more information and rationale in the design
document: https://gist.github.com/renaynay/5bec2de19fde66f4d04c535fd24f0775.

There is also a new feature in Node's Go API: it is now possible to
register arbitrary handlers on the user-facing HTTP server. In geth, this
facility is used to enable GraphQL.

There is a single minor change relevant for geth users in this PR: The
GraphQL API is no longer available separately from the JSON-RPC HTTP
server. If you want GraphQL, you need to enable it using the
./geth --http --graphql flag combination.

The --graphql.port and --graphql.addr flags are no longer available.
2020-08-03 19:40:46 +02:00
fudongbai
2ac4ae8395 add consensus engine Parlia implement 2020-05-20 12:19:24 +08:00
Steven E. Harris
40283d0522
node: shut down all node-related HTTP servers gracefully (#20956)
Rather than just closing the underlying network listener to stop our
HTTP servers, use the graceful shutdown procedure, waiting for any
in-process requests to finish.
2020-04-27 11:16:00 +02:00
rene
07d909ff32
node: allow websocket and HTTP on the same port (#20810)
This change makes it possible to run geth with JSON-RPC over HTTP and
WebSocket on the same TCP port. The default port for WebSocket
is still 8546. 

    geth --rpc --rpcport 8545 --ws --wsport 8545

This also removes a lot of deprecated API surface from package rpc.
The rpc package is now purely about serving JSON-RPC and no longer
provides a way to start an HTTP server.
2020-04-08 13:33:12 +02:00
rene
d7851e6359
graphql, node, rpc: fix typos in comments (#20824) 2020-03-27 13:52:53 +01:00
Martin Holst Swende
0734c4b820
node, cmd/clef: report actual port used for http rpc (#20789) 2020-03-23 16:26:56 +01:00
Felix Lange
38d1b0cba2
cmd/geth: enable DNS discovery by default (#20660)
* node: expose config in service context

* eth: integrate p2p/dnsdisc

* cmd/geth: add some DNS flags

* eth: remove DNS URLs

* cmd/utils: configure DNS names for testnets

* params: update DNS URLs

* cmd/geth: configure mainnet DNS

* cmd/utils: rename DNS flag and fix flag processing

* cmd/utils: remove debug print

* node: fix test
2020-02-13 15:38:30 +02:00
gary rong
f7cdea2bdc all: on-chain oracle checkpoint syncing (#19543)
* all: implement simple checkpoint syncing

cmd, les, node: remove callback mechanism

cmd, node: remove callback definition

les: simplify the registrar

les: expose checkpoint rpc services in the light client

les, light: don't store untrusted receipt

cmd, contracts, les: discard stale checkpoint

cmd, contracts/registrar: loose restriction of registeration

cmd, contracts: add replay-protection

all: off-chain multi-signature contract

params: deploy checkpoint contract for rinkeby

cmd/registrar: add raw signing mode for registrar

cmd/registrar, contracts/registrar, les: fixed messages

* cmd/registrar, contracts/registrar: fix lints

* accounts/abi/bind, les: address comments

* cmd, contracts, les, light, params: minor checkpoint sync cleanups

* cmd, eth, les, light: move checkpoint config to config file

* cmd, eth, les, params: address comments

* eth, les, params: address comments

* cmd: polish up the checkpoint admin CLI

* cmd, contracts, params: deploy new version contract

* cmd/checkpoint-admin: add another flag for clef mode signing

* cmd, contracts, les: rename and regen checkpoint oracle with abigen
2019-06-28 10:34:02 +03:00
Péter Szilágyi
006c21efc7
cmd, core, eth, les, node: chain freezer on top of db rework 2019-05-16 10:39:29 +03:00
Péter Szilágyi
29bc982d75
cmd/geth, internal, node, vendor: nuke geth monitor 2019-04-05 12:13:56 +03:00
Steve Ruckdashel
a8dd1f93c6 node: switching prometheus flock location to tsdb (#19376)
* node: switching prometheus flock location to tsdb

* rookie mistake
2019-04-04 16:59:18 +03:00
gary rong
d5cae48bae accounts, cmd, internal: disable unlock account on open HTTP (#17037)
* cmd, accounts, internal, node, rpc, signer: insecure unlock protect

* all: strict unlock API by rpc

* cmd/geth: check before printing warning log

* accounts, cmd/geth, internal: tiny polishes
2019-04-04 14:03:10 +03:00
Péter Szilágyi
054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Matthew Halpern
7b3e3d549e node: prefer nil slices over zero-length slices (#19083) 2019-02-18 13:31:22 +02:00
Janoš Guljaš
26aea73673 cmd, node, p2p/simulations: fix node account manager leak (#19004)
* node: close AccountsManager in new Close method

* p2p/simulations, p2p/simulations/adapters: handle node close on shutdown

* node: move node ephemeralKeystore cleanup to stop method

* node: call Stop in Node.Close method

* cmd/geth: close node.Node created with makeFullNode in cli commands

* node: close Node instances in tests

* cmd/geth, node: minor code style fixes

* cmd, console, miner, mobile: proper node Close() termination
2019-02-07 12:40:36 +02:00
Péter Szilágyi
af8daf91a6
node, rpc: log cleanups in ipc listener function (#18124)
node,rpc: remove unused log in ipc listener function
2018-12-10 13:15:03 +02:00
Franklin
21dd59bd04 . 2018-11-20 22:16:40 +08:00
Martin Holst Swende
493903eede
core: better side-chain importing 2018-11-20 12:28:43 +02:00
Felix Lange
dcae0d348b
p2p/simulations: fix a deadlock and clean up adapters (#17891)
This fixes a rare deadlock with the inproc adapter:

- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
  waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
  which waits for Network.lock.

The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.

Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.

In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.

A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
2018-10-11 20:32:14 +02:00
Ryan Schneider
5d7e18539e rpc: make HTTP RPC timeouts configurable, raise defaults (#17240)
* rpc: Make HTTP server timeout values configurable

* rpc: Remove flags for setting HTTP Timeouts, configuring via .toml is sufficient.

* rpc: Replace separate constants with a single default struct.

* rpc: Update HTTP Server Read and Write Timeouts to 30s.

* rpc: Remove redundant NewDefaultHTTPTimeouts function.

* rpc: document HTTPTimeouts.

* rpc: sanitize timeout values for library use
2018-07-31 12:16:14 +03:00
Kurkó Mihály
a9835c1816 cmd, dashboard, log: log collection and exploration (#17097)
* cmd, dashboard, internal, log, node: logging feature

* cmd, dashboard, internal, log: requested changes

* dashboard, vendor: gofmt, govendor, use vendored file watcher

* dashboard, log: gofmt -s -w, goimports

* dashboard, log: gosimple
2018-07-11 10:59:04 +03:00
gary rong
7beccb29be all: get rid of error when creating memory database (#16716)
* all: get rid of error when create mdb

* core: clean up variables definition

* all: inline mdb definition
2018-05-09 15:24:25 +03:00
Felix Lange
52b046c9b6
rpc: clean up IPC handler (#16524)
This avoids logging accept errors on shutdown and removes
a bit of duplication. It also fixes some goimports lint warnings.
2018-04-18 12:27:20 +02:00
Martin Holst Swende
ec3db0f56c cmd/clef, signer: initial poc of the standalone signer (#16154)
* signer: introduce external signer command

* cmd/signer, rpc: Implement new signer. Add info about remote user to Context

* signer: refactored request/response, made use of urfave.cli

* cmd/signer: Use common flags

* cmd/signer: methods to validate calldata against abi

* cmd/signer: work on abi parser

* signer: add mutex around UI

* cmd/signer: add json 4byte directory, remove passwords from api

* cmd/signer: minor changes

* cmd/signer: Use ErrRequestDenied, enable lightkdf

* cmd/signer: implement tests

* cmd/signer: made possible for UI to modify tx parameters

* cmd/signer: refactors, removed channels in ui comms, added UI-api via stdin/out

* cmd/signer: Made lowercase json-definitions, added UI-signer test functionality

* cmd/signer: update documentation

* cmd/signer: fix bugs, improve abi detection, abi argument display

* cmd/signer: minor change in json format

* cmd/signer: rework json communication

* cmd/signer: implement mixcase addresses in API, fix json id bug

* cmd/signer: rename fromaccount, update pythonpoc with new json encoding format

* cmd/signer: make use of new abi interface

* signer: documentation

* signer/main: remove redundant  option

* signer: implement audit logging

* signer: create package 'signer', minor changes

* common: add 0x-prefix to mixcaseaddress in json marshalling + validation

* signer, rules, storage: implement rules + ephemeral storage for signer rules

* signer: implement OnApprovedTx, change signing response (API BREAKAGE)

* signer: refactoring + documentation

* signer/rules: implement dispatching to next handler

* signer: docs

* signer/rules: hide json-conversion from users, ensure context is cleaned

* signer: docs

* signer: implement validation rules, change signature of call_info

* signer: fix log flaw with string pointer

* signer: implement custom 4byte databsae that saves submitted signatures

* signer/storage: implement aes-gcm-backed credential storage

* accounts: implement json unmarshalling of url

* signer: fix listresponse, fix gas->uint64

* node: make http/ipc start methods public

* signer: add ipc capability+review concerns

* accounts: correct docstring

* signer: address review concerns

* rpc: go fmt -s

* signer: review concerns+ baptize Clef

* signer,node: move Start-functions to separate file

* signer: formatting
2018-04-16 15:04:32 +03:00
GuiltyMorishita
e2f2bb3e2e node: fix typo hvosts -> vhosts (#16096) 2018-02-15 12:38:39 +02:00
Martin Holst Swende
6c6247a690
node, rpc: fix linter issues 2018-02-12 14:12:55 +01:00
Martin Holst Swende
589b603a9b rpc: dns rebind protection (#15962)
* cmd,node,rpc: add allowedHosts to prevent dns rebinding attacks

* p2p,node: Fix bug with dumpconfig introduced in r54aeb8e4c0bb9f0e7a6c67258af67df3b266af3d

* rpc: add wildcard support for rpcallowedhosts + go fmt

* cmd/geth, cmd/utils, node, rpc: ignore direct ip(v4/6) addresses in rpc virtual hostnames check

* http, rpc, utils: make vhosts into map, address review concerns

* node: change log messages to use geth standard (not sprintf)

* rpc: fix spelling
2018-02-12 14:52:07 +02:00
Lewis Marshall
54aeb8e4c0 p2p/simulations: various stability fixes (#15198)
p2p/simulations: introduce dialBan

- Refactor simulations/network connection getters to support
  avoiding simultaneous dials between two peers If two peers dial
  simultaneously, the connection will be dropped to help avoid
  that, we essentially lock the connection object with a
  timestamp which serves as a ban on dialing for a period of time
  (dialBanTimeout).

- The connection getter InitConn can be wrapped and passed to the
  nodes via adapters.NodeConfig#Reachable field and then used by
  the respective services when they initiate connections. This
  massively stablise the emerging connectivity when running with
  hundreds of nodes bootstrapping a network.

p2p: add Inbound public method to p2p.Peer

p2p/simulations: Add server id to logs to support debugging
in-memory network simulations when multiple peers are logging.

p2p: SetupConn now returns error. The dialer checks the error and
only calls resolve if the actual TCP dial fails.
2017-12-01 12:49:04 +01:00
Lewis Marshall
9feec51e2d p2p: add network simulation framework (#14982)
This commit introduces a network simulation framework which
can be used to run simulated networks of devp2p nodes. The
intention is to use this for testing protocols, performing
benchmarks and visualising emergent network behaviour.
2017-09-25 10:08:07 +02:00
Felix Lange
7e57fee355 node: fix instance dir locking and improve error message
The lock file was ineffective because opening leveldb storage in
read-only mode doesn't really take the lock. Fix it by including a
dedicated flock library (which is actually split out of goleveldb).
2017-08-18 12:14:00 +02:00
Chase Wright
1802682f65 node: Rename TrusterNodes (#14827)
* node: Rename TrusterNodes

* node: Rename TrusterNodes
2017-07-18 11:58:46 +03:00
Victor Farazdagi
5f7eb78918 node: fixes deadlock on Wait() 2017-04-25 18:04:02 +03:00
bas-vk
5e29f4be93 cmd/utils, node: remove unused solc references and improve RPC config (#14324)
Currently http cors and websocket origins are a comma separated string in the
config object. These are replaced with string arrays that are more expressive in
case of a config file.
2017-04-12 23:04:14 +02:00
Felix Lange
30d706c35e cmd/geth: add --config file flag (#13875)
* p2p/discover, p2p/discv5: add marshaling methods to Node

* p2p/netutil: make Netlist decodable from TOML

* common/math: encode nil HexOrDecimal256 as 0x0

* cmd/geth: add --config file flag

* cmd/geth: add missing license header

* eth: prettify Config again, fix tests

* eth: use gasprice.Config instead of duplicating its fields

* eth/gasprice: hide nil default from dumpconfig output

* cmd/geth: hide genesis block in dumpconfig output

* node: make tests compile

* console: fix tests

* cmd/geth: make TOML keys look exactly like Go struct fields

* p2p: use discovery by default

This makes the zero Config slightly more useful. It also fixes package
node tests because Node detects reuse of the datadir through the
NodeDatabase.

* cmd/geth: make ethstats URL settable through config file

* cmd/faucet: fix configuration

* cmd/geth: dedup attach tests

* eth: add comment for DefaultConfig

* eth: pass downloader.SyncMode in Config

This removes the FastSync, LightSync flags in favour of a more
general SyncMode flag.

* cmd/utils: remove jitvm flags

* cmd/utils: make mutually exclusive flag error prettier

It now reads:

   Fatal: flags --dev, --testnet can't be used at the same time

* p2p: fix typo

* node: add DefaultConfig, use it for geth

* mobile: add missing NoDiscovery option

* cmd/utils: drop MakeNode

This exposed a couple of places that needed to be updated to use
node.DefaultConfig.

* node: fix typo

* eth: make fast sync the default mode

* cmd/utils: remove IPCApiFlag (unused)

* node: remove default IPC path

Set it in the frontends instead.

* cmd/geth: add --syncmode

* cmd/utils: make --ipcdisable and --ipcpath mutually exclusive

* cmd/utils: don't enable WS, HTTP when setting addr

* cmd/utils: fix --identity
2017-04-12 17:27:23 +03:00
Péter Szilágyi
24b9860c1b cmd/geth, node: surface geth architecture into version (#13866) 2017-04-05 21:51:01 +02:00
Péter Szilágyi
d4fd06c3dc
all: blidly swap out glog to our log15, logs need rework 2017-02-23 12:16:44 +02:00
Felix Lange
e5edd3b983 cmd/bootnode, cmd/geth, cmd/bzzd: add --netrestrict 2016-11-22 22:21:18 +01:00
Péter Szilágyi
b61f48e5aa
cmd, mobile, node, p2p: surface the discovery V5 bootnodes 2016-11-14 18:00:14 +02:00
Felix Lange
cc0064b267 node: add InstanceDir method 2016-11-10 17:47:07 +01:00
Zsolt Felfoldi
49da42983a p2p/discv5: added new topic discovery package 2016-11-09 02:12:53 +01:00
Felix Lange
eeb322ae64 node: ensure datadir can be co-inhabited by different instances
This change ensures that nodes started with different Name but same
DataDir values don't use the same nodekey and IPC socket.
2016-09-16 15:24:31 +02:00
Felix Lange
312263c7d9 cmd/utils, node: create account manager in package node
The account manager was previously created by packge cmd/utils as part
of flag processing and then passed down into eth.Ethereum through its
config struct. Since we are starting to create nodes which do not have
eth.Ethereum as a registered service, the code was rearranged to
register the account manager as its own service. Making it a service is
ugly though and it doesn't really fix the root cause: creating nodes
without eth.Ethereum requires duplicating lots of code.

This commit splits utils.MakeSystemNode into three functions, making
creation of other node/service configurations easier. It also moves the
account manager into Node so it can be used by those configurations
without requiring package eth.
2016-08-17 17:39:03 +02:00
Felix Lange
91b7690428 rpc: add new client, use it everywhere
The new client implementation supports concurrent requests,
subscriptions and replaces the various ad hoc RPC clients
throughout go-ethereum.
2016-07-22 23:21:27 +02:00
Felix Lange
542b839ec7 node, p2p: move network config out of Server
This silences a go vet message about copying p2p.Server in package node.
2016-05-18 12:19:04 +02:00