2ce00adb55
* focus on performance improvement in many aspects. 1. Do BlockBody verification concurrently; 2. Do calculation of intermediate root concurrently; 3. Preload accounts before processing blocks; 4. Make the snapshot layers configurable. 5. Reuse some object to reduce GC. add * rlp: improve decoder stream implementation (#22858) This commit makes various cleanup changes to rlp.Stream. * rlp: shrink Stream struct This removes a lot of unused padding space in Stream by reordering the fields. The size of Stream changes from 120 bytes to 88 bytes. Stream instances are internally cached and reused using sync.Pool, so this does not improve performance. * rlp: simplify list stack The list stack kept track of the size of the current list context as well as the current offset into it. The size had to be stored in the stack in order to subtract it from the remaining bytes of any enclosing list in ListEnd. It seems that this can be implemented in a simpler way: just subtract the size from the enclosing list context in List instead. * rlp: use atomic.Value for type cache (#22902) All encoding/decoding operations read the type cache to find the writer/decoder function responsible for a type. When analyzing CPU profiles of geth during sync, I found that the use of sync.RWMutex in cache lookups appears in the profiles. It seems we are running into CPU cache contention problems when package rlp is heavily used on all CPU cores during sync. This change makes it use atomic.Value + a writer lock instead of sync.RWMutex. In the common case where the typeinfo entry is present in the cache, we simply fetch the map and lookup the type. * rlp: optimize byte array handling (#22924) This change improves the performance of encoding/decoding [N]byte. name old time/op new time/op delta DecodeByteArrayStruct-8 336ns ± 0% 246ns ± 0% -26.98% (p=0.000 n=9+10) EncodeByteArrayStruct-8 225ns ± 1% 148ns ± 1% -34.12% (p=0.000 n=10+10) name old alloc/op new alloc/op delta DecodeByteArrayStruct-8 120B ± 0% 48B ± 0% -60.00% (p=0.000 n=10+10) EncodeByteArrayStruct-8 0.00B 0.00B ~ (all equal) * rlp: optimize big.Int decoding for size <= 32 bytes (#22927) This change grows the static integer buffer in Stream to 32 bytes, making it possible to decode 256bit integers without allocating a temporary buffer. In the recent commit 088da24, Stream struct size decreased from 120 bytes down to 88 bytes. This commit grows the struct to 112 bytes again, but the size change will not degrade performance because Stream instances are internally cached in sync.Pool. name old time/op new time/op delta DecodeBigInts-8 12.2µs ± 0% 8.6µs ± 4% -29.58% (p=0.000 n=9+10) name old speed new speed delta DecodeBigInts-8 230MB/s ± 0% 326MB/s ± 4% +42.04% (p=0.000 n=9+10) * eth/protocols/eth, les: avoid Raw() when decoding HashOrNumber (#22841) Getting the raw value is not necessary to decode this type, and decoding it directly from the stream is faster. * fix testcase * debug no lazy * fix can not repair * address comments Co-authored-by: Felix Lange <fjl@twurst.com> |
||
|---|---|---|
| .. | ||
| internal | ||
| crawl.go | ||
| discv4cmd.go | ||
| discv5cmd.go | ||
| dns_cloudflare.go | ||
| dns_route53_test.go | ||
| dns_route53.go | ||
| dnscmd.go | ||
| enrcmd.go | ||
| keycmd.go | ||
| main.go | ||
| nodeset.go | ||
| nodesetcmd.go | ||
| README.md | ||
| rlpxcmd.go | ||
| runtest.go | ||
The devp2p command
The devp2p command line tool is a utility for low-level peer-to-peer debugging and protocol development purposes. It can do many things.
ENR Decoding
Use devp2p enrdump <base64> to verify and display an Ethereum Node Record.
Node Key Management
The devp2p key ... command family deals with node key files.
Run devp2p key generate mynode.key to create a new node key in the mynode.key file.
Run devp2p key to-enode mynode.key -ip 127.0.0.1 -tcp 30303 to create an enode:// URL
corresponding to the given node key and address information.
Maintaining DNS Discovery Node Lists
The devp2p command can create and publish DNS discovery node lists.
Run devp2p dns sign <directory> to update the signature of a DNS discovery tree.
Run devp2p dns sync <enrtree-URL> to download a complete DNS discovery tree.
Run devp2p dns to-cloudflare <directory> to publish a tree to CloudFlare DNS.
Run devp2p dns to-route53 <directory> to publish a tree to Amazon Route53.
You can find more information about these commands in the DNS Discovery Setup Guide.
Node Set Utilities
There are several commands for working with JSON node set files. These files are generated by the discovery crawlers and DNS client commands. Node sets also used as the input of the DNS deployer commands.
Run devp2p nodeset info <nodes.json> to display statistics of a node set.
Run devp2p nodeset filter <nodes.json> <filter flags...> to write a new, filtered node
set to standard output. The following filters are supported:
-limit <N>limits the output set to N entries, taking the top N nodes by score-ip <CIDR>filters nodes by IP subnet-min-age <duration>filters nodes by 'first seen' time-eth-network <mainnet/rinkeby/goerli/ropsten>filters nodes by "eth" ENR entry-les-serverfilters nodes by LES server support-snapfilters nodes by snap protocol support
For example, given a node set in nodes.json, you could create a filtered set containing
up to 20 eth mainnet nodes which also support snap sync using this command:
devp2p nodeset filter nodes.json -eth-network mainnet -snap -limit 20
Discovery v4 Utilities
The devp2p discv4 ... command family deals with the Node Discovery v4
protocol.
Run devp2p discv4 ping <enode/ENR> to ping a node.
Run devp2p discv4 resolve <enode/ENR> to find the most recent node record of a node in
the DHT.
Run devp2p discv4 crawl <nodes.json path> to create or update a JSON node set.
Discovery v5 Utilities
The devp2p discv5 ... command family deals with the Node Discovery v5
protocol. This protocol is currently under active development.
Run devp2p discv5 ping <ENR> to ping a node.
Run devp2p discv5 resolve <ENR> to find the most recent node record of a node in
the discv5 DHT.
Run devp2p discv5 listen to run a Discovery v5 node.
Run devp2p discv5 crawl <nodes.json path> to create or update a JSON node set containing
discv5 nodes.
Discovery Test Suites
The devp2p command also contains interactive test suites for Discovery v4 and Discovery v5.
To run these tests against your implementation, you need to set up a networking environment where two separate UDP listening addresses are available on the same machine. The two listening addresses must also be routed such that they are able to reach the node you want to test.
For example, if you want to run the test on your local host, and the node under test is also on the local host, you need to assign two IP addresses (or a larger range) to your loopback interface. On macOS, this can be done by executing the following command:
sudo ifconfig lo0 add 127.0.0.2
You can now run either test suite as follows: Start the node under test first, ensuring
that it won't talk to the Internet (i.e. disable bootstrapping). An easy way to prevent
unintended connections to the global DHT is listening on 127.0.0.1.
Now get the ENR of your node and store it in the NODE environment variable.
Start the test by running devp2p discv5 test -listen1 127.0.0.1 -listen2 127.0.0.2 $NODE.
Eth Protocol Test Suite
The Eth Protocol test suite is a conformance test suite for the eth protocol.
To run the eth protocol test suite against your implementation, the node needs to be initialized as such:
- initialize the geth node with the
genesis.jsonfile contained in thetestdatadirectory - import the
halfchain.rlpfile in thetestdatadirectory - run geth with the following flags:
geth --datadir <datadir> --nodiscover --nat=none --networkid 19763 --verbosity 5
Then, run the following command, replacing <enode> with the enode of the geth node:
devp2p rlpx eth-test <enode> cmd/devp2p/internal/ethtest/testdata/chain.rlp cmd/devp2p/internal/ethtest/testdata/genesis.json
Repeat the above process (re-initialising the node) in order to run the Eth Protocol test suite again.
Eth66 Test Suite
The Eth66 test suite is also a conformance test suite for the eth 66 protocol version specifically.
To run the eth66 protocol test suite, initialize a geth node as described above and run the following command,
replacing <enode> with the enode of the geth node:
devp2p rlpx eth66-test <enode> cmd/devp2p/internal/ethtest/testdata/chain.rlp cmd/devp2p/internal/ethtest/testdata/genesis.json