Integrates the custom rate limiter in the previous PR into the rest of the application. Also takes the opportunity to clean up how we instantiate Redis clients so that we can share them among multiple different services.
There are some config changes in this PR. Specifically, you must specify a `base_rate` and `base_interval` in the rate limit config.
Our current proxyd deployment does not share rate limit state across multiple servers within a backend group. This means that rate limits on the public endpoint are artifically high.
This PR adds a Redis-based rate limiter to fix this problem. While our current rate limiting library (github.com/sethvargo/go-limiter) _does_ support Redis, the client library it uses is not type safe, is less performant, and would require us to update the other places we use Redis. To avoid these issues, I created a simple rate limiting interface with both Redis and memory backend.
Note that this PR only adds the new implementations - it does not integrate them with the rest of the codebase. I'll do that in a separate PR to make review easier.
* proxyd: Add batch size metric and configurable max
The max batch size will be overwritten if it is over `MaxBatchRPCCallsHardLimit`.
Builds on https://github.com/ethereum-optimism/optimism/pull/3544.
* changeset
* fix lint
* fix test
* proxyd: Add customizable whitelist error message
Alchemy is asking for this so that we can include their affiliate link.
* add missing test case
* fix error message
* goimports
* proxyd: Fix error message in tests
The custom error message wasn't being carried over in batch responses.
* review fixes
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* proxyd: Add frontend rate limiting
To give us more flexibiltiy with rate limiting, proxyd now supports rate limiting of client (frontend) requests in addition to upstream (backend) requests. This PR also gives us the ability to exempt certain user agents/origins from rate limiting.
* lint
We never want to expose Geth's accounts to the public internet, so proxyd will now return `[]` for `eth_accounts` RPC calls without hitting the backend.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Fixes a panic in the websocket proxyd logic. Normally, the `clientPump` and `backendPump` methods in `WSProxier` send data in one direction. However, when the client sends an invalid RPC, the `clientPump` will send a response _directly to the client_ in order to avoid unnecessary roundtrips to the backend. This could be interleaved with concurrent writes to the client's WS in `backendPump`, and would cause a panic in the WS library.
To test this, this PR includes a dedicated integration test that reliably triggers the issue. In addition, this PR adds additional testing for WS functionality.
This fixes a bug where the infura backend would be labeled offline because it
returns an unexpected JSON-RPC response. Unexpected, but well-formed,
JSON-RPC response are handled specially. Such errors are surfaced up to
the backend proxier so failover still occurs.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* opnode: Reconcile epochs and handle reorgs (#309)
* opnode: Switch to uint64 in FindL2Heads function
* opnode: Properly initialize state loop
This properly sets the L1 head, L2 Head, and L2 Unsafe Head during
`state.Start()` in normal operation and when the current L1 head
has not reached the L1 genesis.
* opnode: Rename output functions
The old names where not good.
In addition, this standardizes the order of arguements and extends the
returns. The returns are not fully used, but will be.
* opnode: Add sanity check to L1 window in insertEpoch
This make sure that the L1 window starts at the correct block.
* opnode: Split out loop
Now each action occurs in a seperate function. The state loop is
still responsible for sequencing follow-up actions, but is not
responsible for setting state.
* opnode: Better name for batch prepare functions
This name indicates that it provides missing batches.
* opnode: Verify epochs as sequencer
Sequencers now verify epochs and will reorg if they see that verified
epochs do not match what they created.
* opnode: Mute driver test
* opnode: Set L2 Geth forkchoice on reorg
This is required to handle L1 reorgs (rather than just missing
batches).
* opnode: Bump op-geth version
Needed to fix a bug where the L2 geth node was missing the `mint` field
in deposit transactions when returning them over the JSON RPC server.
* opnode: Don't incorrectly advance safehead
There is some L2 block that is possible to fully derive from L1;
however, if it is ahead of the current state's safe head, do not
advance the current state's safehead because the chain has not
been verified up to that point.
* opnode: Don't rebuild L1 Genesis Block ID
Co-authored-by: Diederik Loerakker <proto@protolambda.com>
* opnode: More comments
Co-authored-by: Diederik Loerakker <proto@protolambda.com>
* opnode: Don't cast slice
Co-authored-by: Diederik Loerakker <proto@protolambda.com>
* opnode: PR comments
* opnode: Better error reason in attributesMatch
Include the details of the mismatched fields in addition to what
field it was. Will make diagnosing errors much easier.
Co-authored-by: Diederik Loerakker <proto@protolambda.com>
* go.mod: update to squashed l2 geth diff d5e1fc1a74bda3cbe5715d5732621460f9e00908
* opnode/rollup: use L2 block height in L1-info deposit tx block-height field (tx field, not info field) for uniqueness of deposit tx hash on L2
* feat: DepositFeed receive function
It is convenient for users to be able to deposit funds
directly to L2 by simply doing an ETH send to an address.
The prevents needing an ABI.
* Update slither db
* Add solidity test on df.receive() function
* Add warning message on DepositFeed.receive()
* specs: Add Withdrawals specification
* contracts: Add withdrawor and start of tests
* ops: Add withdrawor predeploy to docker setup
ops: Add contract code to Makefile and Dockerfile
* Add gaslimit to withdrawal encoding
* test(contracts): Withdrawal creation and proof gen
* deps: install entire goddamn monorepo using forge.
I needed the trie contracts over here, so now the whole monorepo
is a submodule of the specs repo. Happy now @norswap?
Why, you ask, did I do this rather than the alternatives available to
me?
1. Copying and pasting is gross.
2. `yarn add` would have installed a smaller dep, but it's also gross.
3. git submodules, while also gross in their own right, is at least
consistent with how we're currently handling other dependencies, so
I went with that.
* Add output root inclusion proof
Adds a check to ensure that withdrawal root is included in the
l2 output root.
* Add storage inclusion proof
Adds a check to ensure that the withdrawal message hash is included
in the mapping of withdrawals in the withdrawer contract
* Improve testing against L2 node
* Add finalizationWindow var to WithdrawalVerifier
* Add WithdrawalVerified event
* Remove unused constructor argument
* Clean up Forge test of WithdrawalVerifier
* ops: Docker get l2 genesis hash on startup
Docker fixes
* contracts: ignore dirty submodules
* Don't run hardhat tests in CI
* Create Optimism Portal contract
* Move receive function to OptimismPortal
* specs: Add overview page (#286)
* Convert WithdrawalVerifier to a library
This changes the WithdrawalVerifier to be stateless, and moves all
checks into the OptimismPortal
* contracts: Rename Withdrawor to Withdrawer
* specs: define timestamp bounds of derivation process, clear up block derivation, define missing noTxPool field
* opnode/rollup: implement new timestamp bounds on block derivation batch filtering and filling
* opnode: fix sequencer L1 origin advancing, and fix e2e genesis time
* contracts: Add burn function to the Withdrawer contract
* contracts: Add natspec comments
* contracts: Remove timestamp from OutputRootProof
* contracts: undoL1ToL2Alias on withdrawals from a contract
Also uses AddressAliasHelper lib for aliasing instead of reimplementing
it.
* specs/rollup-node: improve spec based on review suggestions from @norswap
* opnode/rollup: implement review suggestions from @trianglesphere
* specs/rollup-node: document timestamp invariant between L2 and L1 origin
* opnode/rollup/driver: continue sequencing on current epoch if next L1 origin timestamp is not reached yet
* opnode/rollup: fix maxL2Time excl bound, and return adjusted maxL2time in long L1 gap case
* contracts: Add withdrawer tests
fixup! contracts: Add burn function to the Withdrawer contract
* Rename Finalization Window to Finalization Period
* contracts: Validate output root proof in portal
Moves the comparison into the Portal contract, and changes the
WithdrawalVerifier function from _verifyWithdrawerStorageRoot to
_deriveOutputRoot.
* contracts: Use solidity custom errors
* contracts: Use unchecked math when safe
* contracts: Add section dividers to contract layout
* specs: Update withdrawals spec to match impl
* contracts: Fix ETH burning in the withdrawer contract
* specs: Address review comments
* ops: Address review comments
* Update Dockerfile.opnode
* ops: Fix devnet
- Fixes the devnet config to correctly generate rollup/genesis files.
- Fixes a bug that causes the rollup node to get stuck when there's a large timestamp delta between genesis and the current time, as is the case in the devnet.
* Update opnode/rollup/driver/state.go
Co-authored-by: Diederik Loerakker <proto@protolambda.com>
* Update rollup.json
* Update predeploy addresses
The new predeploy addresses are incremented by 1 in order to avoid
conflicts with the system addresses.
* tests: Add tests for batch requests when the parent request fails
This test makes sure that we do not retry when the parent request fails. We don't, but it wasn't immediately clear.
I also removed a duplicate assertion lib since we're using `testify` everywhere else.
* opnode: Don't start sequencing blocks until past the L1 genesis (#325)
This has to be reverted to ensure that the sequencer starts producing
blocks at the same location as the verifier. Sometimes the previous
code would cause reorgs on the sequencer (expected), and sometimes
it would cause a chain split (unexpected and unexplained as of
writing this commit message 4/6/22).
* opnode: update to latest L2 geth
* readme: fix typos and break lines (#327)
* Update ref optimistic geth image (#328)
* ci: Improve timestamp management in devnet (#329)
This change sets timestamps to the current timestamp on fresh devnet deploys. This avoids the devnet having to fill in a bunch of blocks on startup. The timestamp is preserved between runs; `make devnet-clean` will erase the timestamp data if you want to start anew.
* contracts: Move withdrawal logic into abstract contract
* contracts: Move withdrawal hashing logic into library
* contracts: Update .gas-snapshot
* contracts: silence compiler warnings
* feat: expose debug http api (#334)
* opnode: Use l2 safe head instead of l2 head
Epochs should be inserted on top of the L2 Safe Head, not the
unsafe head. This was causing problems when a reorg would occur
and the node would start building in the wrong spot.
* opnode: Clear l1WindowBuf on insertEpoch failure
This is done to handle the case that the l1WindowBuf does not start
in the correct location (direct check is difficult with context).
The correct response to that is to clear the window and initialize
it from the correct block.
* hardhat: clean up the hardhat deposit task
* fix: oe => optimism (#338)
* opnode: Rework state test to build on safe head
This also got caught up in the unsafe/safe head confusion.
* opnode: fix setL1BlockValues selector used for deposits (#339)
Solidity ABI only looks at the function name and types, but does not
consider the name of the function parameters.
* rollup: fix address of the L1 Block Attributes contracts
* fix: typo in l1InfoPredeployAddress
* fix: gofmt and goimports
CI was asking for goimport'd when all it needed was gofmt.
* opnode: Import constants
* ops: fix formatting, add docker buildkit support
* ops: prune volumes on devnet-down
* fix: replace fraud with fault (#337)
* opnode: Fix bug in block creation (#345)
The NoTxPool flag was incorrectly set to falses in the InsertEpoch
function.
* opnode: Don't allow arbitrary mints (#346)
The ordering of `mint, value` was switched to be `value, mint`
when parsing emitted logs. This allows users to control their
mint rather than using `msg.value` which is being fed directly
into the event.
Thanks to Mofi (Inphi) for finding this.
* opnode,specs: deposit tx-hash uniqueness based on L1
* opnode/l1: verify critical receipt/log info we get from RPC
* opnode/test: reconstruct deposit hash in e2e test, fetch l2 deposit receipt
* opnode/l1: fix panic on channel send after close, sema channel can stay open
* go.mod: l2 geth update with improved deposit tx hash
* README: remove alpha branch + update description (#332)
* allow building everything from top-level makefile
* allow testing from top level makefile
* Update Makefile
Co-authored-by: Joshua Gutow <jgutow@optimism.io>
* opnode/l1: wrap errors and fmt test imports
* opnode: don't expose types.Block to derive or driver packages, prefer l2.ExecutionPayload
* opnode: add comments, fix genesis base case to get l1 origin
* init: integration tests (#344)
Co-authored-by: Javed Khan <tuxcanfly@gmail.com>
* opnode: Fix possible panic in sequencer batch creation (#342)
Previously we assumed that every deposit would be included in the L2
block. However, the L2 execution engine removes depoists that are
not valid. As such the number of deposits submitted to the execution
engine could be greater than the number of transactions returned
causing a panic.
This fixes the issue by counting the number of deposits returned
and using that to slice the payload transactions for batch submission.
(Deposits are not submitted in batches and instead read directly from L1).
This also does more sanity checks on the transactions included in the L2
block and rejects blocks with invalid deposit tx configurations. This is
done prior to marking the payload canonical with the FCU.
* opnode: Test Utilities (#319)
* opnode: Deduplicate fake chain source
The fake chain source is an easy way to create and L1 and L2 chains from
a simple string defition. There were previously two implementations and
now they are in a shared testutils package.
* opnode: Dedicated wait functions
This deduplicates wait for transactions on L1 and L2.
* opnode: New e2e test system
This splits out the initialization from the actual tests to eanble
multiple e2e tests. Some of the config is still not as dedpulicated
as I would like, but it is more centralized than it previously was.
Aims to move the setup from the tests and enable easier e2e tests.
* opnode: Silence FindSyncStartTest
* opnode: Add test for missing batches
This ensures the the sequencer follows the chain that is derived
from L1 rather than what it sequenced in the case that it misses
batch submission.
* opnode: Pull out system config to default cfg
This reduces duplication and makes tests easier to understand.
* opnode: Address PR comments
* opnode: Reduce flakiness in missing batch test
The proper way to do this is to wait for a safe block, but the API
is not yet ready for that.
* Fix sync-start bug + improve sync-start comments (#355)
Fix bug (L1 origin head with non-canonical ancestors)
This happened in the scenario where `l2ahead == true` (L1 origin of start is
ahead of know L1 head) and `start` had an ancestor whose L1 origin was known not
to be canonical.
Example: L1 head is at block X. `start` has an L1 origin with number X + 1 but its
parent L2 block has an L1 origin with number X whose blockhash is not canonical.
In this case, the algorithm still returned `start` as the unsafe L1 head, which is
incorrect.
This also touches a bunch of comments to generally improve understanbility.
* feat: deposit tx js helpers (#352)
* feat: js deposit tx
Hardhat task now lists the tx hash
* rollup: new deposit hashing
* nice api!
* fix: dockerfile
* devnet: Hardcode deposit feed bytecode (#359)
* opnode,l2os: Better lifecycle management (#358)
* opnode: Properly wait for limit client to shut down
We removed closing the semaphore channel because it would cause a
panic as we did not wait for in flight requests (and new requests)
to stop before closing the L1 source.
* l2os: Change shutdown procedure
This is now blocking, but also reuces the amount of error logs.
* opnode: Shutdown rollup nodes before clients/geth nodes
Also actually shuts down the L2 clients in the rollup node.
* opnode: Properly shut everything down
* style: clean up and comments for state.go (#356)
* opnode: Parameterize deposit contract address (#361)
The contract can be deployed to any address. It is set in the
rollup config file.
* opnode/test: Add reverted deposit with mint test
* fix(ci): use Foundry GHA (#367)
Nightly Foundry releases are kept around for 3 days. This means we cannot be pinning the download link on CI to a specific commit.
Instead, we use the Foundry Toolchain GHA which always uses the latest nightly version.
* opnode/test: check nonce increment in deposit fail
* integration-tests: End-to-end withdrawal test (#362)
Includes several fixes to make this work:
- Modifies the devnet to deploy the L1 contracts rather than include them in genesis.
- Changes the deployment of L2OutputOracle to support a custom start timestamp.
- Updates UserDeposits to properly filter out non-TransactionDeposited events.
- Updates the L2 output submitter to post the current L1 block with each output rather than the L2 checkpoint block
- Updates the opnode API to allow all vhosts and CORS domains.
* go.mod: bump l2geth version
* opnode: Update and deploy contracts (#360)
This is rather than hardcoding the contract bytecode. It slows down the e2e
tests (by having to wait 1 L1 block for contracts to be ready), but it
is more accurate to what occurs (and is now required as we initialize
actions).
This also pins the L2OutputOracle to v0.8.10 of solidity.
* specs: Update output root derivation (#357)
* specs: Update output root derivation
* specs: Separate version from payload
* opnode: libp2p setup
* p2p flags, config, and libp2p + discv5 setup
* specs: initial rollup node p2p spec
* specs: Fix definition of output root
* solidity: fail if updated gas snapshot isn't included
A new gas snapshot can be generated by running
`forge snapshot`
* p2p setup: implement review suggestions, fix toc, fix lint
* opnode/p2p: test with require instead of assert
* contracts: Fix reentrancy attack in WithdrawalsRelay (#378)
* contracts: Output oracle improvements (#370)
* contracts: Use correct uppercase for immutables
* itest: shorten submission interval for faster devnet tests
* contracts: Ensure submission interval is multiple of block time
* itest: Simplify withdrawals spec
* itest: Increase withdrawal finalization timeout
* itest: Bump withdrawal timeout to +5 minutes
* Bump timeout
* Bump again
* Add gas snap
Co-authored-by: Matthew Slipper <me@matthewslipper.com>
* Clean up withdrawals itest (#372)
* contracts: Use correct uppercase for immutables
* itest: shorten submission interval for faster devnet tests
* contracts: Ensure submission interval is multiple of block time
* itest: Simplify withdrawals spec
* itest: Increase withdrawal finalization timeout
* itest: Bump withdrawal timeout to +5 minutes
* itest: Extract logic into getTargetOutput
* Update 000_withdrawals.spec.ts
Co-authored-by: Matthew Slipper <me@matthewslipper.com>
* fix dead anchor link
* higher difficulty rather than number
* 'L1 attributes block' is not a thing + add link
* try to align sentences with lines
* ci: Add automated docker builds for opnode and l2 output submitter (#369)
* contracts: Add deleteL2Output function
* specs: Add deleteL2Output
* ci: Update Docker version in CircleCi (#381)
We ran into this on the monorepo too. Alpine 3.14 removes the `faccessat2` syscall. This caused "operation not permitted" errors while building. Upgrading Circle's Docker version fixes this. See https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.14.0#faccessat2 for more information.
* contracts: reuse code (#380)
* contracts: reuse the library to compute withdrawal hash
The `WithdrawalVerifier` has a method to compute the
withdrawal hash as part of a library. Use this library
in both the L1 and L2 contracts so that code can be reused.
* contracts: gas snapshot
* opnode: Better parsing of the TransactionDepositedEvent (#382)
This now asserts that the offset for the bytes field of the event
is the correct value rather than the previous incorrect check. This
check is not fully necessary, but good to have to validate that the
data is well formed.
* Makefile,ops: docker compose up instead of run to make devnet-down work, and fix makefile label (#385)
* opnode: Do not drop all deposits on parsing error (#383)
* Change deposit gasLimit from uint256 to uint64
This prevents the user from setting a gas limit that the rollup node
is not able to parse.
* opnode: Do not drop all deposits on parsing error
Only drop the affected deposit. This does mean loss of funds, but
this is also a case that should never happen. In addition, halting
the chain means that while funds are safe, we are exposed to a
denial of service attack if it would be possible to cause a parsing
error from the deposit feed contract.
* opnode/test: Provide flag to access geth logs (#389)
This enables a more verbose output without having to modify the test
itself. Provide the flag `gethlogs` to see logs from geth.
* opnode: Fix L1 Info Transactions
There are three issues being fixed:
1. Set the `from` field to the correct magic value (found by ToB)
2. Update the L2 EE to provide gas to deposits. Otherwise the
deposit transactions immediatley out of gas and fail.
3. Update the manual ABI encoding to match what solidity expects.
The previous version was similar to packed encodeding, but as
such could not be parsed by solidity.
This also includes a regression test that the parsing of the L1 info
tx and what is recorded in the state for each block matches.
* contracts: import messengers (#393)
* chore: copy L1 and L2 Messengers from Monorepo
* test: Add Messenger test files
Monorepo test cases are copied in as comments from the monorepo ts tests.
* chore: Add OZ upgradable contracts
New remappings were also added in order to avoid excessively
long import statements.
* chore: Importe OZ and OP contracts as node_modules
Necessary because hardhat does not support remappings!?!
y u no?
* refactor: Remove replayMessage()
We no longer need this function as it was only necessary when the CTC
a maximum gas limit per epoch concept. In order to remove the function
I had to copy in the L1xDM interface rather than import from the
node_modules.
* chore: Remove unused files
The ts test file is made redundant by an itest in the itest package.
The sol test file was not being used for anything
* refactor: Remove Address Manager and Resolver
Instead the CTC and SCC are state variables
* refactor: Replace CTC with OptimismPortal
* refactor: Remove SCC
We don't need to replace it with the L2OutputOracle in the L1xDM,
because the verification is now done in the OptimismPortal itself.
* forge install: forge-std
* forge install: ds-test
* refactor: Move boilerplate test code into CommonTest
* test: Add sendMessage and pause tests for L1xDM
* test: L1CrossDomainMessenger sendMessage and pause
* refactor: replace L2ToL1MessagePasser with Withdrawer contract
Also adds a lib with constant values for new bedrock predeploys.
* chore: Make functions external, and reorder for CEI
For whatever reason a bunch of functions on the messengers were public,
when they could have been external. I fixed that, and removed the
slither annotations. Where possible (in the sendMessage functions), I
also reordered the events and calls to respect
Checks-Effects-Interactions. There was no risk previously, but this
removes any question, and quiets slither.
* refactor: Reorganize ts helpers
Move helpers/index into utils.ts, and add other files which are exported in
the new index.ts.
* test: Add mock proof generation script and helpers
* test: Add L1xDM relayMessageSucceeds tests
* test: Add proof generation scripts and helpers
* refactor: Add l2Sender check in L2xDM
* refactor: Copy in the L1 and L2 standard bridge
At this point they are simply verbatim.
* refactor: Bridges - fix import paths
* refactor: Token Bridge - replace messengers with Portal
Also remove the CrossDomainEnabled lib.
refactor: Token Bridge - replace messengers with Portal
Also remove the CrossDomainEnabled lib.
* style: Address/remove some slither disable comments
style: Address/remove some slither disable comments
* refactor: extract l2Sender check into a modifier
* refactor: Support deposits of ETH in L2 Bridge
This copies in the IL2ERC20Bridge interface so that payable can be added.
In the case that the L2 token address matches OVM_ETH, the value of the
call will be forwarded.
* interface: add IWithdrawer.sol
* contracts: add comments to L1 contracts
* contracts: use unchecked
* contracts: fix imports in common test
* test: L1CrossDomainMessenger
* test: L2CrossDomainMessenger
* contracts: fix typo
* tests: bridge tests
* contracts: remove extra message assignment
* contracts: update gas snapshot
* forge install: solmate
* contracts: remove usage of OVM_ETH
All `OVM_ETH` will be migrated to `ETH` with the upgrade
to bedrock. We do not want to allow for the creation of new
`OVM_ETH` by depositing `ETH` into the bridge and have it
create `OVM_ETH` on L2.
* contracts: add in L2StandardERC20
* contracts: add in token factory
The token factory will deploy tokens on L2
that correspond to tokens on L1. This allows for
easy deposits through the bridge.
* contracts: test rlp lib for computing contract addrs
This library lets you compute the contract address
based on the deployment account and nonce.
h/t @t11s
* contracts: test infra for bridge
* contracts: add note to self
* contracts: fix compiler warnings
* contracts: update snapshot
* contracts: add IDepositFeed
* contracts: type cast uint256 to uint64 in messenger
* test: fix merge
* contracts: modify paths to compile with hardhat
* hardhat: update config
* contracts: fix build
* forge tests: first yarn install
* contracts: lint
* contracts: update snapshot
Co-authored-by: Maurelian <maurelian@protonmail.ch>
* contracts: fix abi encoding in proof verification (#395)
Use `abi.encode` instead of `abi.encodePacked` to ensure
a constant serialization. `abi.encode` will be sure to
pad the value to its size while `abi.encodePacked` will not
when operating on integers. There should not have been a bug here
because it was being called with a `bytes32`, which should
always be 32 bytes when returned from `abi.encodePacked`.
* contracts: don't unalias on L2->L1
Co-authored-by: smartcontracts <kelvin@optimism.io>
* opnode: attach p2p host and discovery to rollup node, update e2e test (#388)
* deps: update forge-std
* contracts: fix build
* build: fix bindings build
* fix: builds
* forge install: ds-test
* build: fix for good
* l2os: update bindings
* opnode: add seq nr to l1-info to fix driver bug
* feat: create opnode/predeploy package
This resolves an import cycle when trying to use
WithdrawalContractAddress in opnode/node. Otherwise the compiler
complains that opnode imports opnode/node which imports opnode.
* opnode: refactor RPC server / config, prototype batch serving
* feat: enforce size contraints on batch bundles
This algorithm is modeled mostly off the one used in the existing batch
submitter. The additional complexity enables us to configure a min and
max value such that max - min < max_l2_tx_size. Applied strictly,
submitting a large L2 tx would cause the batch construct to halt, since
the batch constructed without including batch `i` can be under the min,
while including batch `i` resulting in exceeding the maximum. The
additional complexity in the pruning algorithm is used to identify this
case and submit an undersized batch when it cannot be avoided. In the
general case, however, this strategy increases our expected
profitibability since the average batch size is closer to the desired
maximum.
* feat: add BundleBuilder helper class
* fix: typo in l2os config docs
* feat: add GetBatchBundle method to rollupclient.RollupClient
* feat: simplify l2os rollupclient init
* feat: reduce BSS poll interval
The current setup assumes the batches are submitted very soon after the
block is created on L2. Increasing the poll interval ensures that we
discover and publish new batches in response. Without modifying this
value, the subsequent changes will still fail even though the code is
correct.
* feat: add sequencer history db
* feat: add sequencer BSS driver to submit batches
* feat: add bss configuration
* feat: add LOG_TERMINAL flag to l2os, use non-global log instance
Also modifies the shutdown logic of l2os.Service to abort if stuck
publishing a transaction.
* feat: enable standalone bss, disable simple bss
* feat: remove simple bss
* feat: add bss to devnet
* fix: ci build bindings (#406)
* ci: build bindings in ci
* ci: install abigen
* ci: fix geth install
* bindings: regenerate
* contracts: no metadata hash in contracts
* bindings: regenerate
* opnode: Avoid busy-waiting while L2 head is behind L1
In the state select loop, we immediately request for new L2 blocks whenever
the latest L1 origin is behind L1. This can be an issue as we attempt to
re-request the latest L2 block without any delay. This is a DoS hazard
particularly when an L2 block cannot be retrieved because either the `L1Chain`
or `L2Chain` backends have errors.
Another problem with this is that other, more useful events in the state
select loop, are less likely to be scheduled by the Go runtime due to
the busy-wait.
Adding a small delay, before calling `reql2BlockCreation` of about 10ms
should be enough to prevent issues.
* opnode: Fix Incorrect error handling when creating an L2 block (#391)
* opnode: Fix Incorrect error handling when creating an L2 block
This is an issue ToB identified (issue #8). What happens is that
the ethereum.NotFound error would never be returned, but in general
if there is an error in the lookup, the state loop should keep
going.
* opnode: Properly wrap errors in l1/source.go
* opnode: Fuzz manual ABI parsing (#384)
* opnode: Add OptimismPortal deployed bytecode
This is important for fuzzing
* opnode: Add differential fuzzing test for deposit events
fixup: Proper tests
* build: Add fuzz target
* opnode: Fix ineffctual error assignment
* opnode: More fuzzing cleanup
The purpose of this is to make it more likely that the deposit
succeeds and that we fuzz what we actually want to fuzz (the
parsing).
* fix: DepositFeed.sol link
* opnode: gossip blocks topic validator and subscriber
* opnode/l2: ExecutionPayload SSZ encoding/decoding
* opnode/l2: check payload block hash consistency
* opnode/p2p,specs: update p2p block gossip validation
* opnode: refactor node initialization, change l1-head fan out, implement l2 payload driver receiver, setup block signing, update config / cli / e2e
* opnode/l2,opnode/rollup/driver: process incoming unsafe L2 blocks
* specs/rollup-node-p2p: fix toc
* opnode/p2p: compress published messages
* specs/rollup-node-p2p: clarify hardfork version start
* opnode/node: fix l1 head notify timeout bug
* opnode: fix receive p2p payload timeout bug, fix publishing buffer missing reset
* opnode: fix/improve new loggers
* opnode: fix p2p block distribution
* opnode/test: connect peers only after starting p2p application-layer protocols
* ops,opnode/p2p: fix opnode start up in devnet
* opnode/l2: use common.Big0 instead of big.NewInt(0), thx mark
* opnode: Add debug tracer to geth in e2e test (#418)
* Update reference geth entrypoint with variable parameters
* opnode: fix imports and code newlines style
* opnode,specs: gossip blocks validation seen cache fixes
* opnode/rollup/driver: no heuristics in driver, rely more on engine to handle reorgs
* opnode: tracer to watch node events during testing
* opnode,specs: clarify gossip params
* opnode/l2/ssz: fix lousy copy range
* opnode/rollup/driver/state: fix doc typo
Co-authored-by: Joshua Gutow <jgutow@optimism.io>
* Add version RPC (#432)
Meta:
- Fixes ENG-2200
* refactor: new messengers (#421)
* contracts: consolidate OptimismPortal
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: delete abstracts
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: base `CrossDomainMessenger` + L1 and L2
Implement the `L1CrossDomainMessenger` and
`L2CrossDomainMessenger` based on the base
`CrossDomainMessenger`. This makes the interfaces
the same on both sides.
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: `StandardBridge`
Also implement the `L1StandardBridge` and
`L2StandardBridge` based off of the base
`StandardBridge`
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: L2OutputOracle
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: standard bridge tests
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: cross domain hashing lib + tests
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: tests for cross domain messengers
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: L2ToL1MessagePasser
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: safe call lib
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: optimism mintable erc20
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: common test setup
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: optimism mintable token factory
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: update forge-std
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: test OptimismPortal
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: L1Block attributes
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: update libraries
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: update foundry.toml
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: Burner
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: delete dead code
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: update hh deploy scripts
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: update gas snapshot
Co-authored-by: smartcontracts <kelvin@optimism.io>
* integration-tests: update for new messengers
Co-authored-by: smartcontracts <kelvin@optimism.io>
* ops: devnet up script new genesis
Co-authored-by: smartcontracts <kelvin@optimism.io>
* opnode: new bindings
Co-authored-by: smartcontracts <kelvin@optimism.io>
* l2os: new bindings
Co-authored-by: smartcontracts <kelvin@optimism.io>
* contracts: update gas snapshot
Co-authored-by: smartcontracts <kelvin@optimism.io>
* specs: L2ToL1MessagePasser
* specs: messengers
* specs: bridges
* specs: fix linting issues
* contracts: better contract for is optimism mintable
* contracts: fix some comments
* contracts: assert finalization window has passed
* bindings: regenerate
* contracts: address comments
Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com>
* opnode: p2p RPC, fix static-peers
* StateViz: Visualize rollup state changes (#419)
* StateViz: Visualize rollup state changes
* remove DEBUGMEs
* Add L1WindowBuf to viz
* move stateviz to opnode/cmd
* Add stateviz to docker-compose
* html lint
* Add SRI
* Fix linter
* Merge fixes
* Fixes from code review
* Imports
* Fix space
Co-authored-by: Matthew Slipper <me@matthewslipper.com>
* contracts: add backup logic for deposits
Introduces backup logic for deposits that prevents users from
accidentally making bad deposits. Deposits that don't complete
successfully will be returned back to the layer where the deposits were
made.
* contracts: add base gas to sent messages
Introduces base gas to the CrossDomainMessenger's sendMessage function.
Base gas is used to guarantee that all messages sent between messengers
will at the very least be able to store the message hash on the other
chain and therefore be replayable. Base gas scales dynamically with the
size of the message.
* contracts: use clearer message encoding
Uses the clearer abi.encodeWithSelector within CrossDomainMessenger's
sendMessage function. Doing this makes it much more apparent that a call
to the relayMessage function is being triggered.
* contracts: add L1BlockNumber predeploy
Re-introduces the L1BlockNumber predeployed contract for backwards
compatibility with the previous OVM_L1BlockNumber contract.
* opnode: p2p rpc client bindings, test update, minor p2p flag update
* opnode/p2p: set up connection notification before starting host B
* opnode/p2p: bundle p2p components into p2p node for separate testing
* opnode/p2p: minor style fix + rename
* opnode/p2p: add method to get peer info about self
* Fix SRI on stateviz (#440)
* Fix L2 Output Timestamps (#416)
There was an off by one error in the L2 Output Oracle contract in the timestamp to
block number conversion. In addition, the L2 Output Submitter (op proposer) did
not recognize that there was a mismatch between the timestamp/block number in
the header and the timestamp/block number from the contract.
This bug causes problems when trying to do withdrawals.
Fixes ENG-2128
* opnode: Withdrawal E2E test in go (#423)
This does the following:
- Adds withdrawal utilities (to opnode/withdrawals)
- Adds an end to end test in go of withdrawals
- Adds the L2 withdrawer contract
- Updates to a newer version of reference-go-ethereum
Fixes ENG-2202
* Adopt go.work, rename modules to prep for monorepo (#441)
Co-Authored-By: Matthew Slipper <me@matthewslipper.com>
Co-authored-by: protolambda <proto@protolambda.com>
* remove git submodules
* optimistic-specs: monorepo merge, mv into protocol dir
* bedrock contracts dependencies
* monorepo merge: fix Go env
* monorepo merge: move contracts-bedrock into non-Lerna package
* monorepo merge: bring back CI
Known issues:
- There are broken links that `lychee` picks up on. These still need to be updated.
- Slither returns errors, both here and in `optimistic-specs`.
- `go-bip39` was updated to a newer version. The newer version broke a `bss-core` test, which had to be fixed.
- Forge is not compatible with Lerna. As a result, the `contracts-bedrock` package had to be moved out of the `packages` hierarchy.
- The devnet itests don't work because the Go modules aren't on the default branch. We need to decide if we merge to develop, or stay on a feature branch before fixing this.
* outline bedrock dirs, rm stale protocol repo files (#2562)
* monorepo merge: re-run yarn
Co-authored-by: Joshua Gutow <jgutow@optimism.io>
Co-authored-by: Diederik Loerakker <proto@protolambda.com>
Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com>
Co-authored-by: Maurelian <maurelian@protonmail.ch>
Co-authored-by: smartcontracts <kelvin@optimism.io>
Co-authored-by: Murphy Law <Inphi@users.noreply.github.com>
Co-authored-by: norswap <norswap@gmail.com>
Co-authored-by: Javed Khan <tuxcanfly@gmail.com>
Co-authored-by: Murphy Law <mlaw2501@gmail.com>
Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
Co-authored-by: Conner Fromknecht <conner@alum.mit.edu>
Co-authored-by: Luca Donno <30298476+lucadonnoh@users.noreply.github.com>
Co-authored-by: Ben Wilson <bwilson@optimism.io>
Co-authored-by: Ben Wilson <82120899+optimisticben@users.noreply.github.com>
- Adopts Go workspaces for future compatibility with the Bedrock move into the monorepo
- Moves Go packages to the root of the repo in order to fix import paths
- Rewrites existing Go import paths
- Removes Stackman, since it's not needed anymore
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
