mergify[bot]
64334e75df
Merge branch 'develop' into felipe/fix-consensus-shutdown
2023-05-18 04:26:41 +00:00
mergify[bot]
3fe6123648
Merge branch 'develop' into felipe/event-dispatcher
2023-05-18 04:23:04 +00:00
mergify[bot]
4abf351490
Merge branch 'develop' into felipe/redis-namespace
2023-05-18 04:19:40 +00:00
Felipe Andrade
2bf0517c3f
signature should be base64
2023-05-15 20:50:28 -07:00
Felipe Andrade
9757609d61
nits
2023-05-15 20:25:38 -07:00
Felipe Andrade
7db2391624
fix metric copypasta
2023-05-15 19:38:09 -07:00
Felipe Andrade
af481985fa
lint
2023-05-15 19:30:03 -07:00
Felipe Andrade
490624b25e
refactor(proxy): cache only immutable and hash-based methods
2023-05-15 19:20:08 -07:00
Felipe Andrade
29f7aa88ab
feat(proxyd): redis namespace
2023-05-13 22:33:09 -07:00
Felipe Andrade
bae755b825
fix(proxyd): add missing shutdown for consensus poller
2023-05-13 22:19:32 -07:00
Felipe Andrade
ef42dde6e1
revert cache.go
2023-05-11 15:45:33 -07:00
Felipe Andrade
491369d32f
lint
2023-05-11 15:44:12 -07:00
Felipe Andrade
e025bcc4b4
proxyd: emit event on consensus broken
2023-05-11 15:41:44 -07:00
Felipe Andrade
5c4b805efc
ban logic
2023-05-11 15:33:47 -07:00
Felipe Andrade
8e45b5d5ca
feat(proxyd): prevent banning out-of-sync backend
2023-05-11 15:19:52 -07:00
mergify[bot]
fdfbd9ce49
Merge branch 'develop' into felipe/moar-consensus-metrics
2023-05-10 19:36:37 +00:00
Felipe Andrade
88a172f739
skip reporting peer count according to config
2023-05-09 19:31:49 -07:00
Felipe Andrade
5f61935bc4
add filtered and total counts
2023-05-09 19:17:25 -07:00
Felipe Andrade
651b526c50
lint
2023-05-09 19:11:29 -07:00
Felipe Andrade
cfb26e6a8a
convert update delay to ms
2023-05-09 17:50:25 -07:00
Felipe Andrade
cde638b11d
moar consensus metrics
2023-05-09 17:21:25 -07:00
Felipe Andrade
53e4a369ab
test edge cases
2023-05-09 14:20:23 -07:00
Felipe Andrade
c7caf2af4d
rebase
2023-05-09 14:17:29 -07:00
Felipe Andrade
8d51c20079
use a global lag comparison, non order dependent
2023-05-09 14:14:19 -07:00
Felipe Andrade
3fd2abea84
test on edge
2023-05-09 14:13:38 -07:00
Felipe Andrade
2856a2c9e1
proxyd: add limit to consensus block lag
2023-05-09 14:13:35 -07:00
mergify[bot]
96ebb5a1c4
Merge branch 'develop' into felipe/fix-cache-for-eth2-block-tags
2023-05-09 19:55:22 +00:00
mergify[bot]
5b635ce32d
Merge branch 'develop' into felipe/fix-bs-access
2023-05-09 19:52:11 +00:00
Felipe Andrade
020bc66243
fix test
2023-05-09 11:33:38 -07:00
Felipe Andrade
b95f2a6a62
make tag earliest cacheable
2023-05-09 09:39:11 -07:00
Felipe Andrade
cc9f6022de
proxyd/fix: eth2 block tags {safe, finalized} should be valid tag values and avoid cache
2023-05-09 09:39:11 -07:00
Felipe Andrade
d4b2693ffd
ban / isbanned
2023-05-09 03:58:35 -07:00
Felipe Andrade
32064105c8
prevent direct access to backend state struct
2023-05-09 02:59:32 -07:00
Felipe Andrade
92b54553c8
proxyd/fix: use correct context for auth
2023-05-08 19:19:46 -07:00
mergify[bot]
f5a8f900d6
Merge branch 'develop' into felipe/log-level
2023-05-04 23:55:14 +00:00
Felipe Andrade
5f4ead67a7
adjust log level
2023-05-04 16:40:11 -07:00
mergify[bot]
3e9d5468a8
Merge branch 'develop' into felipe/skip-peer-count
2023-05-04 23:36:30 +00:00
Felipe Andrade
75dcfd9a7a
proxyd/config: skip peer count check
2023-05-04 16:10:45 -07:00
Felipe Andrade
1d3970995e
goimports
2023-05-04 15:36:37 -07:00
Felipe Andrade
538492a0e7
rewrite should support blockorhash
2023-05-04 15:33:08 -07:00
felipe-op
bb4a592d77
proxyd/fix: error rate tolerance ( #5606 )
2023-05-04 09:53:11 -07:00
mergify[bot]
c9e2f5b28f
Merge branch 'develop' into felipe/configs
2023-05-04 15:22:38 +00:00
felipe-op
7c9dd519e0
proxyd: re-write block tags to enforce consensus ( #5586 )
2023-05-04 08:21:48 -07:00
Felipe Andrade
d752c9bf9e
externalize configs
2023-05-03 16:23:43 -07:00
Felipe Andrade
c2ba3b1c55
address comments
2023-04-28 11:51:05 -07:00
Felipe Andrade
58e3a35677
proxyd: round-robin lb for consensus group
2023-04-27 17:22:54 -07:00
Felipe Andrade
b7f559c4c2
fix http code for unhealthy backends
2023-04-27 12:10:30 -07:00
Felipe Andrade
f447404608
sliding window thread safe
2023-04-26 13:07:01 -07:00
Felipe Andrade
26f7d10e16
proxyd: integrate health checks
2023-04-26 12:51:42 -07:00
Felipe Andrade
a61c8f3f2a
sliding window pkg
2023-04-24 13:35:31 -07:00
Felipe Andrade
1686d06e5a
addressing final comments
2023-04-21 10:41:28 -07:00
Felipe Andrade
05536fa181
proxyd: add consensus poller
2023-04-20 16:27:36 -07:00
dependabot[bot]
5f6df9ac5e
build(deps): bump golang.org/x/crypto in /proxyd
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.0.0-20220307211146-efcb8507fb70 to 0.1.0.
- [Release notes](https://github.com/golang/crypto/releases )
- [Commits](https://github.com/golang/crypto/commits/v0.1.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-27 19:11:28 +00:00
dependabot[bot]
9aa85e10fb
build(deps): bump golang.org/x/sys in /proxyd
...
Bumps [golang.org/x/sys](https://github.com/golang/sys ) from 0.0.0-20220310020820-b874c991c1a5 to 0.1.0.
- [Release notes](https://github.com/golang/sys/releases )
- [Commits](https://github.com/golang/sys/commits/v0.1.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sys
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-25 11:41:45 +00:00
Matthew Slipper
d4655c6cb9
ci: Remove proxyd and indexer from changesets workflow
2023-02-22 18:57:32 -07:00
github-actions[bot]
efe1879957
Version Packages
2023-02-23 01:53:04 +00:00
dependabot[bot]
2fc790aeda
chore(deps): bump github.com/prometheus/client_golang in /proxyd
...
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang ) from 1.11.0 to 1.11.1.
- [Release notes](https://github.com/prometheus/client_golang/releases )
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md )
- [Commits](https://github.com/prometheus/client_golang/compare/v1.11.0...v1.11.1 )
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-15 20:01:27 +00:00
Matthew Slipper
2427bfeadf
Merge pull request #4897 from ethereum-optimism/feat/remove-logs
...
proxyd: Remove logging
2023-02-15 10:11:05 -07:00
Matthew Slipper
336c3a7c54
proxyd: Remove logging
2023-02-15 10:07:44 -07:00
github-actions[bot]
4037ef8b28
Version Packages
2023-02-15 16:36:28 +00:00
Matthew Slipper
c17bcc9b83
proxyd: Add global flag to method overrides
...
This allows us to set global limits on individual RPCs that ignore any origin/user agent exemption.
2023-02-15 00:44:19 -07:00
Matthew Slipper
454bc10e44
Merge pull request #4797 from ethereum-optimism/bugfix/include-nonce-in-rate-limit
...
proxyd: Add nonce to sender-based rate limit
2023-01-27 12:42:30 -07:00
Matthew Slipper
d7c80a5a8c
proxyd: Add nonce to sender-based rate limit
2023-01-27 12:36:20 -07:00
inphi
2a87efb68a
proxyd: Add missing req_id to log entry
2023-01-27 13:23:20 -05:00
github-actions[bot]
868d622e4f
Version Packages
2023-01-23 19:35:31 +00:00
Matthew Slipper
aa3a5c859a
proxyd: Add sender-based rate limiter
...
This PR adds a sender-based rate limiter to proxyd. This rate limiter throttles eth_sendRawTransaction requests based on sender in order to prevent abusive behavior on the sequencer.
Integration tests have been added to test this functionality.
2023-01-23 10:59:40 -07:00
github-actions[bot]
f1b582c385
Version Packages
2022-10-17 23:21:20 +00:00
mergify[bot]
019f800e90
Merge branch 'develop' into feat/proxyd-allow-backend-lim-disable
2022-10-17 15:53:44 +00:00
Matthew Slipper
fe8a89e61d
proxyd: Support pattern matching in origin and user agent
...
Adds support for pattern matchin in exempt origins. This should help with some of the issues Synthetix and others are seeing.
2022-10-17 07:08:33 -06:00
Matthew Slipper
fd5e974bd5
fix tests
2022-10-15 09:59:11 -05:00
Zach Howard
24a3668bc8
adds log level conf to proxyd ( #3704 )
2022-10-14 13:56:27 -05:00
Matthew Slipper
8a624bb2ca
proxyd: Allow disabling backend rate limiting
...
The backend rate limiter is in place to protect upstreams like the sequencer. However, in many cases it isn't needed and it causes unnecessary requests to Redis. This PR allows this to be disabled, and disables this by default.
2022-10-13 15:02:35 -05:00
github-actions[bot]
053ce4ddfd
Version Packages
2022-10-13 15:32:03 +00:00
Matthew Slipper
3f19d26ea8
Merge pull request #3681 from ethereum-optimism/10-09-proxyd_Integrate_custom_rate_limiter
...
proxyd: Integrate custom rate limiter
2022-10-12 15:34:14 -05:00
Matthew Slipper
7fd63c6c6c
Merge pull request #3679 from ethereum-optimism/10-09-proxyd_Custom_rate_limiter_implementation
...
proxyd: Custom rate limiter implementation
2022-10-12 15:34:02 -05:00
Matthew Slipper
f737002baa
proxyd: Integrate custom rate limiter
...
Integrates the custom rate limiter in the previous PR into the rest of the application. Also takes the opportunity to clean up how we instantiate Redis clients so that we can share them among multiple different services.
There are some config changes in this PR. Specifically, you must specify a `base_rate` and `base_interval` in the rate limit config.
2022-10-11 17:20:23 -05:00
Matthew Slipper
fa7425683a
proxyd: Custom rate limiter implementation
...
Our current proxyd deployment does not share rate limit state across multiple servers within a backend group. This means that rate limits on the public endpoint are artifically high.
This PR adds a Redis-based rate limiter to fix this problem. While our current rate limiting library (github.com/sethvargo/go-limiter) _does_ support Redis, the client library it uses is not type safe, is less performant, and would require us to update the other places we use Redis. To avoid these issues, I created a simple rate limiting interface with both Redis and memory backend.
Note that this PR only adds the new implementations - it does not integrate them with the rest of the codebase. I'll do that in a separate PR to make review easier.
2022-10-11 17:20:23 -05:00
Chris Wessels
15a59bad99
fix(proxyd): Fix compliance with JSON-RPC 2.0 spec by adding optional RPCError.Data ( #3683 )
...
* fix: add optional data field to RPCError struct
* fix: formatting lint
* feat(proxyd): add changeset
2022-10-11 08:51:02 -06:00
github-actions[bot]
09552fddea
Version Packages
2022-09-29 01:37:00 +00:00
Matthew Slipper
537717610e
proxyd: Add batch size metric and configurable max ( #3545 )
...
* proxyd: Add batch size metric and configurable max
The max batch size will be overwritten if it is over `MaxBatchRPCCallsHardLimit`.
Builds on https://github.com/ethereum-optimism/optimism/pull/3544 .
* changeset
* fix lint
* fix test
2022-09-23 15:06:02 -06:00
Matthew Slipper
485258d3c2
proxyd: Add customizable whitelist error message ( #3544 )
...
* proxyd: Add customizable whitelist error message
Alchemy is asking for this so that we can include their affiliate link.
* add missing test case
* fix error message
* goimports
2022-09-23 14:21:12 -06:00
github-actions[bot]
70557fdafd
Version Packages
2022-09-20 15:21:51 +00:00
Matthew Slipper
d228a55bd7
proxyd: Fix error message in tests ( #3503 )
...
* proxyd: Fix error message in tests
The custom error message wasn't being carried over in batch responses.
* review fixes
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-19 16:44:08 +00:00
Matthew Slipper
3fb4debfe1
proxyd: Add logging for origin and remote IP ( #3482 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com>
2022-09-16 17:28:56 +00:00
github-actions[bot]
320d6ebeea
Version Packages
2022-09-15 08:51:21 +00:00
Matthew Slipper
ccf0934459
proxyd: Support per-RPC rate limits ( #3471 )
...
* proxyd: Support per-RPC rate limits
* add log
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-15 08:46:11 +00:00
Joshua Gutow
8e92c91e6d
Last unwrapped errors
...
These where missed in local lints but when a full lint was run these
popped up.
2022-09-09 04:56:59 -07:00
protolambda
9a4626856a
style(batch-submitter,bss-core,proxyd): Fix lint Go ( #3328 )
...
* style(batch-submitter): fix lint
* style(bss-core): fix lint
* chore(proxyd): use io and os instead of deprecated ioutil methods, fixes lint
2022-08-26 19:39:30 -06:00
github-actions[bot]
fc9487bf83
Version Packages
2022-08-04 18:48:19 +00:00
Matthew Slipper
7a2b4bdf45
proxyd: Add rate limit logging ( #3173 )
2022-08-04 18:44:46 +00:00
github-actions[bot]
7c599f758f
Version Packages
2022-08-04 17:40:22 +00:00
Matthew Slipper
f3d3492a81
proxyd: Add frontend rate limiting ( #3166 )
...
* proxyd: Add frontend rate limiting
To give us more flexibiltiy with rate limiting, proxyd now supports rate limiting of client (frontend) requests in addition to upstream (backend) requests. This PR also gives us the ability to exempt certain user agents/origins from rate limiting.
* lint
2022-08-04 11:34:43 -06:00
Matthew Slipper
4ea6a054c3
proxyd: Unwrap single RPC batches ( #3165 )
...
* proxyd: Unwrap single RPC batches
* Update backend.go
2022-08-04 01:36:07 +00:00
Matthew Slipper
cd0afa3176
proxyd: Parameterize full RPC request logging ( #3110 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-07-27 17:12:47 +00:00
Matthew Slipper
614366aa5b
ci: Upgrade golangci-lint version to one that handles Go 1.18 ( #2841 )
...
* ci: Upgrade golangci-lint version to one that handles Go 1.18
* Fix proxyd lint
2022-06-23 04:45:26 +00:00
github-actions[bot]
5da617a5ba
Version Packages
2022-06-18 02:48:41 +00:00
Matthew Slipper
e2e3a622ce
proxyd: Use canned response for eth_accounts ( #2801 )
...
We never want to expose Geth's accounts to the public internet, so proxyd will now return `[]` for `eth_accounts` RPC calls without hitting the backend.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-16 19:02:39 +00:00
github-actions[bot]
c31d7501ea
Version Packages
2022-06-08 20:51:10 +00:00
Matthew Slipper
e41cfc1d94
proxyd: Fix concurrent WS write panic ( #2711 )
...
Fixes a panic in the websocket proxyd logic. Normally, the `clientPump` and `backendPump` methods in `WSProxier` send data in one direction. However, when the client sends an invalid RPC, the `clientPump` will send a response _directly to the client_ in order to avoid unnecessary roundtrips to the backend. This could be interleaved with concurrent writes to the client's WS in `backendPump`, and would cause a panic in the WS library.
To test this, this PR includes a dedicated integration test that reliably triggers the issue. In addition, this PR adds additional testing for WS functionality.
2022-06-08 15:09:32 +00:00
Murphy Law
69f189c0ea
proxyd: Handle unexpected JSON-RPC responses ( #2628 )
...
This fixes a bug where the infura backend would be labeled offline because it
returns an unexpected JSON-RPC response. Unexpected, but well-formed,
JSON-RPC response are handled specially. Such errors are surfaced up to
the backend proxier so failover still occurs.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-08 13:56:24 +00:00