tornadocontrib/infra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
290 Commits 2 Branches 0 Tags 1.1 MiB
c82a9a08de
Commit Graph

173 Commits

Author SHA1 Message Date
Felipe Andrade
83c40076b6 typo 2023-05-26 14:25:56 -07:00
Felipe Andrade
2d9259ee20 better moar tests 2023-05-26 14:22:50 -07:00
Felipe Andrade
af863d39de consensus for {safe,finalized} and rewrite tags 2023-05-25 22:12:27 -07:00
felipe andrade
d5a476c5e8 fix(proxyd): use go-ethereum 1.12.0 parser for rpc.BlockNumberOrHash (#5791) 
* bump go-ethereum to 1.12.0 to support finalized and safe block tags

* fix index out of range error for malformed requests

* goimports
 2023-05-25 22:02:19 -07:00
Matthew Slipper
61db300042 proxyd: Pass through finalized and safe tags 2023-05-25 13:06:43 -06:00
felipe andrade
56938b6b01 normalize latency in ms (#5783) 2023-05-25 11:04:14 -07:00
felipe andrade
1d9406daa6 record sliding window metrics (#5782) 2023-05-25 09:16:16 -07:00
Felipe Andrade
10b8b8e32a fix(proxyd): avoid caching tx hash based methods, since the response may change with a reorg 2023-05-19 16:13:03 -07:00
Felipe Andrade
5616f40a16 also remove from config files 2023-05-18 16:30:33 -07:00
Felipe Andrade
7950eb9d7d fix(proxyd): clean up cache initialization 2023-05-18 16:27:23 -07:00
mergify[bot]
565073df19 Merge branch 'develop' into felipe/bg-shutdown 2023-05-18 05:12:58 +00:00
Felipe Andrade
d4956ee655 rename backend group receivers to be bg instead of just b to avoid misreading as a backend 2023-05-17 21:57:42 -07:00
Felipe Andrade
4c08681043 move consensus shutdown nested inside the backend group 2023-05-17 21:56:55 -07:00
mergify[bot]
991dc2627d Merge branch 'develop' into felipe/new-cache 2023-05-18 04:53:58 +00:00
Felipe Andrade
36e8e7a473 remove unused integration_tests/testdata/backend_rate_limit.toml 2023-05-17 21:43:25 -07:00
Felipe Andrade
3704119064 configs 2023-05-17 21:43:25 -07:00
Felipe Andrade
2f0f949784 refactor(proxyd): clean up backend rate limiter 2023-05-17 21:43:24 -07:00
Felipe Andrade
8aadf7abb1 replace base64 encoding for params with sha256 hash 2023-05-17 21:38:29 -07:00
mergify[bot]
64334e75df Merge branch 'develop' into felipe/fix-consensus-shutdown 2023-05-18 04:26:41 +00:00
mergify[bot]
3fe6123648 Merge branch 'develop' into felipe/event-dispatcher 2023-05-18 04:23:04 +00:00
mergify[bot]
4abf351490 Merge branch 'develop' into felipe/redis-namespace 2023-05-18 04:19:40 +00:00
Felipe Andrade
2bf0517c3f signature should be base64 2023-05-15 20:50:28 -07:00
Felipe Andrade
9757609d61 nits 2023-05-15 20:25:38 -07:00
Felipe Andrade
7db2391624 fix metric copypasta 2023-05-15 19:38:09 -07:00
Felipe Andrade
af481985fa lint 2023-05-15 19:30:03 -07:00
Felipe Andrade
490624b25e refactor(proxy): cache only immutable and hash-based methods 2023-05-15 19:20:08 -07:00
Felipe Andrade
29f7aa88ab feat(proxyd): redis namespace 2023-05-13 22:33:09 -07:00
Felipe Andrade
bae755b825 fix(proxyd): add missing shutdown for consensus poller 2023-05-13 22:19:32 -07:00
Felipe Andrade
ef42dde6e1 revert cache.go 2023-05-11 15:45:33 -07:00
Felipe Andrade
491369d32f lint 2023-05-11 15:44:12 -07:00
Felipe Andrade
e025bcc4b4 proxyd: emit event on consensus broken 2023-05-11 15:41:44 -07:00
Felipe Andrade
5c4b805efc ban logic 2023-05-11 15:33:47 -07:00
Felipe Andrade
8e45b5d5ca feat(proxyd): prevent banning out-of-sync backend 2023-05-11 15:19:52 -07:00
mergify[bot]
fdfbd9ce49 Merge branch 'develop' into felipe/moar-consensus-metrics 2023-05-10 19:36:37 +00:00
Felipe Andrade
88a172f739 skip reporting peer count according to config 2023-05-09 19:31:49 -07:00
Felipe Andrade
5f61935bc4 add filtered and total counts 2023-05-09 19:17:25 -07:00
Felipe Andrade
651b526c50 lint 2023-05-09 19:11:29 -07:00
Felipe Andrade
cfb26e6a8a convert update delay to ms 2023-05-09 17:50:25 -07:00
Felipe Andrade
cde638b11d moar consensus metrics 2023-05-09 17:21:25 -07:00
Felipe Andrade
53e4a369ab test edge cases 2023-05-09 14:20:23 -07:00
Felipe Andrade
c7caf2af4d rebase 2023-05-09 14:17:29 -07:00
Felipe Andrade
8d51c20079 use a global lag comparison, non order dependent 2023-05-09 14:14:19 -07:00
Felipe Andrade
3fd2abea84 test on edge 2023-05-09 14:13:38 -07:00
Felipe Andrade
2856a2c9e1 proxyd: add limit to consensus block lag 2023-05-09 14:13:35 -07:00
mergify[bot]
96ebb5a1c4 Merge branch 'develop' into felipe/fix-cache-for-eth2-block-tags 2023-05-09 19:55:22 +00:00
mergify[bot]
5b635ce32d Merge branch 'develop' into felipe/fix-bs-access 2023-05-09 19:52:11 +00:00
Felipe Andrade
020bc66243 fix test 2023-05-09 11:33:38 -07:00
Felipe Andrade
b95f2a6a62 make tag earliest cacheable 2023-05-09 09:39:11 -07:00
Felipe Andrade
cc9f6022de proxyd/fix: eth2 block tags {safe, finalized} should be valid tag values and avoid cache 2023-05-09 09:39:11 -07:00
Felipe Andrade
d4b2693ffd ban / isbanned 2023-05-09 03:58:35 -07:00
Felipe Andrade
32064105c8 prevent direct access to backend state struct 2023-05-09 02:59:32 -07:00
Felipe Andrade
92b54553c8 proxyd/fix: use correct context for auth 2023-05-08 19:19:46 -07:00
mergify[bot]
f5a8f900d6 Merge branch 'develop' into felipe/log-level 2023-05-04 23:55:14 +00:00
Felipe Andrade
5f4ead67a7 adjust log level 2023-05-04 16:40:11 -07:00
mergify[bot]
3e9d5468a8 Merge branch 'develop' into felipe/skip-peer-count 2023-05-04 23:36:30 +00:00
Felipe Andrade
75dcfd9a7a proxyd/config: skip peer count check 2023-05-04 16:10:45 -07:00
Felipe Andrade
1d3970995e goimports 2023-05-04 15:36:37 -07:00
Felipe Andrade
538492a0e7 rewrite should support blockorhash 2023-05-04 15:33:08 -07:00
felipe-op
bb4a592d77 proxyd/fix: error rate tolerance (#5606) 2023-05-04 09:53:11 -07:00
mergify[bot]
c9e2f5b28f Merge branch 'develop' into felipe/configs 2023-05-04 15:22:38 +00:00
felipe-op
7c9dd519e0 proxyd: re-write block tags to enforce consensus (#5586) 2023-05-04 08:21:48 -07:00
Felipe Andrade
d752c9bf9e externalize configs 2023-05-03 16:23:43 -07:00
Felipe Andrade
c2ba3b1c55 address comments 2023-04-28 11:51:05 -07:00
Felipe Andrade
58e3a35677 proxyd: round-robin lb for consensus group 2023-04-27 17:22:54 -07:00
Felipe Andrade
b7f559c4c2 fix http code for unhealthy backends 2023-04-27 12:10:30 -07:00
Felipe Andrade
f447404608 sliding window thread safe 2023-04-26 13:07:01 -07:00
Felipe Andrade
26f7d10e16 proxyd: integrate health checks 2023-04-26 12:51:42 -07:00
Felipe Andrade
a61c8f3f2a sliding window pkg 2023-04-24 13:35:31 -07:00
Felipe Andrade
1686d06e5a addressing final comments 2023-04-21 10:41:28 -07:00
Felipe Andrade
05536fa181 proxyd: add consensus poller 2023-04-20 16:27:36 -07:00
dependabot[bot]
5f6df9ac5e build(deps): bump golang.org/x/crypto in /proxyd 
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20220307211146-efcb8507fb70 to 0.1.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/commits/v0.1.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-27 19:11:28 +00:00
dependabot[bot]
9aa85e10fb build(deps): bump golang.org/x/sys in /proxyd 
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.0.0-20220310020820-b874c991c1a5 to 0.1.0.
- [Release notes](https://github.com/golang/sys/releases)
- [Commits](https://github.com/golang/sys/commits/v0.1.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-25 11:41:45 +00:00
Matthew Slipper
d4655c6cb9 ci: Remove proxyd and indexer from changesets workflow 2023-02-22 18:57:32 -07:00
github-actions[bot]
efe1879957 Version Packages 2023-02-23 01:53:04 +00:00
dependabot[bot]
2fc790aeda chore(deps): bump github.com/prometheus/client_golang in /proxyd 
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.11.0 to 1.11.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.11.0...v1.11.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-15 20:01:27 +00:00
Matthew Slipper
2427bfeadf Merge pull request #4897 from ethereum-optimism/feat/remove-logs 
proxyd: Remove logging
 2023-02-15 10:11:05 -07:00
Matthew Slipper
336c3a7c54 proxyd: Remove logging 2023-02-15 10:07:44 -07:00
github-actions[bot]
4037ef8b28 Version Packages 2023-02-15 16:36:28 +00:00
Matthew Slipper
c17bcc9b83 proxyd: Add global flag to method overrides 
This allows us to set global limits on individual RPCs that ignore any origin/user agent exemption.
 2023-02-15 00:44:19 -07:00
Matthew Slipper
454bc10e44 Merge pull request #4797 from ethereum-optimism/bugfix/include-nonce-in-rate-limit 
proxyd: Add nonce to sender-based rate limit
 2023-01-27 12:42:30 -07:00
Matthew Slipper
d7c80a5a8c proxyd: Add nonce to sender-based rate limit 2023-01-27 12:36:20 -07:00
inphi
2a87efb68a proxyd: Add missing req_id to log entry 2023-01-27 13:23:20 -05:00
github-actions[bot]
868d622e4f Version Packages 2023-01-23 19:35:31 +00:00
Matthew Slipper
aa3a5c859a proxyd: Add sender-based rate limiter 
This PR adds a sender-based rate limiter to proxyd. This rate limiter throttles eth_sendRawTransaction requests based on sender in order to prevent abusive behavior on the sequencer.

Integration tests have been added to test this functionality.
 2023-01-23 10:59:40 -07:00
github-actions[bot]
f1b582c385 Version Packages 2022-10-17 23:21:20 +00:00
mergify[bot]
019f800e90 Merge branch 'develop' into feat/proxyd-allow-backend-lim-disable 2022-10-17 15:53:44 +00:00
Matthew Slipper
fe8a89e61d proxyd: Support pattern matching in origin and user agent 
Adds support for pattern matchin in exempt origins. This should help with some of the issues Synthetix and others are seeing.
 2022-10-17 07:08:33 -06:00
Matthew Slipper
fd5e974bd5 fix tests 2022-10-15 09:59:11 -05:00
Zach Howard
24a3668bc8 adds log level conf to proxyd (#3704) 2022-10-14 13:56:27 -05:00
Matthew Slipper
8a624bb2ca proxyd: Allow disabling backend rate limiting 
The backend rate limiter is in place to protect upstreams like the sequencer. However, in many cases it isn't needed and it causes unnecessary requests to Redis. This PR allows this to be disabled, and disables this by default.
 2022-10-13 15:02:35 -05:00
github-actions[bot]
053ce4ddfd Version Packages 2022-10-13 15:32:03 +00:00
Matthew Slipper
3f19d26ea8 Merge pull request #3681 from ethereum-optimism/10-09-proxyd_Integrate_custom_rate_limiter 
proxyd: Integrate custom rate limiter
 2022-10-12 15:34:14 -05:00
Matthew Slipper
7fd63c6c6c Merge pull request #3679 from ethereum-optimism/10-09-proxyd_Custom_rate_limiter_implementation 
proxyd: Custom rate limiter implementation
 2022-10-12 15:34:02 -05:00
Matthew Slipper
f737002baa proxyd: Integrate custom rate limiter 
Integrates the custom rate limiter in the previous PR into the rest of the application. Also takes the opportunity to clean up how we instantiate Redis clients so that we can share them among multiple different services.

There are some config changes in this PR. Specifically, you must specify a `base_rate` and `base_interval` in the rate limit config.
 2022-10-11 17:20:23 -05:00
Matthew Slipper
fa7425683a proxyd: Custom rate limiter implementation 
Our current proxyd deployment does not share rate limit state across multiple servers within a backend group. This means that rate limits on the public endpoint are artifically high.

This PR adds a Redis-based rate limiter to fix this problem. While our current rate limiting library (github.com/sethvargo/go-limiter) _does_ support Redis, the client library it uses is not type safe, is less performant, and would require us to update the other places we use Redis. To avoid these issues, I created a simple rate limiting interface with both Redis and memory backend.

Note that this PR only adds the new implementations - it does not integrate them with the rest of the codebase. I'll do that in a separate PR to make review easier.
 2022-10-11 17:20:23 -05:00
Chris Wessels
15a59bad99 fix(proxyd): Fix compliance with JSON-RPC 2.0 spec by adding optional RPCError.Data (#3683) 
* fix: add optional data field to RPCError struct

* fix: formatting lint

* feat(proxyd): add changeset
 2022-10-11 08:51:02 -06:00
github-actions[bot]
09552fddea Version Packages 2022-09-29 01:37:00 +00:00
Matthew Slipper
537717610e proxyd: Add batch size metric and configurable max (#3545) 
* proxyd: Add batch size metric and configurable max

The max batch size will be overwritten if it is over `MaxBatchRPCCallsHardLimit`.

Builds on https://github.com/ethereum-optimism/optimism/pull/3544.

* changeset

* fix lint

* fix test
 2022-09-23 15:06:02 -06:00
Matthew Slipper
485258d3c2 proxyd: Add customizable whitelist error message (#3544) 
* proxyd: Add customizable whitelist error message

Alchemy is asking for this so that we can include their affiliate link.

* add missing test case

* fix error message

* goimports
 2022-09-23 14:21:12 -06:00
github-actions[bot]
70557fdafd Version Packages 2022-09-20 15:21:51 +00:00