Bryan Stitt
36cc884112
Merge remote-tracking branch 'origin/main' into devel
2023-05-11 20:05:47 -07:00
Bryan Stitt
fc8ca4ba4f
less max lag
2023-05-11 17:04:33 -07:00
Bryan Stitt
6e8536d477
add support for http basic auth
2023-05-11 13:21:12 -07:00
Rory Trent
ec11e210ee
Peak ewma ( #63 )
...
* use peak-ewma instead of head for latency calculation
* Implement some suggested changes from PR
* move latency to new package in workspace root
* fix unit tests which now require peak_latency on Web3Rpc
* Switch to atomics for peak-ewma
This change is to avoid locking from tokio::sync::watch.
* add decay calculation to latency reads in peak-ewma
* Add some tests for peak-ewma
* Sensible latency defaults and not blocking on full
* Cleanup and a couple additional comments
2023-05-11 13:09:15 -07:00
Bryan Stitt
96a9a39789
add support for http basic auth
2023-05-11 13:07:38 -07:00
Bryan Stitt
0531e2f8dd
add 'quota usage' rate limit substring
2023-05-08 11:25:27 -06:00
Bryan Stitt
ffb554d0b3
online editor isn't actually nice when it misses syntax errors :(
2023-04-29 18:50:26 -07:00
Bryan Stitt
91acbcaf75
Fix sort order
2023-04-29 18:38:31 -07:00
Bryan Stitt
1fab1b1d46
oops. lower this log level
2023-04-24 11:21:53 -07:00
Bryan Stitt
56fdf48129
Send transactions to protected and public rpcs ( #57 )
...
* move protected transactions into their own function and dry stats sending
* cargo upgrade
* comments
* time to live instead of time to idle
* minor workaround for eth_chainId
* cargo upgrade
2023-04-24 11:00:12 -07:00
Bryan Stitt
7a087415a2
hash instead of block_hash
2023-04-20 19:55:18 -07:00
Bryan Stitt
4889c3e1ce
Merge branch 'main' into devel
2023-04-18 20:38:43 -07:00
Bryan Stitt
e921d02eb2
remove stale todo
2023-04-17 17:10:09 -07:00
Bryan Stitt
5ff28943da
fix incorrect context
2023-04-14 03:44:32 -07:00
Bryan Stitt
2652f88f52
attach more contexts
2023-04-14 03:43:49 -07:00
Bryan Stitt
c2710858e1
only warn if rpc_configs is too short
2023-04-14 00:41:51 -07:00
Bryan Stitt
8ed71e1cf1
more goerli fixes
2023-04-14 00:36:46 -07:00
Bryan Stitt
3621d71037
if not watching heads, send to any server
2023-04-14 00:15:27 -07:00
Bryan Stitt
d035049c8f
add /backups_needed endpoint for easy alerts
2023-04-13 17:15:01 -07:00
Bryan Stitt
b1120e635a
change some error codes jsonrpc error handling
2023-04-10 22:29:02 -07:00
Bryan Stitt
b5ed0c4710
sometimes watch_for_block instead of returning an error
2023-04-10 22:28:31 -07:00
Bryan Stitt
ed5ca0575c
logging improvements
2023-04-10 16:05:53 -07:00
Bryan Stitt
29c015508a
fix warnings in cargo doc
2023-04-10 15:06:40 -07:00
Bryan Stitt
7250a0c155
upgrade ethers to 2.0.2
2023-04-10 15:01:36 -07:00
Bryan Stitt
b460f1701a
rename revert_handler to error_handler
2023-04-05 15:34:28 -07:00
Bryan Stitt
7994c894bb
cargo clippy and fmt
2023-04-05 15:15:20 -07:00
Bryan Stitt
1a160a8c7d
Merge remote-tracking branch 'rorytrent/Web3ProxyError' into devel
2023-04-05 15:08:57 -07:00
yenicelik
0069e76040
latest changes from bryan
2023-04-04 14:40:22 +02:00
Rory Neithinger
f0d8a0c8c9
Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError
2023-04-01 00:04:44 -07:00
Rory Neithinger
ffdf25787f
Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError
2023-03-31 23:23:30 -07:00
yenicelik
0bb3a2dc06
simple changes around bryans comments
2023-03-31 12:43:41 +01:00
yenicelik
345fa6c513
git status
2023-03-30 13:42:56 +01:00
yenicelik
53c7541fed
seems to wait until everything is flushed. I will have to debug some stuff that isnt saved properly in the mysql (and then check influx)
2023-03-25 17:56:45 +01:00
Bryan Stitt
2205a472fd
add optional tokio-console and improve reconnect logic
2023-03-22 18:43:13 -07:00
Bryan Stitt
7b223efa4d
improve reconnect logic
2023-03-22 16:16:15 -07:00
Bryan Stitt
b4ca0fb62d
better log when servers skipped
2023-03-22 15:19:09 -07:00
Bryan Stitt
a69737db98
serialize best_rpcs
2023-03-22 14:23:14 -07:00
Bryan Stitt
8eff48611c
more direct consensus finding code
...
this hopefully has less bugs. speed isn't super important since this isn't on the host path.
2023-03-22 13:10:15 -07:00
Bryan Stitt
9fae137e45
shorten log message
2023-03-22 02:06:26 -07:00
Bryan Stitt
5465d927b4
remove a debug log
2023-03-21 21:22:16 -07:00
Bryan Stitt
d992bb2dc9
skip, don't error
2023-03-21 16:00:34 -07:00
Rory Neithinger
fe2a760c8a
more conversions to Web3ProxyError
2023-03-20 13:45:21 -07:00
Rory Neithinger
beac7ee017
better error handling for proxy_web3_rpc()
2023-03-19 18:52:28 -07:00
Rory Neithinger
c32d12b5e0
better error handling for ip_is_authorized()
2023-03-19 15:50:25 -07:00
Rory Neithinger
f3fc4924dc
Refactor FrontendErrorResponse into Web3ProxyError
...
Renamed FrontendResponse to Web3ProxyResponse and introduced
a new generic type alias Web3ProxyResult.
Fixed a few noisy cargo warnings.
2023-03-16 22:21:20 -07:00
Bryan Stitt
7b9bcf6881
include head block in the log
2023-03-09 11:22:40 -08:00
Bryan Stitt
df8cc3b954
fix log order
2023-03-07 16:16:32 -08:00
Bryan Stitt
6717999ced
actually use backup rpcs
2023-03-07 16:02:22 -08:00
Bryan Stitt
dbd7860416
wip influxdb2
2023-03-04 20:54:04 +01:00
Bryan Stitt
eb4d05a520
stats v2
...
rebased all my commits and squashed them down to one
2023-03-04 20:48:49 +01:00
Bryan Stitt
36e4183373
todos
2023-03-03 11:19:25 -08:00
Bryan Stitt
94089eb72b
combine comparison
2023-03-03 07:34:04 -08:00
Bryan Stitt
3b61e009f8
group cfgs
2023-03-02 17:51:41 -08:00
Bryan Stitt
3098791ad9
add optional kafka feature
2023-03-03 01:39:50 +00:00
Bryan Stitt
d1b955275d
clean up wait_for_ functions
2023-03-02 18:27:32 +00:00
Bryan Stitt
907a147afa
wait_for_provider helper function
2023-03-02 18:14:17 +00:00
Bryan Stitt
0d07e20d7c
make it compile
2023-03-02 17:38:42 +00:00
Bryan Stitt
258e889f69
wait for a provider
2023-03-02 16:52:28 +00:00
Bryan Stitt
499d50b5fc
handle empty private_rpcs
2023-03-01 23:21:09 +00:00
Bryan Stitt
c8bc0d288f
high log level
2023-03-01 23:04:20 +00:00
Bryan Stitt
fdba8e7864
test all_connections with Nones
2023-03-01 22:50:44 +00:00
Bryan Stitt
f68886fafd
add simple test for all connections and include more rpcs
2023-03-01 22:32:01 +00:00
Bryan Stitt
baf07f7de5
sync sort should include backup
2023-03-01 21:35:50 +00:00
Bryan Stitt
2e9803de72
remove arbitrary sleep timer
2023-03-01 20:56:00 +00:00
Bryan Stitt
b87c988439
lint
2023-03-01 11:23:59 -08:00
Bryan Stitt
20851af51b
log seconds instead of milliseconds
2023-02-28 11:09:49 -08:00
Bryan Stitt
d83a1f0169
use Web3Rpc instead of the name as a key
2023-02-28 11:01:34 -08:00
Bryan Stitt
26970c2d88
move wait on new connection
2023-02-27 13:40:13 -08:00
Bryan Stitt
c5e2402d6e
disconnect is noisy but it works
2023-02-27 13:29:07 -08:00
Bryan Stitt
6067369ee3
add inotify and rpc disconnect
2023-02-27 12:50:57 -08:00
Bryan Stitt
f8f5e7a1c8
add thread for config file watching and run clippy lint
2023-02-26 23:08:00 -08:00
Bryan Stitt
bf79d677b0
by_name needs to a lock
2023-02-26 20:00:13 -08:00
Bryan Stitt
e96f09a9c4
refactors to make configs partially reloadable
2023-02-25 23:52:33 -08:00
Bryan Stitt
c9e5661c5b
wip
2023-02-25 09:55:28 -08:00
Bryan Stitt
580965ef83
get instead of remove
2023-02-24 23:31:10 -08:00
Bryan Stitt
91b853d21e
archive instead of Some(u64::MAX) in the logs
2023-02-21 21:14:49 -08:00
Bryan Stitt
6c33b1ae1e
block number instead of hash during health check
2023-02-21 20:25:54 -08:00
Bryan Stitt
738815244d
change ewma calculation
2023-02-16 00:26:58 -08:00
Bryan Stitt
4916188d5c
theres a perf regression in here somewhere
2023-02-15 21:54:07 -08:00
Bryan Stitt
cb2b5c16ad
lower log level
2023-02-15 21:30:42 -08:00
Bryan Stitt
b7dfec9c63
fix params in health check query
2023-02-15 21:13:28 -08:00
Bryan Stitt
91cab8ffe2
comment
2023-02-15 21:09:39 -08:00
Bryan Stitt
bc306f62d4
make it work
2023-02-15 21:05:41 -08:00
Bryan Stitt
3c9576c13b
yes health, no channel
2023-02-15 15:52:42 -08:00
Bryan Stitt
63adbc0587
remove health for now?
2023-02-15 15:42:27 -08:00
Bryan Stitt
268ad3a7c5
dont hold a lock open
2023-02-15 15:37:43 -08:00
Bryan Stitt
056670d29a
health check and latency recordings
2023-02-15 15:31:59 -08:00
Bryan Stitt
774bd5b232
undo a rename in the json
2023-02-15 12:59:03 -08:00
Bryan Stitt
1fb4dd6ccc
i think it works
2023-02-15 12:33:43 -08:00
Bryan Stitt
df668a5dfd
add time to idle to more caches
2023-02-15 11:42:25 -08:00
Bryan Stitt
c65ffc9ce0
getting close
2023-02-14 17:41:40 -08:00
Bryan Stitt
f1e6de3677
fixed one test. still more to fix though
2023-02-14 12:41:05 -08:00
Bryan Stitt
824a6fa1f0
well it compiles. doesnt work yet. but it compiles
2023-02-14 12:14:50 -08:00
Bryan Stitt
70105bc7bb
variable rename
2023-02-12 10:22:20 -08:00
Bryan Stitt
0e2e45cf90
cargo upgrade
2023-02-12 01:47:06 -08:00
Bryan Stitt
c008f50943
use http and ws providers in a single config
2023-02-12 01:40:01 -08:00
Bryan Stitt
f1bc00082a
include to_block more places
2023-02-10 20:45:57 -08:00
Bryan Stitt
c959110986
better sorting of connections
2023-02-10 20:24:20 -08:00
Bryan Stitt
1a93e39b09
FuturesUnordered on the spawn handles
2023-02-09 11:56:07 -08:00
Bryan Stitt
6741cd72ac
lint
2023-02-09 11:55:54 -08:00
Bryan Stitt
5ffe2aa72a
eth_sendRawTransactions should use backups more often
2023-02-09 11:47:56 -08:00
Bryan Stitt
1efa8e7772
improve log message
...
we don't always get an empty block because we aren't synced. sometimes it is because we got disconnected
2023-02-08 11:30:16 -08:00
Bryan Stitt
0b1929a820
add minimal config for quick testing
2023-02-06 15:20:36 -08:00
Bryan Stitt
0edd11349f
web3 rpc spawn should be unordered
2023-02-06 14:13:43 -08:00
Bryan Stitt
19e51ce328
check http servers twice as often
2023-02-06 13:48:50 -08:00
Bryan Stitt
0f280ce483
cargo upgrade and shorten variable names
...
also begin adding a latency tracker for rpc stats
2023-02-06 09:55:27 -08:00
Bryan Stitt
f2d35ba5eb
remove metered in favor of influxdb stats
2023-02-05 18:17:26 -08:00
Bryan Stitt
6d959e2c1f
add archive depth to app config
2023-02-05 17:58:03 -08:00
Bryan Stitt
37830f1156
Jenkinsfile from llamanodes/erigon
2023-02-05 00:13:14 -08:00
Bryan Stitt
ca1e550370
improve sort order during eth_sendRawTransaction
2023-02-02 14:48:23 -08:00
Bryan Stitt
cac31303f3
include backup in web3connection serializer
2023-02-02 08:00:59 -08:00
Bryan Stitt
a76a45f63a
clone instead of take
2023-01-31 09:47:19 -08:00
Bryan Stitt
8bcf40b62d
Revert "Revert "include from_block and to_block for caching getLogs""
...
This reverts commit 13bab2c9cfcc09f8d503ff24f6e2c2a3fc0faaa2.
2023-01-31 08:30:24 -08:00
Bryan Stitt
13bab2c9cf
Revert "include from_block and to_block for caching getLogs"
...
This reverts commit 203969f62833668c4081acbb2188989fd6545c24.
2023-01-31 08:30:11 -08:00
Bryan Stitt
203969f628
include from_block and to_block for caching getLogs
2023-01-30 15:47:17 -08:00
Bryan Stitt
f2260ecdba
don't require hard limit send
2023-01-25 10:01:36 -08:00
Bryan Stitt
cffc60e7f6
improve responses when blocks are not available
2023-01-24 22:45:20 -08:00
Bryan Stitt
694e552b5d
improve waiting for sync when rate limited
2023-01-24 20:45:03 -08:00
Bryan Stitt
641d11a19b
remove excess continues
2023-01-24 10:09:12 -08:00
Bryan Stitt
106dec294f
better handling when method not available
2023-01-24 09:36:07 -08:00
Bryan Stitt
a242244a35
broadcast transactions to more servers
2023-01-24 01:58:31 -08:00
Bryan Stitt
47daab3b67
drop log level for backup servers
2023-01-23 21:37:23 -08:00
Bryan Stitt
86e3f2991f
use watch instead of arcswap
2023-01-22 22:02:08 -08:00
Bryan Stitt
b9a12756b9
collect request_metadata.response_from_backup_rpc
2023-01-19 21:46:47 -08:00
Bryan Stitt
e53030e053
move backup indicator
2023-01-19 18:30:30 -08:00
Bryan Stitt
c3b53eb5f2
add backup indicator to more logs
2023-01-19 18:14:47 -08:00
Bryan Stitt
ad169eda0e
log cleanup
2023-01-19 17:57:36 -08:00
Bryan Stitt
52a9ba604c
remove allowed lag
2023-01-19 03:05:39 -08:00
Bryan Stitt
2cb6dde052
more log improvements
2023-01-19 02:26:54 -08:00
Bryan Stitt
76e51e3d11
better log level
2023-01-19 02:21:39 -08:00
Bryan Stitt
0c05b5bdee
major refactor to only use backup servers when absolutely necessary
2023-01-19 02:20:06 -08:00
Bryan Stitt
90d3371eee
improved rate limiting on websockets
2023-01-18 16:17:47 -08:00
Bryan Stitt
d7c75f843e
add stub try_proxy_connection
2023-01-16 22:54:40 -08:00
Bryan Stitt
8c02e4420d
remove weight now that we use tiers
2023-01-13 13:45:48 -08:00
Bryan Stitt
9f7d28b538
retry if we get the method X is not available
2023-01-12 22:40:47 -08:00
Bryan Stitt
e4f1716f06
broadcast txs to less servers
2023-01-11 14:55:41 -08:00
Bryan Stitt
e2d0369cd5
lower level for reconnect
2023-01-09 18:23:27 -08:00
Bryan Stitt
ec43d69462
fake weight for now
2023-01-04 14:33:39 -08:00
Bryan Stitt
664ecf5924
use None if on head block
2023-01-04 12:12:44 -08:00
Bryan Stitt
045065986a
move allowed_lag around
2023-01-04 12:07:53 -08:00
Bryan Stitt
c0fc999e02
change weight to tier
2023-01-03 22:37:51 -08:00
Bryan Stitt
5a567ebeea
try a new way to check block data limit if no longer syncing but limit is 0
2023-01-03 11:54:24 -08:00
Bryan Stitt
d5a17c914d
lint
2023-01-03 11:38:06 -08:00
Bryan Stitt
1aefcf166d
different polling for different chains
2023-01-03 11:37:42 -08:00
Bryan Stitt
5be5128c93
partial refactor of allowed lag
2023-01-03 08:33:49 -08:00
Bryan Stitt
43dd9628e6
Revert "Revert "Revert "self.check_block_data_limit when syncing changes"""
...
This reverts commit 36a33a8a1f1ed054a539736cc439e76bd2e7f945.
2023-01-03 07:51:18 -08:00
Bryan Stitt
36a33a8a1f
Revert "Revert "self.check_block_data_limit when syncing changes""
...
This reverts commit ccb79522b6698625c2a00c71f8e71644008ca943.
2023-01-03 06:08:40 -08:00
Bryan Stitt
ccb79522b6
Revert "self.check_block_data_limit when syncing changes"
...
This reverts commit f3802a48357bd95d4e2b61e650341a458323aa45.
2023-01-03 05:26:21 -08:00
Bryan Stitt
f3802a4835
self.check_block_data_limit when syncing changes
2023-01-02 21:01:04 -08:00
Bryan Stitt
9b6b0ab7d2
remove extra brace
2023-01-02 10:42:27 -08:00
Bryan Stitt
206d339d45
all_backend_connections skips syncing servers
2023-01-02 10:34:16 -08:00
Bryan Stitt
6a201e1e47
don't set block_data_limit to 0
2022-12-27 18:17:11 -08:00
Bryan Stitt
4a837b35cc
improve eth_sendRawTransaction
2022-12-23 17:35:35 -08:00
Bryan Stitt
e60b9dc3fe
lower eth_getBlockByNumber level to Warn
2022-12-20 22:01:35 -08:00
Bryan Stitt
79075f9974
lowest log level for eth_sendRawTransaction
...
these are showing up in sentry with things like 'not enough funds'. its an error for the user to see, not us
2022-12-20 21:55:12 -08:00
Bryan Stitt
c0b457cec8
fix allow_not_ready
2022-12-20 21:47:41 -08:00
Bryan Stitt
f27c764a07
more logs
2022-12-19 15:59:01 -08:00
Bryan Stitt
2a19d9791b
more logging
2022-12-19 13:53:38 -08:00
Bryan Stitt
f04905698a
improve request caching
2022-12-16 20:05:01 -08:00
Bryan Stitt
4c2d3634c5
default to less detailed stats
2022-12-11 20:39:54 -08:00
Bryan Stitt
86f6b16761
another pass at server selection
2022-12-07 22:54:38 -08:00
Bryan Stitt
78a2119c07
quieter syncing nodes
2022-12-05 16:55:18 -08:00
Bryan Stitt
b02c851237
raise log level for block data limit
2022-12-05 16:47:27 -08:00
Bryan Stitt
c10a0e9d3c
better error context
2022-12-05 16:35:51 -08:00
Bryan Stitt
a54a061bd2
handle null case
2022-12-05 16:18:31 -08:00
Bryan Stitt
33f7256236
clones to avoid deadlock
2022-12-05 16:06:28 -08:00
Bryan Stitt
f010166ee0
better lag error
2022-12-05 14:45:12 -08:00
Bryan Stitt
dc5c944545
rename head_block_id to head_block
2022-12-05 14:38:54 -08:00
Bryan Stitt
5bec8bb5b9
much smarter connection logic
2022-12-05 14:11:56 -08:00
Bryan Stitt
f736aeb027
move config safty checks earlier
2022-12-04 17:25:23 -08:00
Bryan Stitt
69f0125eba
tests must include the current time
2022-12-04 17:10:20 -08:00
Bryan Stitt
0b93cb966e
trailing commas
2022-12-02 21:55:46 -08:00
Bryan Stitt
9101c767e3
logs match
2022-12-02 21:49:41 -08:00
Bryan Stitt
b8ac77a342
skip nodes that are syncing
2022-12-02 21:38:19 -08:00
Bryan Stitt
eb4b487aae
better block saving
2022-12-02 21:31:03 -08:00
Bryan Stitt
1b0cab9f54
better errors
2022-12-01 02:01:41 +00:00
Bryan Stitt
0d2816c487
add old block detection
2022-11-30 22:11:14 +00:00
Bryan Stitt
04d95f7234
allow unused imports for now
2022-11-29 21:34:42 +00:00
Bryan Stitt
519ba473d9
improve rpc filtering
2022-11-28 07:01:34 +00:00
Bryan Stitt
d06aa3b170
health compass cli
2022-11-28 06:10:34 +00:00
Bryan Stitt
40548bec4f
lower log level
2022-11-26 04:57:25 +00:00
Bryan Stitt
f09d836dfe
better weights
2022-11-25 07:41:53 +00:00
Bryan Stitt
4dfb7a5451
fix archive node detection when nodes used to be archive but than changed flags
2022-11-25 00:48:17 +00:00
Bryan Stitt
5b1621ead4
fix bug with not using synced_connections correctly
2022-11-22 22:45:22 +00:00
Bryan Stitt
87b0ecc916
test has_block_data for pruned node
2022-11-22 20:44:23 +00:00
Bryan Stitt
527218d229
test has_block_data for archive nodes
2022-11-22 20:23:08 +00:00
Bryan Stitt
311f6f1ac6
more ArcBlock
2022-11-21 17:49:57 +00:00
Bryan Stitt
217a7b3fd3
eth_subscribe rpc_accounting logging
2022-11-19 22:05:51 +00:00
Bryan Stitt
4d8808b886
a little bit more trace logging
2022-11-16 08:34:17 +00:00
Bryan Stitt
2829ce7a7d
lint more
2022-11-16 07:19:56 +00:00
Bryan Stitt
ea5f791560
simple lock around database migrations
2022-11-14 18:24:52 +00:00
Bryan Stitt
0f16fdb97a
comments
2022-11-13 21:25:58 +00:00
Bryan Stitt
96a486866c
add optional display name
2022-11-13 21:05:37 +00:00
Bryan Stitt
0e1cf5767c
cut out tracing for now
2022-11-12 08:24:32 +00:00
Bryan Stitt
9ae2337d1d
thread fast rng
2022-11-12 06:11:58 +00:00
Bryan Stitt
1329084621
cargo upgrade
2022-11-11 18:40:52 +00:00
Bryan Stitt
c35dd96cfb
save rpc_key_id or origin. needs some testing
2022-11-11 06:19:14 +00:00
Bryan Stitt
c33342d9dd
simplify authorization types so we can pass them deeper easily
2022-11-11 06:19:12 +00:00
Bryan Stitt
e01f8505fd
log params too
...
i'm not sure we want this, but maybe
2022-11-07 22:25:52 +00:00
Bryan Stitt
31395c9934
rpc_key_id in the redirect. weights in the /status page
2022-11-07 22:10:19 +00:00
