Bryan Stitt
6717999ced
actually use backup rpcs
2023-03-07 16:02:22 -08:00
Bryan Stitt
dbd7860416
wip influxdb2
2023-03-04 20:54:04 +01:00
Bryan Stitt
eb4d05a520
stats v2
...
rebased all my commits and squashed them down to one
2023-03-04 20:48:49 +01:00
Bryan Stitt
36e4183373
todos
2023-03-03 11:19:25 -08:00
Bryan Stitt
94089eb72b
combine comparison
2023-03-03 07:34:04 -08:00
Bryan Stitt
3b61e009f8
group cfgs
2023-03-02 17:51:41 -08:00
Bryan Stitt
3098791ad9
add optional kafka feature
2023-03-03 01:39:50 +00:00
Bryan Stitt
d1b955275d
clean up wait_for_ functions
2023-03-02 18:27:32 +00:00
Bryan Stitt
907a147afa
wait_for_provider helper function
2023-03-02 18:14:17 +00:00
Bryan Stitt
0d07e20d7c
make it compile
2023-03-02 17:38:42 +00:00
Bryan Stitt
258e889f69
wait for a provider
2023-03-02 16:52:28 +00:00
Bryan Stitt
499d50b5fc
handle empty private_rpcs
2023-03-01 23:21:09 +00:00
Bryan Stitt
c8bc0d288f
high log level
2023-03-01 23:04:20 +00:00
Bryan Stitt
fdba8e7864
test all_connections with Nones
2023-03-01 22:50:44 +00:00
Bryan Stitt
f68886fafd
add simple test for all connections and include more rpcs
2023-03-01 22:32:01 +00:00
Bryan Stitt
baf07f7de5
sync sort should include backup
2023-03-01 21:35:50 +00:00
Bryan Stitt
2e9803de72
remove arbitrary sleep timer
2023-03-01 20:56:00 +00:00
Bryan Stitt
b87c988439
lint
2023-03-01 11:23:59 -08:00
Bryan Stitt
20851af51b
log seconds instead of milliseconds
2023-02-28 11:09:49 -08:00
Bryan Stitt
d83a1f0169
use Web3Rpc instead of the name as a key
2023-02-28 11:01:34 -08:00
Bryan Stitt
26970c2d88
move wait on new connection
2023-02-27 13:40:13 -08:00
Bryan Stitt
c5e2402d6e
disconnect is noisy but it works
2023-02-27 13:29:07 -08:00
Bryan Stitt
6067369ee3
add inotify and rpc disconnect
2023-02-27 12:50:57 -08:00
Bryan Stitt
f8f5e7a1c8
add thread for config file watching and run clippy lint
2023-02-26 23:08:00 -08:00
Bryan Stitt
bf79d677b0
by_name needs to a lock
2023-02-26 20:00:13 -08:00
Bryan Stitt
e96f09a9c4
refactors to make configs partially reloadable
2023-02-25 23:52:33 -08:00
Bryan Stitt
c9e5661c5b
wip
2023-02-25 09:55:28 -08:00
Bryan Stitt
580965ef83
get instead of remove
2023-02-24 23:31:10 -08:00
Bryan Stitt
91b853d21e
archive instead of Some(u64::MAX) in the logs
2023-02-21 21:14:49 -08:00
Bryan Stitt
6c33b1ae1e
block number instead of hash during health check
2023-02-21 20:25:54 -08:00
Bryan Stitt
738815244d
change ewma calculation
2023-02-16 00:26:58 -08:00
Bryan Stitt
4916188d5c
theres a perf regression in here somewhere
2023-02-15 21:54:07 -08:00
Bryan Stitt
cb2b5c16ad
lower log level
2023-02-15 21:30:42 -08:00
Bryan Stitt
b7dfec9c63
fix params in health check query
2023-02-15 21:13:28 -08:00
Bryan Stitt
91cab8ffe2
comment
2023-02-15 21:09:39 -08:00
Bryan Stitt
bc306f62d4
make it work
2023-02-15 21:05:41 -08:00
Bryan Stitt
3c9576c13b
yes health, no channel
2023-02-15 15:52:42 -08:00
Bryan Stitt
63adbc0587
remove health for now?
2023-02-15 15:42:27 -08:00
Bryan Stitt
268ad3a7c5
dont hold a lock open
2023-02-15 15:37:43 -08:00
Bryan Stitt
056670d29a
health check and latency recordings
2023-02-15 15:31:59 -08:00
Bryan Stitt
774bd5b232
undo a rename in the json
2023-02-15 12:59:03 -08:00
Bryan Stitt
1fb4dd6ccc
i think it works
2023-02-15 12:33:43 -08:00
Bryan Stitt
df668a5dfd
add time to idle to more caches
2023-02-15 11:42:25 -08:00
Bryan Stitt
c65ffc9ce0
getting close
2023-02-14 17:41:40 -08:00
Bryan Stitt
f1e6de3677
fixed one test. still more to fix though
2023-02-14 12:41:05 -08:00
Bryan Stitt
824a6fa1f0
well it compiles. doesnt work yet. but it compiles
2023-02-14 12:14:50 -08:00
Bryan Stitt
70105bc7bb
variable rename
2023-02-12 10:22:20 -08:00
Bryan Stitt
0e2e45cf90
cargo upgrade
2023-02-12 01:47:06 -08:00
Bryan Stitt
c008f50943
use http and ws providers in a single config
2023-02-12 01:40:01 -08:00
Bryan Stitt
f1bc00082a
include to_block more places
2023-02-10 20:45:57 -08:00
Bryan Stitt
c959110986
better sorting of connections
2023-02-10 20:24:20 -08:00
Bryan Stitt
1a93e39b09
FuturesUnordered on the spawn handles
2023-02-09 11:56:07 -08:00
Bryan Stitt
6741cd72ac
lint
2023-02-09 11:55:54 -08:00
Bryan Stitt
5ffe2aa72a
eth_sendRawTransactions should use backups more often
2023-02-09 11:47:56 -08:00
Bryan Stitt
1efa8e7772
improve log message
...
we don't always get an empty block because we aren't synced. sometimes it is because we got disconnected
2023-02-08 11:30:16 -08:00
Bryan Stitt
0b1929a820
add minimal config for quick testing
2023-02-06 15:20:36 -08:00
Bryan Stitt
0edd11349f
web3 rpc spawn should be unordered
2023-02-06 14:13:43 -08:00
Bryan Stitt
19e51ce328
check http servers twice as often
2023-02-06 13:48:50 -08:00
Bryan Stitt
0f280ce483
cargo upgrade and shorten variable names
...
also begin adding a latency tracker for rpc stats
2023-02-06 09:55:27 -08:00
Bryan Stitt
f2d35ba5eb
remove metered in favor of influxdb stats
2023-02-05 18:17:26 -08:00
Bryan Stitt
6d959e2c1f
add archive depth to app config
2023-02-05 17:58:03 -08:00
Bryan Stitt
37830f1156
Jenkinsfile from llamanodes/erigon
2023-02-05 00:13:14 -08:00
Bryan Stitt
ca1e550370
improve sort order during eth_sendRawTransaction
2023-02-02 14:48:23 -08:00
Bryan Stitt
cac31303f3
include backup in web3connection serializer
2023-02-02 08:00:59 -08:00
Bryan Stitt
a76a45f63a
clone instead of take
2023-01-31 09:47:19 -08:00
Bryan Stitt
8bcf40b62d
Revert "Revert "include from_block and to_block for caching getLogs""
...
This reverts commit 13bab2c9cfcc09f8d503ff24f6e2c2a3fc0faaa2.
2023-01-31 08:30:24 -08:00
Bryan Stitt
13bab2c9cf
Revert "include from_block and to_block for caching getLogs"
...
This reverts commit 203969f62833668c4081acbb2188989fd6545c24.
2023-01-31 08:30:11 -08:00
Bryan Stitt
203969f628
include from_block and to_block for caching getLogs
2023-01-30 15:47:17 -08:00
Bryan Stitt
f2260ecdba
don't require hard limit send
2023-01-25 10:01:36 -08:00
Bryan Stitt
cffc60e7f6
improve responses when blocks are not available
2023-01-24 22:45:20 -08:00
Bryan Stitt
694e552b5d
improve waiting for sync when rate limited
2023-01-24 20:45:03 -08:00
Bryan Stitt
641d11a19b
remove excess continues
2023-01-24 10:09:12 -08:00
Bryan Stitt
106dec294f
better handling when method not available
2023-01-24 09:36:07 -08:00
Bryan Stitt
a242244a35
broadcast transactions to more servers
2023-01-24 01:58:31 -08:00
Bryan Stitt
47daab3b67
drop log level for backup servers
2023-01-23 21:37:23 -08:00
Bryan Stitt
86e3f2991f
use watch instead of arcswap
2023-01-22 22:02:08 -08:00
Bryan Stitt
b9a12756b9
collect request_metadata.response_from_backup_rpc
2023-01-19 21:46:47 -08:00
Bryan Stitt
e53030e053
move backup indicator
2023-01-19 18:30:30 -08:00
Bryan Stitt
c3b53eb5f2
add backup indicator to more logs
2023-01-19 18:14:47 -08:00
Bryan Stitt
ad169eda0e
log cleanup
2023-01-19 17:57:36 -08:00
Bryan Stitt
52a9ba604c
remove allowed lag
2023-01-19 03:05:39 -08:00
Bryan Stitt
2cb6dde052
more log improvements
2023-01-19 02:26:54 -08:00
Bryan Stitt
76e51e3d11
better log level
2023-01-19 02:21:39 -08:00
Bryan Stitt
0c05b5bdee
major refactor to only use backup servers when absolutely necessary
2023-01-19 02:20:06 -08:00
Bryan Stitt
90d3371eee
improved rate limiting on websockets
2023-01-18 16:17:47 -08:00
Bryan Stitt
d7c75f843e
add stub try_proxy_connection
2023-01-16 22:54:40 -08:00
Bryan Stitt
8c02e4420d
remove weight now that we use tiers
2023-01-13 13:45:48 -08:00
Bryan Stitt
9f7d28b538
retry if we get the method X is not available
2023-01-12 22:40:47 -08:00
Bryan Stitt
e4f1716f06
broadcast txs to less servers
2023-01-11 14:55:41 -08:00
Bryan Stitt
e2d0369cd5
lower level for reconnect
2023-01-09 18:23:27 -08:00
Bryan Stitt
ec43d69462
fake weight for now
2023-01-04 14:33:39 -08:00
Bryan Stitt
664ecf5924
use None if on head block
2023-01-04 12:12:44 -08:00
Bryan Stitt
045065986a
move allowed_lag around
2023-01-04 12:07:53 -08:00
Bryan Stitt
c0fc999e02
change weight to tier
2023-01-03 22:37:51 -08:00
Bryan Stitt
5a567ebeea
try a new way to check block data limit if no longer syncing but limit is 0
2023-01-03 11:54:24 -08:00
Bryan Stitt
d5a17c914d
lint
2023-01-03 11:38:06 -08:00
Bryan Stitt
1aefcf166d
different polling for different chains
2023-01-03 11:37:42 -08:00
Bryan Stitt
5be5128c93
partial refactor of allowed lag
2023-01-03 08:33:49 -08:00
Bryan Stitt
43dd9628e6
Revert "Revert "Revert "self.check_block_data_limit when syncing changes"""
...
This reverts commit 36a33a8a1f1ed054a539736cc439e76bd2e7f945.
2023-01-03 07:51:18 -08:00
Bryan Stitt
36a33a8a1f
Revert "Revert "self.check_block_data_limit when syncing changes""
...
This reverts commit ccb79522b6698625c2a00c71f8e71644008ca943.
2023-01-03 06:08:40 -08:00
Bryan Stitt
ccb79522b6
Revert "self.check_block_data_limit when syncing changes"
...
This reverts commit f3802a48357bd95d4e2b61e650341a458323aa45.
2023-01-03 05:26:21 -08:00
Bryan Stitt
f3802a4835
self.check_block_data_limit when syncing changes
2023-01-02 21:01:04 -08:00
Bryan Stitt
9b6b0ab7d2
remove extra brace
2023-01-02 10:42:27 -08:00
Bryan Stitt
206d339d45
all_backend_connections skips syncing servers
2023-01-02 10:34:16 -08:00
Bryan Stitt
6a201e1e47
don't set block_data_limit to 0
2022-12-27 18:17:11 -08:00
Bryan Stitt
4a837b35cc
improve eth_sendRawTransaction
2022-12-23 17:35:35 -08:00
Bryan Stitt
e60b9dc3fe
lower eth_getBlockByNumber level to Warn
2022-12-20 22:01:35 -08:00
Bryan Stitt
79075f9974
lowest log level for eth_sendRawTransaction
...
these are showing up in sentry with things like 'not enough funds'. its an error for the user to see, not us
2022-12-20 21:55:12 -08:00
Bryan Stitt
c0b457cec8
fix allow_not_ready
2022-12-20 21:47:41 -08:00
Bryan Stitt
f27c764a07
more logs
2022-12-19 15:59:01 -08:00
Bryan Stitt
2a19d9791b
more logging
2022-12-19 13:53:38 -08:00
Bryan Stitt
f04905698a
improve request caching
2022-12-16 20:05:01 -08:00
Bryan Stitt
4c2d3634c5
default to less detailed stats
2022-12-11 20:39:54 -08:00
Bryan Stitt
86f6b16761
another pass at server selection
2022-12-07 22:54:38 -08:00
Bryan Stitt
78a2119c07
quieter syncing nodes
2022-12-05 16:55:18 -08:00
Bryan Stitt
b02c851237
raise log level for block data limit
2022-12-05 16:47:27 -08:00
Bryan Stitt
c10a0e9d3c
better error context
2022-12-05 16:35:51 -08:00
Bryan Stitt
a54a061bd2
handle null case
2022-12-05 16:18:31 -08:00
Bryan Stitt
33f7256236
clones to avoid deadlock
2022-12-05 16:06:28 -08:00
Bryan Stitt
f010166ee0
better lag error
2022-12-05 14:45:12 -08:00
Bryan Stitt
dc5c944545
rename head_block_id to head_block
2022-12-05 14:38:54 -08:00
Bryan Stitt
5bec8bb5b9
much smarter connection logic
2022-12-05 14:11:56 -08:00
Bryan Stitt
f736aeb027
move config safty checks earlier
2022-12-04 17:25:23 -08:00
Bryan Stitt
69f0125eba
tests must include the current time
2022-12-04 17:10:20 -08:00
Bryan Stitt
0b93cb966e
trailing commas
2022-12-02 21:55:46 -08:00
Bryan Stitt
9101c767e3
logs match
2022-12-02 21:49:41 -08:00
Bryan Stitt
b8ac77a342
skip nodes that are syncing
2022-12-02 21:38:19 -08:00
Bryan Stitt
eb4b487aae
better block saving
2022-12-02 21:31:03 -08:00
Bryan Stitt
1b0cab9f54
better errors
2022-12-01 02:01:41 +00:00
Bryan Stitt
0d2816c487
add old block detection
2022-11-30 22:11:14 +00:00
Bryan Stitt
04d95f7234
allow unused imports for now
2022-11-29 21:34:42 +00:00
Bryan Stitt
519ba473d9
improve rpc filtering
2022-11-28 07:01:34 +00:00
Bryan Stitt
d06aa3b170
health compass cli
2022-11-28 06:10:34 +00:00
Bryan Stitt
40548bec4f
lower log level
2022-11-26 04:57:25 +00:00
Bryan Stitt
f09d836dfe
better weights
2022-11-25 07:41:53 +00:00
Bryan Stitt
4dfb7a5451
fix archive node detection when nodes used to be archive but than changed flags
2022-11-25 00:48:17 +00:00
Bryan Stitt
5b1621ead4
fix bug with not using synced_connections correctly
2022-11-22 22:45:22 +00:00
Bryan Stitt
87b0ecc916
test has_block_data for pruned node
2022-11-22 20:44:23 +00:00
Bryan Stitt
527218d229
test has_block_data for archive nodes
2022-11-22 20:23:08 +00:00
Bryan Stitt
311f6f1ac6
more ArcBlock
2022-11-21 17:49:57 +00:00
Bryan Stitt
217a7b3fd3
eth_subscribe rpc_accounting logging
2022-11-19 22:05:51 +00:00
Bryan Stitt
4d8808b886
a little bit more trace logging
2022-11-16 08:34:17 +00:00
Bryan Stitt
2829ce7a7d
lint more
2022-11-16 07:19:56 +00:00
Bryan Stitt
ea5f791560
simple lock around database migrations
2022-11-14 18:24:52 +00:00
Bryan Stitt
0f16fdb97a
comments
2022-11-13 21:25:58 +00:00
Bryan Stitt
96a486866c
add optional display name
2022-11-13 21:05:37 +00:00
Bryan Stitt
0e1cf5767c
cut out tracing for now
2022-11-12 08:24:32 +00:00
Bryan Stitt
9ae2337d1d
thread fast rng
2022-11-12 06:11:58 +00:00
Bryan Stitt
1329084621
cargo upgrade
2022-11-11 18:40:52 +00:00
Bryan Stitt
c35dd96cfb
save rpc_key_id or origin. needs some testing
2022-11-11 06:19:14 +00:00
Bryan Stitt
c33342d9dd
simplify authorization types so we can pass them deeper easily
2022-11-11 06:19:12 +00:00
Bryan Stitt
e01f8505fd
log params too
...
i'm not sure we want this, but maybe
2022-11-07 22:25:52 +00:00
Bryan Stitt
31395c9934
rpc_key_id in the redirect. weights in the /status page
2022-11-07 22:10:19 +00:00
Bryan Stitt
fc8beb52b0
maybe fix next_upstream_server
2022-11-06 21:05:03 +00:00
Bryan Stitt
058dfa6d8e
send_head_block_result more places
2022-11-06 20:52:11 +00:00
Bryan Stitt
f3e9f6c387
bug fixes
2022-11-03 22:16:27 +00:00
Bryan Stitt
e8f2a13f5d
better stats aggregations
2022-11-02 23:14:16 +00:00
Bryan Stitt
f859ed56fc
period instead of minute in variable names
2022-11-01 19:24:29 +00:00
Bryan Stitt
8b35bf5e63
clean up migration
2022-11-01 18:54:39 +00:00
Bryan Stitt
a534eae968
trace everything
2022-10-28 22:52:47 +00:00
Bryan Stitt
366f2c8f84
rename api_key to rpc_key
2022-10-27 00:12:42 +00:00
Bryan Stitt
8f3e5c0146
user post endpoint
2022-10-26 21:39:26 +00:00
Bryan Stitt
1d24955d6f
finish basic ProtectedAction
2022-10-25 04:12:24 +00:00
Bryan Stitt
4ccbcb8af4
update sea-orm and entities
2022-10-25 03:41:59 +00:00
Bryan Stitt
b7259192e8
more context. don't use unix timestamps
2022-10-20 23:50:23 +00:00
Bryan Stitt
848eb61349
null private transactions instead of empty
2022-10-20 07:03:58 +00:00
Bryan Stitt
1da730daa2
less unwrap
2022-10-20 06:54:45 +00:00
Bryan Stitt
f6c2d29d0e
stricter configs
2022-10-18 23:27:33 +00:00
Bryan Stitt
76c8f1ef96
i think it works
2022-10-11 21:31:34 +00:00
Bryan Stitt
552f3dbffc
proper sizes for caches and emit all stats
2022-10-11 19:58:25 +00:00
Bryan Stitt
25aa68a5bf
add per-user rpc accounting
2022-10-10 04:15:49 +00:00
Bryan Stitt
875ae457ef
emit our first stats-- cache hits and misses
2022-10-03 20:02:05 +00:00
Bryan Stitt
a7b5f25467
unwrap less
2022-09-30 04:18:18 +00:00
Bryan Stitt
9cc34f7aa8
move warning
2022-09-28 17:01:11 +00:00
Bryan Stitt
2b03372e6b
error instead of todo
2022-09-28 16:35:18 +00:00
Bryan Stitt
2c69b0ce9a
fix method and param for saving reverts
2022-09-24 07:36:25 +00:00
Bryan Stitt
a80503ac48
better logging on save reverts checks
2022-09-24 07:04:11 +00:00
Bryan Stitt
5df2469d53
ULID or UUID. Prefer ULID
2022-09-24 05:53:45 +00:00
Bryan Stitt
81deb1103c
login needs its own rate limiter
2022-09-24 03:59:21 +00:00
Bryan Stitt
c12e25a210
no need for this clone
2022-09-24 02:48:23 +00:00
Bryan Stitt
8459dcd1f1
let the frontend handle their own cookies
2022-09-24 02:47:44 +00:00
Bryan Stitt
dbd8ea2429
add logout endpoint and prefix with /rpc
2022-09-23 21:49:18 +00:00
Bryan Stitt
961ccf7cf2
ip, origin, referer, and user agent checks
2022-09-23 05:22:33 +00:00
Bryan Stitt
d55aea2d98
even flatter
2022-09-22 22:42:44 +00:00
Bryan Stitt
7f7be6aa69
flatter
2022-09-22 22:34:43 +00:00
Bryan Stitt
987ed181cd
comments
2022-09-22 22:16:28 +00:00
Bryan Stitt
6d23071640
less clones, but still some
2022-09-22 22:14:24 +00:00
Bryan Stitt
8d011e0cd1
pass db conn through
2022-09-22 22:10:28 +00:00
Bryan Stitt
3854312674
we need to clone :'(
2022-09-22 21:51:52 +00:00
Bryan Stitt
2ed2408ed3
even more authorized_request
2022-09-22 21:04:03 +00:00
Bryan Stitt
43846a7051
pass authorized_request through a bunch of places
2022-09-22 20:27:14 +00:00
Bryan Stitt
6905e9fd46
create a struct for authenticated requests that we need for per-key stats
2022-09-22 19:57:21 +00:00
Bryan Stitt
8481f6d44c
wip
2022-09-21 23:50:55 +00:00
Bryan Stitt
37a1aa554b
revert error saving and extract blockHash from requests
2022-09-21 04:48:21 +00:00
Bryan Stitt
6054c3f340
we do need fetch_sub
2022-09-21 02:48:02 +00:00
Bryan Stitt
abe5001792
atomic and less expires
2022-09-20 06:56:24 +00:00
Bryan Stitt
739947792a
instrument more. add max_wait to wait_for_request_handle
2022-09-20 06:00:27 +00:00
Bryan Stitt
6ae24b1ff9
use cache's expiration and race-free get_with
...
when this was a dashmap, we needed our own expiration and parallel requests would do the same query.
with moka, we can use their expiration code and get_with
2022-09-20 01:33:39 +00:00
Bryan Stitt
28fa424c2a
start weighers
2022-09-17 02:30:06 +00:00
Bryan Stitt
05af0551c9
use ahash
...
this might actually be slower. do real benchmarks
2022-09-17 02:17:20 +00:00
