Bryan Stitt
c959110986
better sorting of connections
2023-02-10 20:24:20 -08:00
Bryan Stitt
1a93e39b09
FuturesUnordered on the spawn handles
2023-02-09 11:56:07 -08:00
Bryan Stitt
6741cd72ac
lint
2023-02-09 11:55:54 -08:00
Bryan Stitt
5ffe2aa72a
eth_sendRawTransactions should use backups more often
2023-02-09 11:47:56 -08:00
Bryan Stitt
1efa8e7772
improve log message
...
we don't always get an empty block because we aren't synced. sometimes it is because we got disconnected
2023-02-08 11:30:16 -08:00
Bryan Stitt
0b1929a820
add minimal config for quick testing
2023-02-06 15:20:36 -08:00
Bryan Stitt
0edd11349f
web3 rpc spawn should be unordered
2023-02-06 14:13:43 -08:00
Bryan Stitt
19e51ce328
check http servers twice as often
2023-02-06 13:48:50 -08:00
Bryan Stitt
0f280ce483
cargo upgrade and shorten variable names
...
also begin adding a latency tracker for rpc stats
2023-02-06 09:55:27 -08:00
Bryan Stitt
f2d35ba5eb
remove metered in favor of influxdb stats
2023-02-05 18:17:26 -08:00
Bryan Stitt
6d959e2c1f
add archive depth to app config
2023-02-05 17:58:03 -08:00
Bryan Stitt
37830f1156
Jenkinsfile from llamanodes/erigon
2023-02-05 00:13:14 -08:00
Bryan Stitt
ca1e550370
improve sort order during eth_sendRawTransaction
2023-02-02 14:48:23 -08:00
Bryan Stitt
cac31303f3
include backup in web3connection serializer
2023-02-02 08:00:59 -08:00
Bryan Stitt
a76a45f63a
clone instead of take
2023-01-31 09:47:19 -08:00
Bryan Stitt
8bcf40b62d
Revert "Revert "include from_block and to_block for caching getLogs""
...
This reverts commit 13bab2c9cfcc09f8d503ff24f6e2c2a3fc0faaa2.
2023-01-31 08:30:24 -08:00
Bryan Stitt
13bab2c9cf
Revert "include from_block and to_block for caching getLogs"
...
This reverts commit 203969f62833668c4081acbb2188989fd6545c24.
2023-01-31 08:30:11 -08:00
Bryan Stitt
203969f628
include from_block and to_block for caching getLogs
2023-01-30 15:47:17 -08:00
Bryan Stitt
f2260ecdba
don't require hard limit send
2023-01-25 10:01:36 -08:00
Bryan Stitt
cffc60e7f6
improve responses when blocks are not available
2023-01-24 22:45:20 -08:00
Bryan Stitt
694e552b5d
improve waiting for sync when rate limited
2023-01-24 20:45:03 -08:00
Bryan Stitt
641d11a19b
remove excess continues
2023-01-24 10:09:12 -08:00
Bryan Stitt
106dec294f
better handling when method not available
2023-01-24 09:36:07 -08:00
Bryan Stitt
a242244a35
broadcast transactions to more servers
2023-01-24 01:58:31 -08:00
Bryan Stitt
47daab3b67
drop log level for backup servers
2023-01-23 21:37:23 -08:00
Bryan Stitt
86e3f2991f
use watch instead of arcswap
2023-01-22 22:02:08 -08:00
Bryan Stitt
b9a12756b9
collect request_metadata.response_from_backup_rpc
2023-01-19 21:46:47 -08:00
Bryan Stitt
e53030e053
move backup indicator
2023-01-19 18:30:30 -08:00
Bryan Stitt
c3b53eb5f2
add backup indicator to more logs
2023-01-19 18:14:47 -08:00
Bryan Stitt
ad169eda0e
log cleanup
2023-01-19 17:57:36 -08:00
Bryan Stitt
52a9ba604c
remove allowed lag
2023-01-19 03:05:39 -08:00
Bryan Stitt
2cb6dde052
more log improvements
2023-01-19 02:26:54 -08:00
Bryan Stitt
76e51e3d11
better log level
2023-01-19 02:21:39 -08:00
Bryan Stitt
0c05b5bdee
major refactor to only use backup servers when absolutely necessary
2023-01-19 02:20:06 -08:00
Bryan Stitt
90d3371eee
improved rate limiting on websockets
2023-01-18 16:17:47 -08:00
Bryan Stitt
d7c75f843e
add stub try_proxy_connection
2023-01-16 22:54:40 -08:00
Bryan Stitt
8c02e4420d
remove weight now that we use tiers
2023-01-13 13:45:48 -08:00
Bryan Stitt
9f7d28b538
retry if we get the method X is not available
2023-01-12 22:40:47 -08:00
Bryan Stitt
e4f1716f06
broadcast txs to less servers
2023-01-11 14:55:41 -08:00
Bryan Stitt
e2d0369cd5
lower level for reconnect
2023-01-09 18:23:27 -08:00
Bryan Stitt
ec43d69462
fake weight for now
2023-01-04 14:33:39 -08:00
Bryan Stitt
664ecf5924
use None if on head block
2023-01-04 12:12:44 -08:00
Bryan Stitt
045065986a
move allowed_lag around
2023-01-04 12:07:53 -08:00
Bryan Stitt
c0fc999e02
change weight to tier
2023-01-03 22:37:51 -08:00
Bryan Stitt
5a567ebeea
try a new way to check block data limit if no longer syncing but limit is 0
2023-01-03 11:54:24 -08:00
Bryan Stitt
d5a17c914d
lint
2023-01-03 11:38:06 -08:00
Bryan Stitt
1aefcf166d
different polling for different chains
2023-01-03 11:37:42 -08:00
Bryan Stitt
5be5128c93
partial refactor of allowed lag
2023-01-03 08:33:49 -08:00
Bryan Stitt
43dd9628e6
Revert "Revert "Revert "self.check_block_data_limit when syncing changes"""
...
This reverts commit 36a33a8a1f1ed054a539736cc439e76bd2e7f945.
2023-01-03 07:51:18 -08:00
Bryan Stitt
36a33a8a1f
Revert "Revert "self.check_block_data_limit when syncing changes""
...
This reverts commit ccb79522b6698625c2a00c71f8e71644008ca943.
2023-01-03 06:08:40 -08:00
Bryan Stitt
ccb79522b6
Revert "self.check_block_data_limit when syncing changes"
...
This reverts commit f3802a48357bd95d4e2b61e650341a458323aa45.
2023-01-03 05:26:21 -08:00
Bryan Stitt
f3802a4835
self.check_block_data_limit when syncing changes
2023-01-02 21:01:04 -08:00
Bryan Stitt
9b6b0ab7d2
remove extra brace
2023-01-02 10:42:27 -08:00
Bryan Stitt
206d339d45
all_backend_connections skips syncing servers
2023-01-02 10:34:16 -08:00
Bryan Stitt
6a201e1e47
don't set block_data_limit to 0
2022-12-27 18:17:11 -08:00
Bryan Stitt
4a837b35cc
improve eth_sendRawTransaction
2022-12-23 17:35:35 -08:00
Bryan Stitt
e60b9dc3fe
lower eth_getBlockByNumber level to Warn
2022-12-20 22:01:35 -08:00
Bryan Stitt
79075f9974
lowest log level for eth_sendRawTransaction
...
these are showing up in sentry with things like 'not enough funds'. its an error for the user to see, not us
2022-12-20 21:55:12 -08:00
Bryan Stitt
c0b457cec8
fix allow_not_ready
2022-12-20 21:47:41 -08:00
Bryan Stitt
f27c764a07
more logs
2022-12-19 15:59:01 -08:00
Bryan Stitt
2a19d9791b
more logging
2022-12-19 13:53:38 -08:00
Bryan Stitt
f04905698a
improve request caching
2022-12-16 20:05:01 -08:00
Bryan Stitt
4c2d3634c5
default to less detailed stats
2022-12-11 20:39:54 -08:00
Bryan Stitt
86f6b16761
another pass at server selection
2022-12-07 22:54:38 -08:00
Bryan Stitt
78a2119c07
quieter syncing nodes
2022-12-05 16:55:18 -08:00
Bryan Stitt
b02c851237
raise log level for block data limit
2022-12-05 16:47:27 -08:00
Bryan Stitt
c10a0e9d3c
better error context
2022-12-05 16:35:51 -08:00
Bryan Stitt
a54a061bd2
handle null case
2022-12-05 16:18:31 -08:00
Bryan Stitt
33f7256236
clones to avoid deadlock
2022-12-05 16:06:28 -08:00
Bryan Stitt
f010166ee0
better lag error
2022-12-05 14:45:12 -08:00
Bryan Stitt
dc5c944545
rename head_block_id to head_block
2022-12-05 14:38:54 -08:00
Bryan Stitt
5bec8bb5b9
much smarter connection logic
2022-12-05 14:11:56 -08:00
Bryan Stitt
f736aeb027
move config safty checks earlier
2022-12-04 17:25:23 -08:00
Bryan Stitt
69f0125eba
tests must include the current time
2022-12-04 17:10:20 -08:00
Bryan Stitt
0b93cb966e
trailing commas
2022-12-02 21:55:46 -08:00
Bryan Stitt
9101c767e3
logs match
2022-12-02 21:49:41 -08:00
Bryan Stitt
b8ac77a342
skip nodes that are syncing
2022-12-02 21:38:19 -08:00
Bryan Stitt
eb4b487aae
better block saving
2022-12-02 21:31:03 -08:00
Bryan Stitt
1b0cab9f54
better errors
2022-12-01 02:01:41 +00:00
Bryan Stitt
0d2816c487
add old block detection
2022-11-30 22:11:14 +00:00
Bryan Stitt
04d95f7234
allow unused imports for now
2022-11-29 21:34:42 +00:00
Bryan Stitt
519ba473d9
improve rpc filtering
2022-11-28 07:01:34 +00:00
Bryan Stitt
d06aa3b170
health compass cli
2022-11-28 06:10:34 +00:00
Bryan Stitt
40548bec4f
lower log level
2022-11-26 04:57:25 +00:00
Bryan Stitt
f09d836dfe
better weights
2022-11-25 07:41:53 +00:00
Bryan Stitt
4dfb7a5451
fix archive node detection when nodes used to be archive but than changed flags
2022-11-25 00:48:17 +00:00
Bryan Stitt
5b1621ead4
fix bug with not using synced_connections correctly
2022-11-22 22:45:22 +00:00
Bryan Stitt
87b0ecc916
test has_block_data for pruned node
2022-11-22 20:44:23 +00:00
Bryan Stitt
527218d229
test has_block_data for archive nodes
2022-11-22 20:23:08 +00:00
Bryan Stitt
311f6f1ac6
more ArcBlock
2022-11-21 17:49:57 +00:00
Bryan Stitt
217a7b3fd3
eth_subscribe rpc_accounting logging
2022-11-19 22:05:51 +00:00
Bryan Stitt
4d8808b886
a little bit more trace logging
2022-11-16 08:34:17 +00:00
Bryan Stitt
2829ce7a7d
lint more
2022-11-16 07:19:56 +00:00
Bryan Stitt
ea5f791560
simple lock around database migrations
2022-11-14 18:24:52 +00:00
Bryan Stitt
0f16fdb97a
comments
2022-11-13 21:25:58 +00:00
Bryan Stitt
96a486866c
add optional display name
2022-11-13 21:05:37 +00:00
Bryan Stitt
0e1cf5767c
cut out tracing for now
2022-11-12 08:24:32 +00:00
Bryan Stitt
9ae2337d1d
thread fast rng
2022-11-12 06:11:58 +00:00
Bryan Stitt
1329084621
cargo upgrade
2022-11-11 18:40:52 +00:00
Bryan Stitt
c35dd96cfb
save rpc_key_id or origin. needs some testing
2022-11-11 06:19:14 +00:00
Bryan Stitt
c33342d9dd
simplify authorization types so we can pass them deeper easily
2022-11-11 06:19:12 +00:00
Bryan Stitt
e01f8505fd
log params too
...
i'm not sure we want this, but maybe
2022-11-07 22:25:52 +00:00
Bryan Stitt
31395c9934
rpc_key_id in the redirect. weights in the /status page
2022-11-07 22:10:19 +00:00
Bryan Stitt
fc8beb52b0
maybe fix next_upstream_server
2022-11-06 21:05:03 +00:00
Bryan Stitt
058dfa6d8e
send_head_block_result more places
2022-11-06 20:52:11 +00:00
Bryan Stitt
f3e9f6c387
bug fixes
2022-11-03 22:16:27 +00:00
Bryan Stitt
e8f2a13f5d
better stats aggregations
2022-11-02 23:14:16 +00:00
Bryan Stitt
f859ed56fc
period instead of minute in variable names
2022-11-01 19:24:29 +00:00
Bryan Stitt
8b35bf5e63
clean up migration
2022-11-01 18:54:39 +00:00
Bryan Stitt
a534eae968
trace everything
2022-10-28 22:52:47 +00:00
Bryan Stitt
366f2c8f84
rename api_key to rpc_key
2022-10-27 00:12:42 +00:00
Bryan Stitt
8f3e5c0146
user post endpoint
2022-10-26 21:39:26 +00:00
Bryan Stitt
1d24955d6f
finish basic ProtectedAction
2022-10-25 04:12:24 +00:00
Bryan Stitt
4ccbcb8af4
update sea-orm and entities
2022-10-25 03:41:59 +00:00
Bryan Stitt
b7259192e8
more context. don't use unix timestamps
2022-10-20 23:50:23 +00:00
Bryan Stitt
848eb61349
null private transactions instead of empty
2022-10-20 07:03:58 +00:00
Bryan Stitt
1da730daa2
less unwrap
2022-10-20 06:54:45 +00:00
Bryan Stitt
f6c2d29d0e
stricter configs
2022-10-18 23:27:33 +00:00
Bryan Stitt
76c8f1ef96
i think it works
2022-10-11 21:31:34 +00:00
Bryan Stitt
552f3dbffc
proper sizes for caches and emit all stats
2022-10-11 19:58:25 +00:00
Bryan Stitt
25aa68a5bf
add per-user rpc accounting
2022-10-10 04:15:49 +00:00
Bryan Stitt
875ae457ef
emit our first stats-- cache hits and misses
2022-10-03 20:02:05 +00:00
Bryan Stitt
a7b5f25467
unwrap less
2022-09-30 04:18:18 +00:00
Bryan Stitt
9cc34f7aa8
move warning
2022-09-28 17:01:11 +00:00
Bryan Stitt
2b03372e6b
error instead of todo
2022-09-28 16:35:18 +00:00
Bryan Stitt
2c69b0ce9a
fix method and param for saving reverts
2022-09-24 07:36:25 +00:00
Bryan Stitt
a80503ac48
better logging on save reverts checks
2022-09-24 07:04:11 +00:00
Bryan Stitt
5df2469d53
ULID or UUID. Prefer ULID
2022-09-24 05:53:45 +00:00
Bryan Stitt
81deb1103c
login needs its own rate limiter
2022-09-24 03:59:21 +00:00
Bryan Stitt
c12e25a210
no need for this clone
2022-09-24 02:48:23 +00:00
Bryan Stitt
8459dcd1f1
let the frontend handle their own cookies
2022-09-24 02:47:44 +00:00
Bryan Stitt
dbd8ea2429
add logout endpoint and prefix with /rpc
2022-09-23 21:49:18 +00:00
Bryan Stitt
961ccf7cf2
ip, origin, referer, and user agent checks
2022-09-23 05:22:33 +00:00
Bryan Stitt
d55aea2d98
even flatter
2022-09-22 22:42:44 +00:00
Bryan Stitt
7f7be6aa69
flatter
2022-09-22 22:34:43 +00:00
Bryan Stitt
987ed181cd
comments
2022-09-22 22:16:28 +00:00
Bryan Stitt
6d23071640
less clones, but still some
2022-09-22 22:14:24 +00:00
Bryan Stitt
8d011e0cd1
pass db conn through
2022-09-22 22:10:28 +00:00
Bryan Stitt
3854312674
we need to clone :'(
2022-09-22 21:51:52 +00:00
Bryan Stitt
2ed2408ed3
even more authorized_request
2022-09-22 21:04:03 +00:00
Bryan Stitt
43846a7051
pass authorized_request through a bunch of places
2022-09-22 20:27:14 +00:00
Bryan Stitt
6905e9fd46
create a struct for authenticated requests that we need for per-key stats
2022-09-22 19:57:21 +00:00
Bryan Stitt
8481f6d44c
wip
2022-09-21 23:50:55 +00:00
Bryan Stitt
37a1aa554b
revert error saving and extract blockHash from requests
2022-09-21 04:48:21 +00:00
Bryan Stitt
6054c3f340
we do need fetch_sub
2022-09-21 02:48:02 +00:00
Bryan Stitt
abe5001792
atomic and less expires
2022-09-20 06:56:24 +00:00
Bryan Stitt
739947792a
instrument more. add max_wait to wait_for_request_handle
2022-09-20 06:00:27 +00:00
Bryan Stitt
6ae24b1ff9
use cache's expiration and race-free get_with
...
when this was a dashmap, we needed our own expiration and parallel requests would do the same query.
with moka, we can use their expiration code and get_with
2022-09-20 01:33:39 +00:00
Bryan Stitt
28fa424c2a
start weighers
2022-09-17 02:30:06 +00:00
Bryan Stitt
05af0551c9
use ahash
...
this might actually be slower. do real benchmarks
2022-09-17 02:17:20 +00:00
Bryan Stitt
12b6d01434
work in progress
2022-09-15 17:57:24 +00:00
Bryan Stitt
7a1433e3c9
no longer need to use total difficulty on ETH 2.0
2022-09-14 19:39:08 +00:00
Bryan Stitt
31227d8c0c
drop in flight. throughput is enough
2022-09-14 07:08:48 +00:00
Bryan Stitt
731a255d2f
add old block to log and more todos
2022-09-14 06:57:24 +00:00
Bryan Stitt
ea8f6091f9
improve (but disable) rate limits
2022-09-14 06:41:34 +00:00
Bryan Stitt
ed505cdd23
fix check for http providers
...
even the websocket providers have an http_client set. maybe drop that?
2022-09-14 06:38:53 +00:00
Bryan Stitt
1730b8c7a5
switch to deadpool
2022-09-14 06:18:13 +00:00
Bryan Stitt
789672be43
try fixing warning about missing blocks
2022-09-14 05:26:46 +00:00
Bryan Stitt
a5ccec76c7
silence some warnings and make private rpcs an option
2022-09-14 04:27:18 +00:00
Bryan Stitt
1d4d5844d0
reconnect -> connect
2022-09-14 03:32:38 +00:00
Bryan Stitt
612f5a48c1
minor polish to error
2022-09-14 03:01:58 +00:00
Bryan Stitt
5a16b9eed8
fix reconnect for http clients
2022-09-14 02:11:48 +00:00
Bryan Stitt
7eaf6f3540
retrying reconnect
2022-09-14 01:43:09 +00:00
Bryan Stitt
519c312e8d
reconnect needs to clear more state
2022-09-12 23:00:10 +00:00
Bryan Stitt
05e94ff81c
send UNAUTHORIZED to unknown keys
2022-09-12 14:33:55 +00:00
Bryan Stitt
7ff319e9b0
reconnect sooner
2022-09-12 14:33:19 +00:00
Bryan Stitt
501f2b0b24
improve logging
2022-09-10 00:58:33 +00:00
Bryan Stitt
879c6e49f2
dry errors so that rate limits dont log so much
2022-09-10 00:12:14 +00:00
Bryan Stitt
c8da98d12e
even more metrics
2022-09-09 03:53:16 +00:00
Bryan Stitt
cae034afb3
better metrics and spawn
2022-09-08 21:01:36 +00:00
Bryan Stitt
cfca16319b
lower uncle log level
2022-09-07 20:24:45 +00:00
Bryan Stitt
c9b2c0c0d2
dry cache code
2022-09-07 03:54:16 +00:00
Bryan Stitt
5eef5173a1
disable redis rate limits
2022-09-06 22:55:17 +00:00
Bryan Stitt
c34e8ef383
improve rate limiting and request counters
2022-09-06 20:13:07 +00:00
Bryan Stitt
041dd5f317
more logs and beginning of a health check
2022-09-06 16:49:07 +00:00
Bryan Stitt
83a7b03dea
improve no-op subscription
...
can still be improved more
2022-09-06 13:14:15 +00:00
Bryan Stitt
bb96757452
None instead of Block::default() more places
2022-09-06 12:29:37 +00:00
Bryan Stitt
e8977d203b
use None more
2022-09-06 03:26:23 +00:00
Bryan Stitt
4502344027
add head_block_id to connection serializer
2022-09-05 16:39:46 +00:00
Bryan Stitt
8080794a8b
skip dupes earlier
2022-09-05 16:29:21 +00:00
Bryan Stitt
068c05cf4f
improve fork logic again
2022-09-05 16:25:21 +00:00
Bryan Stitt
2092e74fd3
fix variable names and log to match
2022-09-05 15:49:53 +00:00
Bryan Stitt
a7cb3d00df
small cleanup
2022-09-05 06:13:36 +00:00
Bryan Stitt
6e12edd555
use sized Caches
2022-09-05 05:53:58 +00:00
Bryan Stitt
62d747c1b3
Vec instead of IndexSet
2022-09-05 03:40:13 +00:00
Bryan Stitt
593eb461b8
more dashmap
2022-09-05 01:52:59 +00:00
Bryan Stitt
8225285bb8
bug fix and better logs
2022-09-03 02:59:30 +00:00
Bryan Stitt
6f0ae1ec35
slower but correct save_block
2022-09-02 21:35:03 +00:00
Bryan Stitt
010669cf81
handle a missing block
2022-09-02 20:46:39 +00:00
Bryan Stitt
ac6296c5ac
better connection pool sizing
2022-09-02 20:16:20 +00:00
Bryan Stitt
2606844c61
almost done with heaviest chain tracking
2022-09-02 05:40:56 +00:00
Bryan Stitt
6450a4cd42
more fork detection work
2022-09-01 05:58:55 +00:00
Bryan Stitt
e5e137f76c
fix deadlock
2022-08-30 20:01:42 +00:00
Bryan Stitt
11c66636bb
rewrite cannonical block
2022-08-27 23:49:41 +00:00
Bryan Stitt
5e239c05c8
add min_sum_soft_limit to config
2022-08-27 03:13:23 +00:00
Bryan Stitt
8703532ed7
better logs and minor cleanup
2022-08-27 02:14:53 +00:00
Bryan Stitt
9fabb8e1e1
add min_synced_rpcs to config
2022-08-27 00:33:45 +00:00
Bryan Stitt
5719397466
i think it works
2022-08-26 23:44:25 +00:00
Bryan Stitt
81254a24be
wip
2022-08-26 17:26:17 +00:00
Bryan Stitt
04dc716250
move more things into their own files
2022-08-24 00:59:05 +00:00
Bryan Stitt
72312a686b
names instead of urls
...
urls might have secrets in them, and we dont want those in logs
2022-08-24 00:32:16 +00:00
Bryan Stitt
59eb9a889f
rename and lints
2022-08-24 00:14:49 +00:00
Bryan Stitt
2cc1b8e297
even more files
2022-08-24 00:11:49 +00:00
Bryan Stitt
4d357977e9
split Web3Connections across multiple files
2022-08-23 23:56:47 +00:00
Bryan Stitt
3496c828b8
rearrange code
2022-08-23 23:13:56 +00:00
