Commit Graph

523 Commits

Author SHA1 Message Date
Bryan Stitt
91a9d62ce1 pass head_block to internal_request 2023-10-11 00:48:08 -07:00
Bryan Stitt
57f8c4bd40 split web3_proxy/src/jsonrpc.rs into multiple files and DRY it up 2023-10-11 00:12:20 -07:00
Bryan Stitt
a9d5006ced get rid of async recursion 2023-10-10 23:32:50 -07:00
Bryan Stitt
ceaed4f239 move rpc filtering to later 2023-10-10 23:05:14 -07:00
Bryan Stitt
1666a45564 faster head_block_num 2023-10-10 21:30:55 -07:00
Bryan Stitt
86f70332be use web3_rpcs.blocks_by_hash.get instead of something that might make queries 2023-10-10 20:54:17 -07:00
Bryan Stitt
bf49b84eae skip outers earlier 2023-10-10 20:02:20 -07:00
Bryan Stitt
38d0c7f2cc don't wait for rpcs
come back to this later
2023-10-10 19:24:58 -07:00
Bryan Stitt
c27f0d065b try wait_for_sync a new way with a min sleep 2023-10-10 18:41:31 -07:00
Bryan Stitt
6906eee0dd wait_for_sync was waiting too long 2023-10-10 17:15:03 -07:00
Bryan Stitt
47749df94e catch one more rate limit type 2023-10-10 16:05:28 -07:00
Bryan Stitt
9ba2045dbd improve health checking 2023-10-10 15:46:11 -07:00
Bryan Stitt
a10a283023 trust just removing uncles from the cache 2023-10-10 14:21:54 -07:00
Bryan Stitt
78edfee6b9 timeout and server selection fixes
add a short connect timeout separate from the overall request timeout.

also fix a bug when only 1 server was in the rpc list causing a very tight loop that made tokio sad
2023-10-09 23:30:17 -07:00
Bryan Stitt
ab16f6b2e8 check for MDBX_PANIC 2023-10-09 09:38:58 -07:00
Bryan Stitt
d04f99a0e4 DRY 2023-10-07 16:52:54 -07:00
Bryan Stitt
36ce2f6524 derp 2023-10-07 16:16:42 -07:00
Bryan Stitt
7e0b6c3034 next_available should not be reversed and other bugs 2023-10-07 16:12:51 -07:00
Bryan Stitt
5a1a87a314 stream should count as success. lagged return the rpc 2023-10-07 15:18:55 -07:00
Bryan Stitt
580f4c1865 lint 2023-10-07 14:26:10 -07:00
Bryan Stitt
de5674254b hopefully handle min_synced_rpcs properly 2023-10-07 13:45:04 -07:00
Bryan Stitt
9ed0c70ecd wait for lagged rpcs. v1.43.19 2023-10-07 11:58:47 -07:00
Bryan Stitt
349be463f3 initial sort on median latency instead of weighted peak 2023-10-07 10:31:33 -07:00
Bryan Stitt
b2ab5bd6d6 fix from_rpcs 2023-10-07 10:06:01 -07:00
Bryan Stitt
cc2eb9e862 no more overflow on the pending txid firehose 2023-10-07 03:41:11 -07:00
Bryan Stitt
36363c8921 lint 2023-10-07 00:35:22 -07:00
Bryan Stitt
d38a35903e cargo update 2023-10-07 00:35:22 -07:00
Bryan Stitt
97f7e1673e finish todo for waiting for head_block_sender 2023-10-07 00:35:22 -07:00
Bryan Stitt
159fef9e8d include ws_url in hash 2023-10-07 00:35:22 -07:00
Bryan Stitt
f7997a4878 split inner and outer 2023-10-07 00:35:22 -07:00
Bryan Stitt
949c3eeb5d DRY throttling 2023-10-07 00:35:22 -07:00
Bryan Stitt
9a7777e3f9 fix >= to < 2023-10-07 00:35:22 -07:00
Bryan Stitt
8180290be7 chain isn't ExactSizeIterator 2023-10-07 00:35:22 -07:00
Bryan Stitt
c1df05f8b6 fix sort order (hopefully) 2023-10-07 00:35:22 -07:00
Bryan Stitt
0dcc324b61 disable tests that need refactor to match new code 2023-10-07 00:35:22 -07:00
Bryan Stitt
6756559ec2 wow this got big 2023-10-07 00:35:22 -07:00
Bryan Stitt
f704d1c4aa more refs will cut down on cloning an Arc multiple times that isn't really necessary 2023-10-07 00:35:22 -07:00
Bryan Stitt
376048a76b bump returndata limit to 16mb 2023-10-07 00:35:22 -07:00
Bryan Stitt
9258d36e6a refactor rpc selection 2023-10-07 00:35:22 -07:00
Bryan Stitt
32862937ad fix newHead serialization 2023-10-05 11:01:24 -07:00
Bryan Stitt
c1c127e6bd try_send_all_synced_connections for getting transactions 2023-10-03 21:35:25 -07:00
Bryan Stitt
e917a11d6c
Suprisingly large refactor to get ids everywhere (#222)
* cargo upgrade --incompatible and update

* first draft at suprisingly_large_refactor_to_get_ids_everywhere

* put app in a task_local

* ref cleanup

* use a OnceLock instead of a tokio local

* test more methods

* APP isn't set in all tests

* it compiles. tests fail. todos still open

* use the app only when necessary

* more tests. less panic

* less verbose debug impl

* short enum names

* move kafka and secrets to their own files

* main tests pass

* add debug chain block time

* helper for stats that ignores internal stats

* Update Jenkinsfile (#223)

* more tests

---------

Co-authored-by: Pewxz <124064710+pewxz@users.noreply.github.com>
2023-10-03 13:46:27 -07:00
Bryan Stitt
f0b6465069
streaming responses 2 (#219)
* streaming responses compile, still small TODOs & cleanup

* refactor to allow short curcuits on uncacheable requests (#220)

* refactor to allow short curcuits on uncacheable requests

* handle error inside an Arc from moka

* arc instead of box

* lint

* more lint

* add bonus rate limits and keep all semaphores after rate limits (#221)

* add bonus rate limits and keep all semaphores after rate limits

* remove stale todos

* better function names

* cargo upgrade and update

* make some panics warn. more todo

* pass request_metadata through so streaming responses include their final size

* remove stale TODO

* remove more stale todos

* add file i missed

* make to_json_string work well enough

* check config for free_subscriptions

---------

Co-authored-by: Rory Neithinger <rory@llamanodes.com>
2023-09-26 18:18:06 -07:00
Bryan Stitt
9b2f27862d remove unnecessary yield_nows 2023-09-21 11:22:20 -07:00
Bryan Stitt
f0be8b992b always wait for watch_ranked_rpcs change 2023-09-19 14:42:31 -07:00
Bryan Stitt
41314df25b move another yield_now 2023-09-18 23:42:54 -07:00
Bryan Stitt
fc287d24ea borrow_and_update instead of borrow
i think this won't matter and the update is probably a waste, but it feels wrong to leave it in a changed state
2023-09-18 23:29:01 -07:00
Bryan Stitt
a68a98ce41 cargo upgrade and update
moka 0.12 required some migration help: https://github.com/moka-rs/moka/blob/main/MIGRATION-GUIDE.md
2023-09-18 11:04:22 -07:00
Bryan Stitt
357dae108a loop, dont crash 2023-09-14 12:51:40 -07:00
Bryan Stitt
8e5a27a184 use Ordering::Relaxed more 2023-09-13 12:35:58 -07:00