Commit Graph

566 Commits

Author SHA1 Message Date
Bryan Stitt
c192e03fa2 count all the tiers at once 2023-06-12 21:03:38 -07:00
Bryan Stitt
bba7ccf7eb change divisor for tier calculation 2023-06-12 20:44:52 -07:00
Bryan Stitt
c771479d94 use WyRand instead of Xoshiro 2023-06-12 19:46:36 -07:00
Bryan Stitt
1a8f799969
we need our own Reconnect (#124)
* first pass at handling different return data limits

* put ws_provider in an arcswap

* add min max_latency

* add min max_latency

* subscribe with reconnect

* better logging around reconnect

* select on both watches

* subscribe to the correct watch
2023-06-12 18:58:51 -07:00
Bryan Stitt
f1176685dd pause time on more tests 2023-06-12 11:08:33 -07:00
Bryan Stitt
4a9c1a0ce6 move user registration into a dedicated function 2023-06-09 18:31:47 -07:00
Bryan Stitt
41950c886c improvements for web3-this-then-that 2023-06-09 16:35:14 -07:00
Bryan Stitt
bb900c61a5 todo 2023-06-09 13:31:46 -07:00
Bryan Stitt
8bf0ee473d loud log on rollback 2023-06-09 13:30:00 -07:00
Bryan Stitt
bd87fcb13c move sort and shuffle for loadbalancing into proper functions 2023-06-09 13:09:58 -07:00
Bryan Stitt
1f3040f6c7 automated tiers 2023-06-09 12:21:55 -07:00
Bryan Stitt
c37b320286 add cache names and actually serialize 2023-06-07 17:55:34 -07:00
Bryan Stitt
40bee30730 back to moka 2023-06-07 17:30:05 -07:00
Bryan Stitt
c2a3a4a0a9 Merge remote-tracking branch 'origin/main' into devel 2023-06-07 13:29:12 -07:00
Bryan Stitt
91d288315f add queue lengths to status page 2023-06-07 13:05:15 -07:00
Bryan Stitt
b50579ea3a add block caches to web3rpcs serializer 2023-06-07 12:04:39 -07:00
Bryan Stitt
aa57dd2fcf serialize the caches on /status 2023-06-07 10:58:30 -07:00
Bryan Stitt
f0a9f03076 smaller caches 2023-06-07 10:57:43 -07:00
Bryan Stitt
c4b2ce5266 smaller caches 2023-06-07 10:57:01 -07:00
Bryan Stitt
2b814e7a4d serialize the caches on /status 2023-06-07 10:48:55 -07:00
Bryan Stitt
7947cb95ff use bloom filters and support transactions with multiple deposit events 2023-05-31 11:20:17 -07:00
Bryan Stitt
fd8bba4bdd most of these are for the whole app, not just the frontend 2023-05-30 21:26:11 -07:00
Bryan Stitt
2273637a07 move cli into a separate packge and clean up dependencies 2023-05-30 20:58:24 -07:00
Bryan Stitt
cebe8ed1ea
Request refactor (#99)
* wip

* AsRef finally works like i wanted

* actually return the block

* start adding async trait

* remove stale import

* include id in the error response when possible

* remove stale comments
2023-05-30 16:32:34 -07:00
Bryan Stitt
c1eba556a5 fix missing ttl send and add name to cache 2023-05-29 17:19:05 -07:00
Bryan Stitt
bb50efb7f9
quick cache ttl max item size, better function names, and tests (#97)
* Revert "remove cache on /status and /health for now"

This reverts commit 166b0d810cd25b895560d50d00383a57e1269ad4.

* Revert "remove cache on backups needed, too"

This reverts commit 4597967def2f1daa71a277ec6731db4739ead640.

* add tests

* add max_item_weight

* comment

* add some helper functions, max weights, and tests
2023-05-29 15:48:22 -07:00
Bryan Stitt
b05d866f8a Merge remote-tracking branch 'origin/main' into devel 2023-05-29 09:42:21 -07:00
Bryan Stitt
dc6e0559f7 improve will_work_eventually 2023-05-29 09:41:19 -07:00
Bryan Stitt
84990bb7b6 check hard_limit_until when finding if an rpc will work 2023-05-29 09:41:19 -07:00
Bryan Stitt
08859ba04d wait for consensus rpcs to change before trying again 2023-05-29 09:41:19 -07:00
Bryan Stitt
7cb07cc49f debug -> trace for some verbose logs 2023-05-27 01:53:49 -07:00
Bryan Stitt
b1a0bcac57 add max wait to fix tests 2023-05-23 20:46:27 -07:00
Bryan Stitt
55f9c57827 include error in warning 2023-05-23 15:03:16 -07:00
Bryan Stitt
dd347fd916 add units to serialized variables 2023-05-23 14:57:24 -07:00
Bryan Stitt
77c9b0ab12 peak latency instead of head latency 2023-05-23 14:52:45 -07:00
Bryan Stitt
651f0fcad3 lint 2023-05-23 14:51:34 -07:00
Bryan Stitt
78f247fc6c split http and ws provider 2023-05-23 14:40:34 -07:00
Bryan Stitt
3ac15558e3 fix decay time 2023-05-22 15:43:39 -07:00
Bryan Stitt
91eeee23e2 use ether's reconnects instead of our own which need a lock (wip) 2023-05-22 15:32:27 -07:00
Bryan Stitt
c443f76623 refactor tiers again 2023-05-19 14:09:03 -07:00
Bryan Stitt
5aae318620 logs and comments 2023-05-18 15:11:29 -07:00
Bryan Stitt
9c584354d9 add to skip list earlier 2023-05-18 13:51:28 -07:00
Bryan Stitt
a92c93706b dont check heads while waiting
this isn't great. but should work better than what we have
2023-05-18 13:47:00 -07:00
Bryan Stitt
b61675e928 replace all moka with quick_cache_ttl 2023-05-18 13:34:22 -07:00
Bryan Stitt
1c61390ad1 check if watch_consensus_rpcs.changed has failed
i dont think it ever will, but better safe than sorry
2023-05-16 23:42:42 -07:00
Bryan Stitt
89961331af refactor wait_for_block into should_wait_for_block 2023-05-16 22:29:36 -07:00
Bryan Stitt
a07da30042 keep usable_rpcs_by_tier_and_head_number around longer 2023-05-16 18:32:33 -07:00
Bryan Stitt
0f79d2e60d more loud logs 2023-05-16 18:29:56 -07:00
Bryan Stitt
a0d8218eb1 best block num has to check has_block_data 2023-05-16 18:18:56 -07:00
Bryan Stitt
30ea532c70 display, not debug 2023-05-16 16:13:56 -07:00
Bryan Stitt
5d64524aa6 add name to web3rpcs 2023-05-16 16:04:17 -07:00
Bryan Stitt
978c385b3c improve wait for block 2023-05-16 14:02:37 -07:00
Bryan Stitt
c66eb6d864 head latency instead of peak latency for now 2023-05-16 12:18:59 -07:00
Bryan Stitt
2080739865
Quick cache (#70)
* quick cache and allocate less

* improve /status cache

* prepare to cache raw transaction hashes so we dont dos our backends

* simple benchmark for /health and /status

* mut not needed with atomics

* DRY all the status pages

* use u64 instead of bytes for subscriptions

* fix setting earliest_retry_at and improve logs

* Revert "use kanal instead of flume or tokio channels (#68)"

This reverts commit 510612d343fc51338a8a4282dcc229b50097835b.

* fix automatic retries

* put relaxed back

* convert error message time to seconds

* assert instead of debug_assert while we debug

* ns instead of seconds

* disable peak_latency for now

* null is the default

* cargo fmt

* comments

* remove request caching for now

* log on exit

* unit weigher for now

* make cache smaller. we need a weigher for prod. just debugging

* oops. we need async

* add todo

* no need for to_string on a RawValue
2023-05-13 11:13:02 -07:00
Bryan Stitt
c3cdf7ef43 fix consensus finding bug 2023-05-12 23:40:15 -07:00
Bryan Stitt
4b7f88367d fix serializing /status 2023-05-12 23:22:20 -07:00
Bryan Stitt
74a602bc9c wip 2023-05-12 23:20:59 -07:00
Bryan Stitt
a3b97fda9b lint 2023-05-12 23:04:56 -07:00
Bryan Stitt
c54970da0a change ordering and move fetch_add and fetch_sub 2023-05-12 23:00:03 -07:00
Bryan Stitt
914c3e03a8 remove stale comments 2023-05-12 22:22:21 -07:00
Bryan Stitt
3cea4c3123 update peak_ewma to use request latency and fix off by one 2023-05-12 22:21:11 -07:00
Bryan Stitt
510612d343
use kanal instead of flume or tokio channels (#68) 2023-05-12 15:47:01 -07:00
Bryan Stitt
8a097dabbe
Bryan devel 2023-05-12 (#67)
* add minor todo

* BadRequest instead of web3_context

* more bad request error codes

* use tokio-uring for the tcp listener

* clear block instead of panic

* clone earlier

* more watch channels instead of rwlocks

* drop uring for now (its single threaded) and combine get/post/put routes

* clean up iter vs into_iter and unnecessary collect

* arcswap instead of rwlock for Web3Rpcs.by_name

* cargo upgrade

* uuid fast-rng and alphabetize

* if protected rpcs, only use protected rpcs

* listenfd

* make connectinfo optional

* try_get_with_by_ref instead of try_get_with

* anyhow ensure. and try_get_with_as_ref isn't actually needed

* fix feature flags

* more refs and less clone

* automatic retry for eth_getTransactionReceipt and eth_getTransactionByHash

thanks for the report Lefteris @ Rotki

* ArcSwap for provider

* set archive_request to true on transaction retrying

* merge durable stats

* Revert "ArcSwap for provider"

This reverts commit 166d77f204cde9fa7722c0cefecbb27008749d47.

* comments

* less clones

* more refs

* fix test

* add optional mimalloc feature

* remove stale dependency

* sort

* cargo upgrade

* lint constants

* add todo

* another todo

* lint

* anyhow::ensure instead of panic

* allow rpc_accounting_v2 entries for requests without an rpc key
2023-05-12 15:15:32 -07:00
David
34ed450fab
User Balance + Referral Logic (#44)
* will implement balance topup endpoint

* will quickly fix other PR reviews

* merging from master

* will finish up godmoe

* will finish up login

* added logic to top up balance (first iteration)

* should implement additional columns soon (currency, amount, tx-hash), as well as a new table for spend

* updated migrations, will account for spend next

* get back to this later

* will merge PR from stats-v2

* stats v2

rebased all my commits and squashed them down to one

* cargo upgrade

* added migrtation for spend in accounting table. will run test-deposit next

* trying to get request from polygon

* first iteration /user/balance/:tx_hash works, needs to add accepted tokens next

* creating the referral code seems to work

* will now check if spending enough credits will lead to both parties receiving credits

* rpcstats takes care of accounting for spend data

* removed track spend from table

* Revert "removed track spend from table"

This reverts commit a50802d6ae75f786864c5ec42d0ceb2cb27124ed.

* Revert "rpcstats takes care of accounting for spend data"

This reverts commit 1cec728bf241e4cfd24351134637ed81c1a5a10b.

* removed rpc request table entity

* updated referral code to use ulid s

* credits used are aggregated

* added a bunch of fields to referrer

* added database logic whenever an aggregate stats is added. will have to iterate over this a couple times i think. go to (1) detecting accepted stables next, (2) fix influxdb bug and (3) start to write test

* removed track spend as this will occur in the database

* will first work on "balance", then referral. these should really be treated as two separate PRs (although already convoluted)

* balance logic initial commit

* breaking WIP, changing the RPC call logic functions

* will start testing next

* got rid of warnings & lint

* will proceed with subtracting / adding to balance

* added decimal points, balance tracking seems to work

* will beautify code a bit

* removed deprecated dependency, and added topic + deposit contract to app.yaml

* brownie test suite does not rely on local contract files it pulls all from polygonscan

* will continue with referral

* should perhaps (in a future revision) recordhow much the referees got for free. marking referrals seems to work rn

* user is upgraded to premium if they deposit more than 10$. we dont accept more than $10M in a single tx

* will start PR, referral seems to be fine so far, perhaps up to some numbers that still may need tweaking

* will start PR

* removed rogue comments, cleaned up payments a bit

* changes before PR

* apply stats

* added unique constraint

* some refactoring such that the user file is not too bloated

* compiling

* progress with subusers, creating a table entry seems to work

* good response type is there as well now, will work on getters from primary user and secondary user next

* subuser logic also seems fine now

* downgrade logic

* fixed bug influxdb does not support different types in same query (which makes sense)

* WIP temporary commit

* merging with PR

* Delete daemon.rs

there are multiple daemons now, so this was moved to `proxyd`

* will remove request clone to &mut

* multiple request handles for payment

* making requests still seem fine

* removed redundant commented out bits

* added deposit endpoint, added deposit amount and deposit user, untested yet

* small bug with downgrade tier id

* will add authorization so balance can be received for users

* balance history should be set now too

* will check balance over time again

* subususer can see rpc key balance if admin or owner

* stats also seems to work fine now with historical balance

* things seem to be building and working

* removed clone from  OpenRequestHandle

* removed influxdb from workspace members

* changed config files

* reran sea-orm generate entities, added a foreign key, should be proper now

* removed contract from commit

* made deposit contract optional

* added topic in polygon dev

* changed deposit contract to deposit factory contract

* added selfrelation on user_tier

* added payment required

* changed chain id to u64

* add wss in polygon llamarpc

* removed origin and method from the table

* added onchain transactions naming (and forgot to add a migration before)

* changed foreign key to be the referrer (id), not the code itself

* forgot to add id as the target foreign key

* WIP adding cache to update role

* fixed merge conflicts

---------

Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
Co-authored-by: Bryan Stitt <bryan@stitthappens.com>
2023-05-12 09:45:15 -07:00
Bryan Stitt
36cc884112 Merge remote-tracking branch 'origin/main' into devel 2023-05-11 20:05:47 -07:00
Bryan Stitt
fc8ca4ba4f less max lag 2023-05-11 17:04:33 -07:00
Bryan Stitt
6e8536d477 add support for http basic auth 2023-05-11 13:21:12 -07:00
Rory Trent
ec11e210ee
Peak ewma (#63)
* use peak-ewma instead of head for latency calculation

* Implement some suggested changes from PR

* move latency to new package in workspace root

* fix unit tests which now require peak_latency on Web3Rpc

* Switch to atomics for peak-ewma

This change is to avoid locking from tokio::sync::watch.

* add decay calculation to latency reads in peak-ewma

* Add some tests for peak-ewma

* Sensible latency defaults and not blocking on full

* Cleanup and a couple additional comments
2023-05-11 13:09:15 -07:00
Bryan Stitt
96a9a39789 add support for http basic auth 2023-05-11 13:07:38 -07:00
Bryan Stitt
0531e2f8dd add 'quota usage' rate limit substring 2023-05-08 11:25:27 -06:00
Bryan Stitt
ffb554d0b3 online editor isn't actually nice when it misses syntax errors :( 2023-04-29 18:50:26 -07:00
Bryan Stitt
91acbcaf75
Fix sort order 2023-04-29 18:38:31 -07:00
Bryan Stitt
1fab1b1d46 oops. lower this log level 2023-04-24 11:21:53 -07:00
Bryan Stitt
56fdf48129
Send transactions to protected and public rpcs (#57)
* move protected transactions into their own function and dry stats sending

* cargo upgrade

* comments

* time to live instead of time to idle

* minor workaround for eth_chainId

* cargo upgrade
2023-04-24 11:00:12 -07:00
Bryan Stitt
7a087415a2 hash instead of block_hash 2023-04-20 19:55:18 -07:00
Bryan Stitt
4889c3e1ce Merge branch 'main' into devel 2023-04-18 20:38:43 -07:00
Bryan Stitt
e921d02eb2 remove stale todo 2023-04-17 17:10:09 -07:00
Bryan Stitt
5ff28943da fix incorrect context 2023-04-14 03:44:32 -07:00
Bryan Stitt
2652f88f52 attach more contexts 2023-04-14 03:43:49 -07:00
Bryan Stitt
c2710858e1 only warn if rpc_configs is too short 2023-04-14 00:41:51 -07:00
Bryan Stitt
8ed71e1cf1 more goerli fixes 2023-04-14 00:36:46 -07:00
Bryan Stitt
3621d71037 if not watching heads, send to any server 2023-04-14 00:15:27 -07:00
Bryan Stitt
d035049c8f add /backups_needed endpoint for easy alerts 2023-04-13 17:15:01 -07:00
Bryan Stitt
b1120e635a change some error codes jsonrpc error handling 2023-04-10 22:29:02 -07:00
Bryan Stitt
b5ed0c4710 sometimes watch_for_block instead of returning an error 2023-04-10 22:28:31 -07:00
Bryan Stitt
ed5ca0575c logging improvements 2023-04-10 16:05:53 -07:00
Bryan Stitt
29c015508a fix warnings in cargo doc 2023-04-10 15:06:40 -07:00
Bryan Stitt
7250a0c155 upgrade ethers to 2.0.2 2023-04-10 15:01:36 -07:00
Bryan Stitt
b460f1701a rename revert_handler to error_handler 2023-04-05 15:34:28 -07:00
Bryan Stitt
7994c894bb cargo clippy and fmt 2023-04-05 15:15:20 -07:00
Bryan Stitt
1a160a8c7d Merge remote-tracking branch 'rorytrent/Web3ProxyError' into devel 2023-04-05 15:08:57 -07:00
yenicelik
0069e76040 latest changes from bryan 2023-04-04 14:40:22 +02:00
Rory Neithinger
f0d8a0c8c9 Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-04-01 00:04:44 -07:00
Rory Neithinger
ffdf25787f Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-03-31 23:23:30 -07:00
yenicelik
0bb3a2dc06 simple changes around bryans comments 2023-03-31 12:43:41 +01:00
yenicelik
345fa6c513 git status 2023-03-30 13:42:56 +01:00
yenicelik
53c7541fed seems to wait until everything is flushed. I will have to debug some stuff that isnt saved properly in the mysql (and then check influx) 2023-03-25 17:56:45 +01:00
Bryan Stitt
2205a472fd add optional tokio-console and improve reconnect logic 2023-03-22 18:43:13 -07:00
Bryan Stitt
7b223efa4d improve reconnect logic 2023-03-22 16:16:15 -07:00
Bryan Stitt
b4ca0fb62d better log when servers skipped 2023-03-22 15:19:09 -07:00
Bryan Stitt
a69737db98 serialize best_rpcs 2023-03-22 14:23:14 -07:00
Bryan Stitt
8eff48611c more direct consensus finding code
this hopefully has less bugs. speed isn't super important since this isn't on the host path.
2023-03-22 13:10:15 -07:00
Bryan Stitt
9fae137e45 shorten log message 2023-03-22 02:06:26 -07:00
Bryan Stitt
5465d927b4 remove a debug log 2023-03-21 21:22:16 -07:00
Bryan Stitt
d992bb2dc9 skip, don't error 2023-03-21 16:00:34 -07:00
Rory Neithinger
fe2a760c8a more conversions to Web3ProxyError 2023-03-20 13:45:21 -07:00
Rory Neithinger
beac7ee017 better error handling for proxy_web3_rpc() 2023-03-19 18:52:28 -07:00
Rory Neithinger
c32d12b5e0 better error handling for ip_is_authorized() 2023-03-19 15:50:25 -07:00
Rory Neithinger
f3fc4924dc Refactor FrontendErrorResponse into Web3ProxyError
Renamed FrontendResponse to Web3ProxyResponse and introduced
a new generic type alias Web3ProxyResult.

Fixed a few noisy cargo warnings.
2023-03-16 22:21:20 -07:00
Bryan Stitt
7b9bcf6881 include head block in the log 2023-03-09 11:22:40 -08:00
Bryan Stitt
df8cc3b954 fix log order 2023-03-07 16:16:32 -08:00
Bryan Stitt
6717999ced actually use backup rpcs 2023-03-07 16:02:22 -08:00
Bryan Stitt
dbd7860416 wip influxdb2 2023-03-04 20:54:04 +01:00
Bryan Stitt
eb4d05a520 stats v2
rebased all my commits and squashed them down to one
2023-03-04 20:48:49 +01:00
Bryan Stitt
36e4183373 todos 2023-03-03 11:19:25 -08:00
Bryan Stitt
94089eb72b combine comparison 2023-03-03 07:34:04 -08:00
Bryan Stitt
3b61e009f8 group cfgs 2023-03-02 17:51:41 -08:00
Bryan Stitt
3098791ad9 add optional kafka feature 2023-03-03 01:39:50 +00:00
Bryan Stitt
d1b955275d clean up wait_for_ functions 2023-03-02 18:27:32 +00:00
Bryan Stitt
907a147afa wait_for_provider helper function 2023-03-02 18:14:17 +00:00
Bryan Stitt
0d07e20d7c make it compile 2023-03-02 17:38:42 +00:00
Bryan Stitt
258e889f69 wait for a provider 2023-03-02 16:52:28 +00:00
Bryan Stitt
499d50b5fc handle empty private_rpcs 2023-03-01 23:21:09 +00:00
Bryan Stitt
c8bc0d288f high log level 2023-03-01 23:04:20 +00:00
Bryan Stitt
fdba8e7864 test all_connections with Nones 2023-03-01 22:50:44 +00:00
Bryan Stitt
f68886fafd add simple test for all connections and include more rpcs 2023-03-01 22:32:01 +00:00
Bryan Stitt
baf07f7de5 sync sort should include backup 2023-03-01 21:35:50 +00:00
Bryan Stitt
2e9803de72 remove arbitrary sleep timer 2023-03-01 20:56:00 +00:00
Bryan Stitt
b87c988439 lint 2023-03-01 11:23:59 -08:00
Bryan Stitt
20851af51b log seconds instead of milliseconds 2023-02-28 11:09:49 -08:00
Bryan Stitt
d83a1f0169 use Web3Rpc instead of the name as a key 2023-02-28 11:01:34 -08:00
Bryan Stitt
26970c2d88 move wait on new connection 2023-02-27 13:40:13 -08:00
Bryan Stitt
c5e2402d6e disconnect is noisy but it works 2023-02-27 13:29:07 -08:00
Bryan Stitt
6067369ee3 add inotify and rpc disconnect 2023-02-27 12:50:57 -08:00
Bryan Stitt
f8f5e7a1c8 add thread for config file watching and run clippy lint 2023-02-26 23:08:00 -08:00
Bryan Stitt
bf79d677b0 by_name needs to a lock 2023-02-26 20:00:13 -08:00
Bryan Stitt
e96f09a9c4 refactors to make configs partially reloadable 2023-02-25 23:52:33 -08:00
Bryan Stitt
c9e5661c5b wip 2023-02-25 09:55:28 -08:00
Bryan Stitt
580965ef83 get instead of remove 2023-02-24 23:31:10 -08:00
Bryan Stitt
91b853d21e archive instead of Some(u64::MAX) in the logs 2023-02-21 21:14:49 -08:00
Bryan Stitt
6c33b1ae1e block number instead of hash during health check 2023-02-21 20:25:54 -08:00
Bryan Stitt
738815244d change ewma calculation 2023-02-16 00:26:58 -08:00
Bryan Stitt
4916188d5c theres a perf regression in here somewhere 2023-02-15 21:54:07 -08:00
Bryan Stitt
cb2b5c16ad lower log level 2023-02-15 21:30:42 -08:00
Bryan Stitt
b7dfec9c63 fix params in health check query 2023-02-15 21:13:28 -08:00
Bryan Stitt
91cab8ffe2 comment 2023-02-15 21:09:39 -08:00
Bryan Stitt
bc306f62d4 make it work 2023-02-15 21:05:41 -08:00
Bryan Stitt
3c9576c13b yes health, no channel 2023-02-15 15:52:42 -08:00
Bryan Stitt
63adbc0587 remove health for now? 2023-02-15 15:42:27 -08:00
Bryan Stitt
268ad3a7c5 dont hold a lock open 2023-02-15 15:37:43 -08:00
Bryan Stitt
056670d29a health check and latency recordings 2023-02-15 15:31:59 -08:00
Bryan Stitt
774bd5b232 undo a rename in the json 2023-02-15 12:59:03 -08:00
Bryan Stitt
1fb4dd6ccc i think it works 2023-02-15 12:33:43 -08:00
Bryan Stitt
df668a5dfd add time to idle to more caches 2023-02-15 11:42:25 -08:00
Bryan Stitt
c65ffc9ce0 getting close 2023-02-14 17:41:40 -08:00
Bryan Stitt
f1e6de3677 fixed one test. still more to fix though 2023-02-14 12:41:05 -08:00
Bryan Stitt
824a6fa1f0 well it compiles. doesnt work yet. but it compiles 2023-02-14 12:14:50 -08:00
Bryan Stitt
70105bc7bb variable rename 2023-02-12 10:22:20 -08:00
Bryan Stitt
0e2e45cf90 cargo upgrade 2023-02-12 01:47:06 -08:00
Bryan Stitt
c008f50943 use http and ws providers in a single config 2023-02-12 01:40:01 -08:00
Bryan Stitt
f1bc00082a include to_block more places 2023-02-10 20:45:57 -08:00
Bryan Stitt
c959110986 better sorting of connections 2023-02-10 20:24:20 -08:00
Bryan Stitt
1a93e39b09 FuturesUnordered on the spawn handles 2023-02-09 11:56:07 -08:00
Bryan Stitt
6741cd72ac lint 2023-02-09 11:55:54 -08:00
Bryan Stitt
5ffe2aa72a eth_sendRawTransactions should use backups more often 2023-02-09 11:47:56 -08:00
Bryan Stitt
1efa8e7772 improve log message
we don't always get an empty block because we aren't synced. sometimes it is because we got disconnected
2023-02-08 11:30:16 -08:00
Bryan Stitt
0b1929a820 add minimal config for quick testing 2023-02-06 15:20:36 -08:00
Bryan Stitt
0edd11349f web3 rpc spawn should be unordered 2023-02-06 14:13:43 -08:00
Bryan Stitt
19e51ce328 check http servers twice as often 2023-02-06 13:48:50 -08:00
Bryan Stitt
0f280ce483 cargo upgrade and shorten variable names
also begin adding a latency tracker for rpc stats
2023-02-06 09:55:27 -08:00
Bryan Stitt
f2d35ba5eb remove metered in favor of influxdb stats 2023-02-05 18:17:26 -08:00
Bryan Stitt
6d959e2c1f add archive depth to app config 2023-02-05 17:58:03 -08:00
Bryan Stitt
37830f1156 Jenkinsfile from llamanodes/erigon 2023-02-05 00:13:14 -08:00
Bryan Stitt
ca1e550370 improve sort order during eth_sendRawTransaction 2023-02-02 14:48:23 -08:00
Bryan Stitt
cac31303f3 include backup in web3connection serializer 2023-02-02 08:00:59 -08:00
Bryan Stitt
a76a45f63a clone instead of take 2023-01-31 09:47:19 -08:00
Bryan Stitt
8bcf40b62d Revert "Revert "include from_block and to_block for caching getLogs""
This reverts commit 13bab2c9cfcc09f8d503ff24f6e2c2a3fc0faaa2.
2023-01-31 08:30:24 -08:00
Bryan Stitt
13bab2c9cf Revert "include from_block and to_block for caching getLogs"
This reverts commit 203969f62833668c4081acbb2188989fd6545c24.
2023-01-31 08:30:11 -08:00
Bryan Stitt
203969f628 include from_block and to_block for caching getLogs 2023-01-30 15:47:17 -08:00
Bryan Stitt
f2260ecdba don't require hard limit send 2023-01-25 10:01:36 -08:00
Bryan Stitt
cffc60e7f6 improve responses when blocks are not available 2023-01-24 22:45:20 -08:00
Bryan Stitt
694e552b5d improve waiting for sync when rate limited 2023-01-24 20:45:03 -08:00
Bryan Stitt
641d11a19b remove excess continues 2023-01-24 10:09:12 -08:00
Bryan Stitt
106dec294f better handling when method not available 2023-01-24 09:36:07 -08:00
Bryan Stitt
a242244a35 broadcast transactions to more servers 2023-01-24 01:58:31 -08:00
Bryan Stitt
47daab3b67 drop log level for backup servers 2023-01-23 21:37:23 -08:00
Bryan Stitt
86e3f2991f use watch instead of arcswap 2023-01-22 22:02:08 -08:00
Bryan Stitt
b9a12756b9 collect request_metadata.response_from_backup_rpc 2023-01-19 21:46:47 -08:00
Bryan Stitt
e53030e053 move backup indicator 2023-01-19 18:30:30 -08:00
Bryan Stitt
c3b53eb5f2 add backup indicator to more logs 2023-01-19 18:14:47 -08:00
Bryan Stitt
ad169eda0e log cleanup 2023-01-19 17:57:36 -08:00
Bryan Stitt
52a9ba604c remove allowed lag 2023-01-19 03:05:39 -08:00
Bryan Stitt
2cb6dde052 more log improvements 2023-01-19 02:26:54 -08:00
Bryan Stitt
76e51e3d11 better log level 2023-01-19 02:21:39 -08:00
Bryan Stitt
0c05b5bdee major refactor to only use backup servers when absolutely necessary 2023-01-19 02:20:06 -08:00
Bryan Stitt
90d3371eee improved rate limiting on websockets 2023-01-18 16:17:47 -08:00
Bryan Stitt
d7c75f843e add stub try_proxy_connection 2023-01-16 22:54:40 -08:00
Bryan Stitt
8c02e4420d remove weight now that we use tiers 2023-01-13 13:45:48 -08:00
Bryan Stitt
9f7d28b538 retry if we get the method X is not available 2023-01-12 22:40:47 -08:00
Bryan Stitt
e4f1716f06 broadcast txs to less servers 2023-01-11 14:55:41 -08:00
Bryan Stitt
e2d0369cd5 lower level for reconnect 2023-01-09 18:23:27 -08:00
Bryan Stitt
ec43d69462 fake weight for now 2023-01-04 14:33:39 -08:00
Bryan Stitt
664ecf5924 use None if on head block 2023-01-04 12:12:44 -08:00
Bryan Stitt
045065986a move allowed_lag around 2023-01-04 12:07:53 -08:00
Bryan Stitt
c0fc999e02 change weight to tier 2023-01-03 22:37:51 -08:00
Bryan Stitt
5a567ebeea try a new way to check block data limit if no longer syncing but limit is 0 2023-01-03 11:54:24 -08:00
Bryan Stitt
d5a17c914d lint 2023-01-03 11:38:06 -08:00
Bryan Stitt
1aefcf166d different polling for different chains 2023-01-03 11:37:42 -08:00
Bryan Stitt
5be5128c93 partial refactor of allowed lag 2023-01-03 08:33:49 -08:00
Bryan Stitt
43dd9628e6 Revert "Revert "Revert "self.check_block_data_limit when syncing changes"""
This reverts commit 36a33a8a1f1ed054a539736cc439e76bd2e7f945.
2023-01-03 07:51:18 -08:00
Bryan Stitt
36a33a8a1f Revert "Revert "self.check_block_data_limit when syncing changes""
This reverts commit ccb79522b6698625c2a00c71f8e71644008ca943.
2023-01-03 06:08:40 -08:00
Bryan Stitt
ccb79522b6 Revert "self.check_block_data_limit when syncing changes"
This reverts commit f3802a48357bd95d4e2b61e650341a458323aa45.
2023-01-03 05:26:21 -08:00
Bryan Stitt
f3802a4835 self.check_block_data_limit when syncing changes 2023-01-02 21:01:04 -08:00
Bryan Stitt
9b6b0ab7d2 remove extra brace 2023-01-02 10:42:27 -08:00
Bryan Stitt
206d339d45 all_backend_connections skips syncing servers 2023-01-02 10:34:16 -08:00
Bryan Stitt
6a201e1e47 don't set block_data_limit to 0 2022-12-27 18:17:11 -08:00
Bryan Stitt
4a837b35cc improve eth_sendRawTransaction 2022-12-23 17:35:35 -08:00
Bryan Stitt
e60b9dc3fe lower eth_getBlockByNumber level to Warn 2022-12-20 22:01:35 -08:00
Bryan Stitt
79075f9974 lowest log level for eth_sendRawTransaction
these are showing up in sentry with things like 'not enough funds'. its an error for the user to see, not us
2022-12-20 21:55:12 -08:00
Bryan Stitt
c0b457cec8 fix allow_not_ready 2022-12-20 21:47:41 -08:00
Bryan Stitt
f27c764a07 more logs 2022-12-19 15:59:01 -08:00
Bryan Stitt
2a19d9791b more logging 2022-12-19 13:53:38 -08:00
Bryan Stitt
f04905698a improve request caching 2022-12-16 20:05:01 -08:00
Bryan Stitt
4c2d3634c5 default to less detailed stats 2022-12-11 20:39:54 -08:00
Bryan Stitt
86f6b16761 another pass at server selection 2022-12-07 22:54:38 -08:00
Bryan Stitt
78a2119c07 quieter syncing nodes 2022-12-05 16:55:18 -08:00
Bryan Stitt
b02c851237 raise log level for block data limit 2022-12-05 16:47:27 -08:00
Bryan Stitt
c10a0e9d3c better error context 2022-12-05 16:35:51 -08:00
Bryan Stitt
a54a061bd2 handle null case 2022-12-05 16:18:31 -08:00
Bryan Stitt
33f7256236 clones to avoid deadlock 2022-12-05 16:06:28 -08:00
Bryan Stitt
f010166ee0 better lag error 2022-12-05 14:45:12 -08:00
Bryan Stitt
dc5c944545 rename head_block_id to head_block 2022-12-05 14:38:54 -08:00
Bryan Stitt
5bec8bb5b9 much smarter connection logic 2022-12-05 14:11:56 -08:00
Bryan Stitt
f736aeb027 move config safty checks earlier 2022-12-04 17:25:23 -08:00
Bryan Stitt
69f0125eba tests must include the current time 2022-12-04 17:10:20 -08:00
Bryan Stitt
0b93cb966e trailing commas 2022-12-02 21:55:46 -08:00
Bryan Stitt
9101c767e3 logs match 2022-12-02 21:49:41 -08:00
Bryan Stitt
b8ac77a342 skip nodes that are syncing 2022-12-02 21:38:19 -08:00
Bryan Stitt
eb4b487aae better block saving 2022-12-02 21:31:03 -08:00
Bryan Stitt
1b0cab9f54 better errors 2022-12-01 02:01:41 +00:00
Bryan Stitt
0d2816c487 add old block detection 2022-11-30 22:11:14 +00:00
Bryan Stitt
04d95f7234 allow unused imports for now 2022-11-29 21:34:42 +00:00
Bryan Stitt
519ba473d9 improve rpc filtering 2022-11-28 07:01:34 +00:00
Bryan Stitt
d06aa3b170 health compass cli 2022-11-28 06:10:34 +00:00
Bryan Stitt
40548bec4f lower log level 2022-11-26 04:57:25 +00:00
Bryan Stitt
f09d836dfe better weights 2022-11-25 07:41:53 +00:00
Bryan Stitt
4dfb7a5451 fix archive node detection when nodes used to be archive but than changed flags 2022-11-25 00:48:17 +00:00
Bryan Stitt
5b1621ead4 fix bug with not using synced_connections correctly 2022-11-22 22:45:22 +00:00
Bryan Stitt
87b0ecc916 test has_block_data for pruned node 2022-11-22 20:44:23 +00:00
Bryan Stitt
527218d229 test has_block_data for archive nodes 2022-11-22 20:23:08 +00:00