tornadocontrib/web3-proxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,147 Commits 1 Branch 0 Tags 12 MiB
c27f0d065b
Commit Graph

1448 Commits

Author SHA1 Message Date
Bryan Stitt
1a7d4c9cbb handle subscriptions and unimplemented 
this also means we handle new methods that haven't been added to our list as only 2 cu
 2023-06-26 21:10:32 -07:00
Bryan Stitt
8cb57e2f88 add EXTRA_RUST_LOG helper 2023-06-26 21:08:48 -07:00
Bryan Stitt
f68964f63d add 3 tries 2023-06-26 21:08:32 -07:00
Bryan Stitt
54107a390b cargo upgrade 2023-06-26 21:07:25 -07:00
Bryan Stitt
8815b5c8cd lint 2023-06-25 21:26:28 -07:00
yenicelik
627c359645 mixed up directions 2023-06-25 21:19:13 -04:00
yenicelik
bbdff1d916 some code got lost in all the commits, deploying again 2023-06-25 21:02:07 -04:00
David
0f4d12e2a5
Merge pull request #149 from yenicelik/devel 
Devel
 2023-06-25 17:55:55 -04:00
Bryan Stitt
0aa94d9c87 don't escape newlines 2023-06-25 14:34:19 -07:00
Bryan Stitt
fb76fd2d1c remove unused ewma crate now that we have our own 2023-06-25 14:32:45 -07:00
yenicelik
88617dc72b added more tracing for balance 2023-06-25 17:25:35 -04:00
yenicelik
32156e981d adding balances 2023-06-25 17:19:31 -04:00
yenicelik
992860a46d forgot to allow aggregate global stats 2023-06-25 16:18:09 -04:00
yenicelik
4c157cfcf7 made paymentrequired error code instead of introducing a new one 2023-06-25 15:05:27 -04:00
yenicelik
935681fde7 stats subuser + premium access logic 2023-06-25 14:29:54 -04:00
Bryan Stitt
fee1a423a1 simpler type when no args 2023-06-24 22:30:57 -07:00
Bryan Stitt
83123f5cdd wait up to a second for it to sync 2023-06-24 21:51:58 -07:00
Bryan Stitt
e7baeb1ba4 rearrange so maybe time works better 2023-06-24 21:41:18 -07:00
Bryan Stitt
c2917f1093 try sleeping 2023-06-24 21:35:36 -07:00
Bryan Stitt
ba702c7647 tests pass local, but jenkins isn't as fast 2023-06-24 21:27:11 -07:00
Bryan Stitt
b5ae11b7f7 lint 2023-06-24 21:21:51 -07:00
Bryan Stitt
a440d51377 test fixture 2023-06-24 21:17:08 -07:00
Bryan Stitt
8cfbbfe1b3 use default derive helpers 2023-06-24 20:39:46 -07:00
Bryan Stitt
efe1e30791 better handling of 0 port and fix test port 2023-06-24 20:38:06 -07:00
Bryan Stitt
ab3d4a91a8 json body instead of params 2023-06-24 18:17:51 -07:00
Bryan Stitt
0c10994835 why wont that import? 2023-06-24 16:45:07 -07:00
Bryan Stitt
94d0a25aaf test logs 2023-06-24 16:43:11 -07:00
Bryan Stitt
38b266156a consistent route name 2023-06-24 15:49:26 -07:00
Bryan Stitt
1d6e1ef9af polish error logs 2023-06-24 15:24:46 -07:00
Bryan Stitt
fef03f089f less verbose db_conn and db_replica helpers 2023-06-24 11:11:07 -07:00
Bryan Stitt
9115419ec0 handle addresses consistently 2023-06-24 10:20:24 -07:00
Bryan Stitt
39a505c0ff POST instead of GET and use more txs 2023-06-24 09:59:08 -07:00
Bryan Stitt
8f76d9320d support EIP1271 login 2023-06-24 09:48:31 -07:00
Bryan Stitt
91cbce6ce8 start adding stat for CU 2023-06-24 00:48:56 -07:00
Bryan Stitt
4e6a80caa2 cargo upgrade 2023-06-24 00:41:48 -07:00
Bryan Stitt
30828ccae2 prioritize block. queries to syncing nodes ends up being slower 2023-06-24 00:41:30 -07:00
Bryan Stitt
ac4b7e46b4 actually import info level 2023-06-23 22:54:02 -07:00
Bryan Stitt
6edf1a5c1b info on new block 2023-06-23 22:52:42 -07:00
Bryan Stitt
89330cfba0 upstream ethers again 2023-06-23 22:23:51 -07:00
Bryan Stitt
fd661689a3
Tracing (#144) 
* tracing

* add tracing and features to match

* use just one hostname crate

* cargo upgrade

* set up pretty logger and sentry layer
 2023-06-23 16:28:45 -07:00
Bryan Stitt
3fac4248d6 Revert "just use Ulid instead of RpcSecretKey type" 
It did work

This reverts commit 377e8356a57d80dd17b746dd05a62b475d9e7d07.
 2023-06-23 01:29:11 -07:00
Bryan Stitt
377e8356a5 just use Ulid instead of RpcSecretKey type 2023-06-23 01:28:47 -07:00
Bryan Stitt
68ab9b6d1d always serialize RpcSecretKey as a ULID 2023-06-23 01:07:54 -07:00
Bryan Stitt
6b6fac396f lower default web3_proxy::rpcs::consensus level 2023-06-22 22:45:43 -07:00
Bryan Stitt
b1b8346db9
#123 role to rpc keys (#142) 
* added roles to rpc keys

* dont unwrap

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
 2023-06-22 12:56:19 -07:00
Bryan Stitt
c79b787e13
allow setting referral code if not already set (#141) 2023-06-22 12:41:05 -07:00
Bryan Stitt
376b9f474d use short syntax for json data in logs 2023-06-22 12:10:23 -07:00
Bryan Stitt
f593667169 fix lifetimes 2023-06-22 10:16:58 -07:00
Bryan Stitt
66212b5cf5 cargo upgrade --incompatible -p itertools 2023-06-22 09:26:57 -07:00
Bryan Stitt
dffff65950 allow no referrer_record 2023-06-22 09:25:12 -07:00
Bryan Stitt
c560a59ef0 add more headers for debug endpoints and use more refs 2023-06-21 22:11:26 -07:00
Bryan Stitt
095a505017 cargo upgrade 2023-06-21 17:57:53 -07:00
Bryan Stitt
662bbb296f add header debug endpoint 2023-06-21 11:28:33 -07:00
Bryan Stitt
63c4f9e591 allow no ip 
this shouldn't be needed, but is a workaround while we debug staging
 2023-06-21 11:28:22 -07:00
Bryan Stitt
7f4da8d43c lint 2023-06-21 10:40:55 -07:00
Bryan Stitt
d962b749a2 use find_with_related, not find_also_related 
find_also_related is 1:1, but we want 1:Many
 2023-06-21 10:37:01 -07:00
Bryan Stitt
1cbb58e8e0 comment 2023-06-21 10:25:10 -07:00
Bryan Stitt
06ce95c386 let serde_json do more of the work 2023-06-21 10:18:36 -07:00
Bryan Stitt
5504062f47 dont error if no referrals 2023-06-21 10:18:06 -07:00
Bryan Stitt
8f1b6fe6c0 lint 2023-06-21 10:02:02 -07:00
Bryan Stitt
4dad743fc3 tuple, not list 2023-06-21 10:01:51 -07:00
Bryan Stitt
f75f3f8d2f serialize caches in a list 2023-06-21 10:00:05 -07:00
Bryan Stitt
3c2f95de10
Improve max wait and timeouts (#137) 
* pass max wait with more functions

* move rpc_head_str higher up to use it in all the logs

* refresh consensus at double the block time

* new pricing that also includes archive check

* lint

* don't clone method

* put caches into one list

they serialize with their name, so it will still be easy to tell them apart

* more consistent waiting. still waits more than I want right now

* clean up default max waits

* more timeouts
 2023-06-20 14:22:14 -07:00
Bryan Stitt
78efad5453 cargo upgrade strum 2023-06-19 17:48:46 -07:00
Bryan Stitt
6d25c41faf
Compute units (#136) 
* wip

* add basic compute unit calculator

* calculate costs with compute units
 2023-06-19 17:47:38 -07:00
Bryan Stitt
b7935f77fc spawn recording latency 2023-06-19 17:46:58 -07:00
Bryan Stitt
6038351cb8 change default tracking in prep for premium-only keys 2023-06-19 13:00:57 -07:00
Bryan Stitt
b35cd58a76 rate limit user_balance_post by bearer OR ip 2023-06-19 10:21:48 -07:00
Bryan Stitt
c3ae4ded2b move data to the right places 2023-06-18 21:42:16 -07:00
Bryan Stitt
58cc129837 forgot to convert tier_sec_size from ms. 1 was a very large default 2023-06-18 17:57:21 -07:00
Bryan Stitt
ba410cf126 better log message 2023-06-18 16:41:01 -07:00
Bryan Stitt
89009dcd42 use web3_context for more errors 2023-06-18 16:40:17 -07:00
Bryan Stitt
af8543c3bf some defaults are needed maybe? 2023-06-18 16:36:23 -07:00
Bryan Stitt
2f78e64c5d create empty balance entry if none exists 2023-06-18 16:25:25 -07:00
Bryan Stitt
6deeabe1e1 pretty print errors 2023-06-18 16:14:06 -07:00
Bryan Stitt
9dd0bae40a text instead of string payload 2023-06-18 16:13:55 -07:00
Bryan Stitt
825ba006f1 move floats and durations around for latency tracking 2023-06-18 10:46:22 -07:00
Bryan Stitt
2f0e6103ac Revert "tier more important than backup" 
This reverts commit bca22537c633822e5e6759fa2840712a837211fd.
 2023-06-18 10:09:21 -07:00
Bryan Stitt
bca22537c6 tier more important than backup 
we filter backups when they aren't needed, so this should be okay
 2023-06-18 10:06:26 -07:00
Bryan Stitt
e54299beca rolling median for choosing tier 2023-06-18 09:53:36 -07:00
Bryan Stitt
8b22c9861b Revert "temp fix while filtering is improved" 
This reverts commit 5367c511953fcea1af3da1d905de06d9e76711ba.
 2023-06-17 12:15:40 -07:00
Bryan Stitt
5367c51195 temp fix while filtering is improved 2023-06-17 12:09:04 -07:00
Bryan Stitt
3b65074f38 cleaner duration multiplication 2023-06-17 11:33:33 -07:00
Bryan Stitt
eab86c7ee8 automatic max_head_block_age 2023-06-17 10:13:40 -07:00
Bryan Stitt
bf13e86a04 warn if no max_head_block_age 2023-06-17 09:19:05 -07:00
Bryan Stitt
3b9314a76f comments 2023-06-17 08:44:18 -07:00
Bryan Stitt
9b9e5341cb lint 2023-06-17 08:44:07 -07:00
Bryan Stitt
6bfa5a69ca latest_balance is part of AuthorizationChecks now 2023-06-17 08:43:51 -07:00
Bryan Stitt
8ffd7d2e1b enable more advanced cost calculation 2023-06-16 23:55:41 -07:00
Bryan Stitt
ba67151706 lint 2023-06-16 23:48:16 -07:00
Bryan Stitt
745870d119 drop a bunch of log levels 2023-06-16 23:46:20 -07:00
Bryan Stitt
74d2333039 try this version of influxdb2 2023-06-16 23:41:12 -07:00
David
d6513cde74
Forgot to handle Option unwrap (#132) 
* made the query much faster

* aggregate queries should be fast now too

* forgot to handle an option unwrap
 2023-06-16 23:18:51 -07:00
Bryan Stitt
86f9e7358a
move balance onto AuthorizationChecks (#131) 
* move balance onto AuthorizationChecks

* todone

* bigger default

* flat cost for testing
 2023-06-16 23:14:43 -07:00
Bryan Stitt
5d9365449f simplify max item weight by having the weigher send MAX for big things 2023-06-16 16:14:11 -07:00
Bryan Stitt
1ec0f14144 upgrade to influxdb 1e5577e14150797584f5ed0ea7aba0bd68f0a678 2023-06-16 14:48:03 -07:00
Bryan Stitt
cb71de7d14 pin influxdb2 to 6d3cf64d61967f41fafb57b3f4553206a8f3f2a3 2023-06-16 13:30:35 -07:00
Bryan Stitt
d9a865fd1b cargo upgrade 2023-06-16 10:58:09 -07:00
Bryan Stitt
3928496be5 start tiers at 1 2023-06-16 10:52:13 -07:00
Bryan Stitt
af7759e491 by_name needs a lock to prevent races during disconnects 2023-06-16 10:40:02 -07:00
Bryan Stitt
7c876ac561 delete stale code 
this was rewritten and put into its own crate
 2023-06-16 10:02:14 -07:00
Bryan Stitt
7707729f77 Merge remote-tracking branch 'origin/main' into devel 2023-06-16 10:01:17 -07:00
David
4772166e97
Make aggregate queries faster as well (#130) 
* made the query much faster

* aggregate queries should be fast now too
 2023-06-16 08:27:46 -07:00
Bryan Stitt
fe8675ddf2 cargo upgrade 2023-06-16 00:48:22 -07:00
Bryan Stitt
a083bc652d
Handle uncle transactions (#129) 
* add more columns to handle uncled transactions

* handle payment uncles

* put relations back

* include all the new columns

* lower log levels

* improve block caching

if we have a block with a number, its canonical. uncles don't get returned

* improve disconnect logic

* lint

* clear first changed for new_top_config_receiver

* better logs around config changing

* i guess we do want one apply top_config at the start

* check correct variable for data limits
 2023-06-16 00:46:27 -07:00
Bryan Stitt
6040ca297f fix eth_unsubscribe 2023-06-15 11:43:24 -07:00
Bryan Stitt
49c60ac1b5 improve websocket error handling 2023-06-15 09:50:21 -07:00
David
5859cd8a8d
made the query much faster (#127) 2023-06-15 09:38:51 -07:00
Bryan Stitt
957ba161b5 move table data around 2023-06-13 12:08:47 -07:00
Bryan Stitt
b8f429d70a improvements to popularity_contesst 2023-06-13 11:51:19 -07:00
Bryan Stitt
f5a1ac274a refactor send_head_block_result 2023-06-13 10:00:08 -07:00
Bryan Stitt
1473ccb085 gentler parsing of new head blocks 2023-06-13 09:31:22 -07:00
Bryan Stitt
602b0117c4 panic if influxdb_bucket is not set when influxdb_host is set 2023-06-13 09:30:41 -07:00
Bryan Stitt
45543be256 add more caches to /status 2023-06-12 23:03:38 -07:00
Bryan Stitt
a7fb12bd72 less random is fine for shuffling 2023-06-12 22:42:22 -07:00
Bryan Stitt
55e8471b19 interval instead of watch_blocks 2023-06-12 22:26:10 -07:00
Bryan Stitt
6324d9c7d9 short circuit if no requests 2023-06-12 21:54:33 -07:00
Bryan Stitt
18835e09da allow empty batches 2023-06-12 21:51:08 -07:00
Bryan Stitt
c192e03fa2 count all the tiers at once 2023-06-12 21:03:38 -07:00
Bryan Stitt
bba7ccf7eb change divisor for tier calculation 2023-06-12 20:44:52 -07:00
Bryan Stitt
c771479d94 use WyRand instead of Xoshiro 2023-06-12 19:46:36 -07:00
Bryan Stitt
1a8f799969
we need our own Reconnect (#124) 
* first pass at handling different return data limits

* put ws_provider in an arcswap

* add min max_latency

* add min max_latency

* subscribe with reconnect

* better logging around reconnect

* select on both watches

* subscribe to the correct watch
 2023-06-12 18:58:51 -07:00
Bryan Stitt
62bda636b8 return the response instead of panicing 2023-06-12 16:38:57 -07:00
Bryan Stitt
c456432cc3 cargo upgrade 2023-06-12 11:44:51 -07:00
Bryan Stitt
c3a7d1646c lint 2023-06-12 11:41:10 -07:00
Bryan Stitt
70d5c09d3e clean up redirect url 2023-06-12 11:38:34 -07:00
Bryan Stitt
f1176685dd pause time on more tests 2023-06-12 11:08:33 -07:00
David
093ca19454
added subuser RPC keys into key access control, as well as fetching RPC keys (#115) 2023-06-12 08:59:23 -07:00
David
0cc4557e8d
Use rpc id instead of key (#118) 
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.

* Revert "added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc."

This reverts commit 022a13cfd9ca24aad9e05f75489396936f14784a.

* mixed up branches
 2023-06-11 19:14:02 -07:00
David
81aa4788b4
David/60 add basic email verification (#119) 
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.

* added rpc key logic to use db numbers instead of UUIDs

* Revert "added rpc key logic to use db numbers instead of UUIDs"

This reverts commit 056771d02725f90ed5f1745b0372e13e630a19ac.
 2023-06-11 19:10:10 -07:00
yenicelik
0b26ae3bad made modify-subuser post 2023-06-11 18:59:27 +02:00
yenicelik
2333b8bbb6 influx query is not 20-40% faster 2023-06-11 17:57:02 +02:00
Bryan Stitt
4a9c1a0ce6 move user registration into a dedicated function 2023-06-09 18:31:47 -07:00
Bryan Stitt
71d3d63524 put ip rate limiting back 2023-06-09 16:35:44 -07:00
Bryan Stitt
41950c886c improvements for web3-this-then-that 2023-06-09 16:35:14 -07:00
Bryan Stitt
bcf4f33543 add payment_factory_address to /status 2023-06-09 14:02:29 -07:00
Bryan Stitt
bb900c61a5 todo 2023-06-09 13:31:46 -07:00
Bryan Stitt
8bf0ee473d loud log on rollback 2023-06-09 13:30:00 -07:00
Bryan Stitt
bd87fcb13c move sort and shuffle for loadbalancing into proper functions 2023-06-09 13:09:58 -07:00
Bryan Stitt
1f3040f6c7 automated tiers 2023-06-09 12:21:55 -07:00
Bryan Stitt
b6ff14210d cargo upgrade 2023-06-08 22:35:14 -07:00
David
d91713e09e
this brings back balance back to the stats request (#112) 
* removed bloom filter temporarily, added some fixes with decimals in payment.rs

* balance is inside the influx query again

* fixed aggregate
 2023-06-08 15:57:53 -07:00
Bryan Stitt
68f73ec0b1 more things should be BadRequest instead of 500 2023-06-08 13:42:45 -07:00
Bryan Stitt
50ec653557 lower log level 2023-06-08 11:46:38 -07:00
Bryan Stitt
74280bc329 lint 2023-06-08 11:33:16 -07:00
Bryan Stitt
334b1c21b3 improve logs 2023-06-08 11:30:48 -07:00
Bryan Stitt
1d05bc5e89 lower log levels 2023-06-08 11:26:09 -07:00
David
3f76b08364
Test balance and referral accounting (#111) 
* removed bloom filter temporarily, added some fixes with decimals in payment.rs

* balance accounting for a single user (paid and free tier) seems to be ok

* fixed some balance accounting

* compute_cost reduced back to 0
 2023-06-08 10:08:29 -07:00
Bryan Stitt
c37b320286 add cache names and actually serialize 2023-06-07 17:55:34 -07:00
Bryan Stitt
5744b459ee lint 2023-06-07 17:43:10 -07:00
Bryan Stitt
6cf738261b fix test 2023-06-07 17:42:34 -07:00
Bryan Stitt
bfaff81626 cargo upgrade 2023-06-07 17:31:29 -07:00
Bryan Stitt
5c1569f9c2 clippy lint 2023-06-07 17:31:15 -07:00
Bryan Stitt
40bee30730 back to moka 2023-06-07 17:30:05 -07:00
David
de7d8919d3
Change balance to record total spend and total deposits (#109) 
* lets test total deposit and total spent

* removed referrer from cache for performance reasons
 2023-06-07 14:45:57 -07:00
Bryan Stitt
4f7144abc6 Arc instead of Box for better caching 2023-06-07 13:57:38 -07:00
Bryan Stitt
c2a3a4a0a9 Merge remote-tracking branch 'origin/main' into devel 2023-06-07 13:29:12 -07:00
Bryan Stitt
91d288315f add queue lengths to status page 2023-06-07 13:05:15 -07:00
Bryan Stitt
b50579ea3a add block caches to web3rpcs serializer 2023-06-07 12:04:39 -07:00
Bryan Stitt
8249b0aefe cache isnt on main yet 2023-06-07 11:18:36 -07:00
Bryan Stitt
aa57dd2fcf serialize the caches on /status 2023-06-07 10:58:30 -07:00
Bryan Stitt
f0a9f03076 smaller caches 2023-06-07 10:57:43 -07:00
Bryan Stitt
c4b2ce5266 smaller caches 2023-06-07 10:57:01 -07:00
Bryan Stitt
fb97ef5180 cargo clippy lints 2023-06-07 10:55:07 -07:00
Bryan Stitt
f6b2b0deab cargo upgrade 2023-06-07 10:50:34 -07:00
Bryan Stitt
2b814e7a4d serialize the caches on /status 2023-06-07 10:48:55 -07:00
David
63499c1564
David/66 downgrade balance logic (#89) 
* tried to make the balance_entry atomic, asking for feedback

* added locked select everywhere and inside the txn, it should be atomic now

* forgot about overwrite (referee vs sender entities were the same, overwriting each other), fixed it

* will now trigger refreshing cache when balance thresholds are met

* will also invalidate cache if balance gets too low (to out of funds)

* added separate balance cache with shorter TTL. need to know if i should modify it (rn its not atomic), or if low TTL is good enough, and we just fetch the value every now and then from the Database

* removed UserTier struct bcs duplicate with database

* removed subuser premium requirement (which would have lead to a merge conflict later on)

* removed user_tier artefact

* replaced cache to use AtomicF64

* a bunch of small changes

* some changes

* will merge with devel

* changed AtomicF64 to RwLock<Decimal>

* changed AtomicF64 to RwLock<Decimal>

* downgrading user when at 0.1$ or if sum_credits_used is very low

* changed caches to be more aggressive in being non-empty

* replaced Arc::clone() by reference
 2023-06-07 09:39:30 -07:00
David
fd69e6acdd
added stats page for referrals (shared, and received) (#95) 
* added stats page for referrals (shared, and received)

* removed referrals address, and return ok response if no referrals used

* merged from devel

* changed unwrap for context

* changes from PR
 2023-06-07 09:38:19 -07:00
Bryan Stitt
172baabb4e even more fields onto popularity contest 2023-06-05 16:40:49 -07:00
Bryan Stitt
640bcd1378 update popularity_contest script to match new status page 2023-06-05 16:40:49 -07:00
David
f66edd961b
Making Balance Changes Atomic (Read and Write) (#84) 
* tried to make the balance_entry atomic, asking for feedback

* added locked select everywhere and inside the txn, it should be atomic now

* forgot about overwrite (referee vs sender entities were the same, overwriting each other), fixed it

* will now trigger refreshing cache when balance thresholds are met

* Revert "will now trigger refreshing cache when balance thresholds are met"

This reverts commit b1215ddc43a1358f6086bddbd106996165431165.

* will not modify userTier in database anymore

* made payment atomic

* made updates mostly atomic, locking only the referral table (bcs more complex logic)

* latest changes
 2023-06-04 09:32:53 -07:00
Bryan Stitt
79e52f4cdb Merge remote-tracking branch 'origin/main' into devel 2023-05-31 17:12:59 -07:00
Bryan Stitt
633bf1f53b need a decimal 0 2023-05-31 17:12:32 -07:00
Bryan Stitt
cb580d4ea8 Merge remote-tracking branch 'origin/main' into devel 2023-05-31 17:02:34 -07:00
Bryan Stitt
1a26260351 if no credits used, return early 2023-05-31 17:02:24 -07:00
Bryan Stitt
8709737d74 lower log level 2023-05-31 16:07:13 -07:00
Bryan Stitt
fbe0ecfbff rate limit on websockets 2023-05-31 16:05:44 -07:00
Bryan Stitt
cadab50692 lint 2023-05-31 15:54:32 -07:00
Bryan Stitt
56ec9439c3 insert on conflict update for balance table 2023-05-31 14:07:25 -07:00
David
88526c3702
Missing subuser logic (#96) 
* will push it for feedback loop

* filtering also to only show rpc keys that the user is an owner of

* revert logs also returning rpc keys

* merged with devel
 2023-05-31 13:46:59 -07:00
Bryan Stitt
62997774b1 use abigen types for decoding the logs 2023-05-31 12:44:02 -07:00
Bryan Stitt
7dcc2b4d93 use MultiAbigen 2023-05-31 11:52:57 -07:00
Bryan Stitt
7947cb95ff use bloom filters and support transactions with multiple deposit events 2023-05-31 11:20:17 -07:00
Bryan Stitt
b9f0824dfe start using the internal provider 2023-05-30 23:52:12 -07:00
Bryan Stitt
d64761403b Cow instead of String 2023-05-30 23:17:05 -07:00
Bryan Stitt
f1636d3b85 unlimited localhost 2023-05-30 22:31:35 -07:00
Bryan Stitt
fcc1843af0 put it back. duplication between cargo.toml isnt worth it 2023-05-30 21:37:58 -07:00
Bryan Stitt
fd8bba4bdd most of these are for the whole app, not just the frontend 2023-05-30 21:26:11 -07:00
Bryan Stitt
0b62ffcdfc no need to move the semaphore around like this 2023-05-30 21:08:21 -07:00
Bryan Stitt
2273637a07 move cli into a separate packge and clean up dependencies 2023-05-30 20:58:24 -07:00
Bryan Stitt
cebe8ed1ea
Request refactor (#99) 
* wip

* AsRef finally works like i wanted

* actually return the block

* start adding async trait

* remove stale import

* include id in the error response when possible

* remove stale comments
 2023-05-30 16:32:34 -07:00
Bryan Stitt
4246342806 lower log levels 2023-05-29 17:26:32 -07:00
Bryan Stitt
c1eba556a5 fix missing ttl send and add name to cache 2023-05-29 17:19:05 -07:00
Bryan Stitt
bb50efb7f9
quick cache ttl max item size, better function names, and tests (#97) 
* Revert "remove cache on /status and /health for now"

This reverts commit 166b0d810cd25b895560d50d00383a57e1269ad4.

* Revert "remove cache on backups needed, too"

This reverts commit 4597967def2f1daa71a277ec6731db4739ead640.

* add tests

* add max_item_weight

* comment

* add some helper functions, max weights, and tests
 2023-05-29 15:48:22 -07:00
David
5f0c99530c
referral code does not have llamanodes- prefix any longer (#90) 2023-05-29 14:41:15 -07:00
Bryan Stitt
9e935e2c44 cargo upgrade 2023-05-29 09:43:51 -07:00
Bryan Stitt
b05d866f8a Merge remote-tracking branch 'origin/main' into devel 2023-05-29 09:42:21 -07:00
Bryan Stitt
a275e2dee4 add blocktime for binance 2023-05-29 09:41:19 -07:00
Bryan Stitt
dc6e0559f7 improve will_work_eventually 2023-05-29 09:41:19 -07:00
Bryan Stitt
84990bb7b6 check hard_limit_until when finding if an rpc will work 2023-05-29 09:41:19 -07:00
Bryan Stitt
08859ba04d wait for consensus rpcs to change before trying again 2023-05-29 09:41:19 -07:00
David
48905f0235
David/76 bring back docs (#91) 
* bring back docs

* added some more rudimentary documentation
 2023-05-29 09:24:41 -07:00
David
ccb3a559ad
removed requirement for subuser logic to be admin (#94) 2023-05-29 08:35:24 -07:00
Bryan Stitt
4597967def remove cache on backups needed, too 2023-05-28 09:39:24 -07:00
Bryan Stitt
166b0d810c remove cache on /status and /health for now 2023-05-28 09:38:51 -07:00
Bryan Stitt
7cb07cc49f debug -> trace for some verbose logs 2023-05-27 01:53:49 -07:00
Bryan Stitt
0e7bd1fbf3 add missing anyhow::Context 2023-05-26 17:00:39 -07:00
Bryan Stitt
4aa34740a8 default balance to 0 2023-05-26 16:54:52 -07:00
David
9a5e9d46a9
Merge branch 'devel' into david/devel/debug-graph 2023-05-24 13:36:45 +02:00
yenicelik
b6cab88c4e removed joined_on variable that was not used 2023-05-24 13:35:10 +02:00
yenicelik
47f07261ed detailed request must come with a bearer token 2023-05-24 13:34:50 +02:00
Bryan Stitt
0d15d5baf7 bump versions to match number of migrations 2023-05-23 20:58:29 -07:00
Bryan Stitt
b1a0bcac57 add max wait to fix tests 2023-05-23 20:46:27 -07:00
Bryan Stitt
dafb69fae1 move timeout deeper 2023-05-23 15:50:29 -07:00
Bryan Stitt
55f9c57827 include error in warning 2023-05-23 15:03:16 -07:00
Bryan Stitt
4c7a7b87cb cargo upgrade 2023-05-23 14:58:13 -07:00
Bryan Stitt
dd347fd916 add units to serialized variables 2023-05-23 14:57:24 -07:00
Bryan Stitt
77c9b0ab12 peak latency instead of head latency 2023-05-23 14:52:45 -07:00
Bryan Stitt
651f0fcad3 lint 2023-05-23 14:51:34 -07:00
Bryan Stitt
d83e2f7015 Merge remote-tracking branch 'origin/devel' into devel 2023-05-23 14:44:40 -07:00
Bryan Stitt
78f247fc6c split http and ws provider 2023-05-23 14:40:34 -07:00
David
27333a0028
David/devel/debug graph (#82) 
* now should also group by archive_needed (turns out this was wanted)

* influxdb query also groups by archive_needed now

* updated error_response

* updated error_response
 2023-05-22 17:08:35 -07:00
David
ddfd1dbff1
Added admin balance increase, and also test (#72) 
* added admin balance increase, and also test

* slightly hacky, but does do an update expression

* added note column to entities

* removed redundant printlns

* flattened items

* removed user tier logic from admin increase balance completely. admin will have to manually increase the user tier
 2023-05-22 17:07:17 -07:00
Bryan Stitt
05f862f3e4 cargo upgrade 2023-05-22 15:50:00 -07:00
Bryan Stitt
3ac15558e3 fix decay time 2023-05-22 15:43:39 -07:00
Bryan Stitt
91eeee23e2 use ether's reconnects instead of our own which need a lock (wip) 2023-05-22 15:32:27 -07:00
yenicelik
59634139c9 updated error_response 2023-05-21 20:27:52 -04:00
yenicelik
29678b025e updated error_response 2023-05-21 20:27:46 -04:00
yenicelik
46afe8b1e4 influxdb query also groups by archive_needed now 2023-05-21 17:13:15 -04:00
yenicelik
795548580a now should also group by archive_needed (turns out this was wanted) 2023-05-21 16:40:42 -04:00
Bryan Stitt
c443f76623 refactor tiers again 2023-05-19 14:09:03 -07:00
Bryan Stitt
5aae318620 logs and comments 2023-05-18 15:11:29 -07:00
Bryan Stitt
9c584354d9 add to skip list earlier 2023-05-18 13:51:28 -07:00
Bryan Stitt
a92c93706b dont check heads while waiting 
this isn't great. but should work better than what we have
 2023-05-18 13:47:00 -07:00
Bryan Stitt
8898ffe578 cargo upgrade 2023-05-18 13:36:35 -07:00
Bryan Stitt
b61675e928 replace all moka with quick_cache_ttl 2023-05-18 13:34:22 -07:00
Bryan Stitt
24439c5143 disable balance join for now 2023-05-18 00:43:33 -07:00
Bryan Stitt
0f367d9035 more quick cache ttl 2023-05-18 00:04:30 -07:00
Bryan Stitt
1c61390ad1 check if watch_consensus_rpcs.changed has failed 
i dont think it ever will, but better safe than sorry
 2023-05-16 23:42:42 -07:00
Bryan Stitt
c818a8fcd7 default balance to zero instead of panicing 2023-05-16 23:40:32 -07:00
Bryan Stitt
89961331af refactor wait_for_block into should_wait_for_block 2023-05-16 22:29:36 -07:00
Bryan Stitt
a07da30042 keep usable_rpcs_by_tier_and_head_number around longer 2023-05-16 18:32:33 -07:00
Bryan Stitt
0f79d2e60d more loud logs 2023-05-16 18:29:56 -07:00
Bryan Stitt
58b7799cc5 for now, always return 0 for cost 2023-05-16 18:25:56 -07:00
Bryan Stitt
a0d8218eb1 best block num has to check has_block_data 2023-05-16 18:18:56 -07:00
Bryan Stitt
30ea532c70 display, not debug 2023-05-16 16:13:56 -07:00
Bryan Stitt
5d64524aa6 add name to web3rpcs 2023-05-16 16:04:17 -07:00
Bryan Stitt
efaaf1bb5f send the correct content type 2023-05-16 15:27:18 -07:00
Bryan Stitt
525f6d4a34 better variable names 2023-05-16 14:58:00 -07:00
Bryan Stitt
40b15579ab Merge remote-tracking branch 'origin/devel' into quick_cache_ttl 2023-05-16 14:47:43 -07:00
Bryan Stitt
f4cebde53f use ttl quick cache 2023-05-16 14:46:05 -07:00
Bryan Stitt
978c385b3c improve wait for block 2023-05-16 14:02:37 -07:00
Bryan Stitt
c66eb6d864 head latency instead of peak latency for now 2023-05-16 12:18:59 -07:00
Bryan Stitt
685c1d039a cargo upgrade 2023-05-13 15:00:24 -07:00
Bryan Stitt
2080739865
Quick cache (#70) 
* quick cache and allocate less

* improve /status cache

* prepare to cache raw transaction hashes so we dont dos our backends

* simple benchmark for /health and /status

* mut not needed with atomics

* DRY all the status pages

* use u64 instead of bytes for subscriptions

* fix setting earliest_retry_at and improve logs

* Revert "use kanal instead of flume or tokio channels (#68)"

This reverts commit 510612d343fc51338a8a4282dcc229b50097835b.

* fix automatic retries

* put relaxed back

* convert error message time to seconds

* assert instead of debug_assert while we debug

* ns instead of seconds

* disable peak_latency for now

* null is the default

* cargo fmt

* comments

* remove request caching for now

* log on exit

* unit weigher for now

* make cache smaller. we need a weigher for prod. just debugging

* oops. we need async

* add todo

* no need for to_string on a RawValue
 2023-05-13 11:13:02 -07:00
Bryan Stitt
c3cdf7ef43 fix consensus finding bug 2023-05-12 23:40:15 -07:00
Bryan Stitt
4b7f88367d fix serializing /status 2023-05-12 23:22:20 -07:00
Bryan Stitt
74a602bc9c wip 2023-05-12 23:20:59 -07:00
Bryan Stitt
a3b97fda9b lint 2023-05-12 23:04:56 -07:00
Bryan Stitt
c54970da0a change ordering and move fetch_add and fetch_sub 2023-05-12 23:00:03 -07:00
Bryan Stitt
914c3e03a8 remove stale comments 2023-05-12 22:22:21 -07:00
Bryan Stitt
3cea4c3123 update peak_ewma to use request latency and fix off by one 2023-05-12 22:21:11 -07:00
Bryan Stitt
0c990b0755 use a cancel-safe channel for stats 2023-05-12 16:02:43 -07:00
Bryan Stitt
510612d343
use kanal instead of flume or tokio channels (#68) 2023-05-12 15:47:01 -07:00
Bryan Stitt
8a097dabbe
Bryan devel 2023-05-12 (#67) 
* add minor todo

* BadRequest instead of web3_context

* more bad request error codes

* use tokio-uring for the tcp listener

* clear block instead of panic

* clone earlier

* more watch channels instead of rwlocks

* drop uring for now (its single threaded) and combine get/post/put routes

* clean up iter vs into_iter and unnecessary collect

* arcswap instead of rwlock for Web3Rpcs.by_name

* cargo upgrade

* uuid fast-rng and alphabetize

* if protected rpcs, only use protected rpcs

* listenfd

* make connectinfo optional

* try_get_with_by_ref instead of try_get_with

* anyhow ensure. and try_get_with_as_ref isn't actually needed

* fix feature flags

* more refs and less clone

* automatic retry for eth_getTransactionReceipt and eth_getTransactionByHash

thanks for the report Lefteris @ Rotki

* ArcSwap for provider

* set archive_request to true on transaction retrying

* merge durable stats

* Revert "ArcSwap for provider"

This reverts commit 166d77f204cde9fa7722c0cefecbb27008749d47.

* comments

* less clones

* more refs

* fix test

* add optional mimalloc feature

* remove stale dependency

* sort

* cargo upgrade

* lint constants

* add todo

* another todo

* lint

* anyhow::ensure instead of panic

* allow rpc_accounting_v2 entries for requests without an rpc key
 2023-05-12 15:15:32 -07:00
David
34ed450fab
User Balance + Referral Logic (#44) 
* will implement balance topup endpoint

* will quickly fix other PR reviews

* merging from master

* will finish up godmoe

* will finish up login

* added logic to top up balance (first iteration)

* should implement additional columns soon (currency, amount, tx-hash), as well as a new table for spend

* updated migrations, will account for spend next

* get back to this later

* will merge PR from stats-v2

* stats v2

rebased all my commits and squashed them down to one

* cargo upgrade

* added migrtation for spend in accounting table. will run test-deposit next

* trying to get request from polygon

* first iteration /user/balance/:tx_hash works, needs to add accepted tokens next

* creating the referral code seems to work

* will now check if spending enough credits will lead to both parties receiving credits

* rpcstats takes care of accounting for spend data

* removed track spend from table

* Revert "removed track spend from table"

This reverts commit a50802d6ae75f786864c5ec42d0ceb2cb27124ed.

* Revert "rpcstats takes care of accounting for spend data"

This reverts commit 1cec728bf241e4cfd24351134637ed81c1a5a10b.

* removed rpc request table entity

* updated referral code to use ulid s

* credits used are aggregated

* added a bunch of fields to referrer

* added database logic whenever an aggregate stats is added. will have to iterate over this a couple times i think. go to (1) detecting accepted stables next, (2) fix influxdb bug and (3) start to write test

* removed track spend as this will occur in the database

* will first work on "balance", then referral. these should really be treated as two separate PRs (although already convoluted)

* balance logic initial commit

* breaking WIP, changing the RPC call logic functions

* will start testing next

* got rid of warnings & lint

* will proceed with subtracting / adding to balance

* added decimal points, balance tracking seems to work

* will beautify code a bit

* removed deprecated dependency, and added topic + deposit contract to app.yaml

* brownie test suite does not rely on local contract files it pulls all from polygonscan

* will continue with referral

* should perhaps (in a future revision) recordhow much the referees got for free. marking referrals seems to work rn

* user is upgraded to premium if they deposit more than 10$. we dont accept more than $10M in a single tx

* will start PR, referral seems to be fine so far, perhaps up to some numbers that still may need tweaking

* will start PR

* removed rogue comments, cleaned up payments a bit

* changes before PR

* apply stats

* added unique constraint

* some refactoring such that the user file is not too bloated

* compiling

* progress with subusers, creating a table entry seems to work

* good response type is there as well now, will work on getters from primary user and secondary user next

* subuser logic also seems fine now

* downgrade logic

* fixed bug influxdb does not support different types in same query (which makes sense)

* WIP temporary commit

* merging with PR

* Delete daemon.rs

there are multiple daemons now, so this was moved to `proxyd`

* will remove request clone to &mut

* multiple request handles for payment

* making requests still seem fine

* removed redundant commented out bits

* added deposit endpoint, added deposit amount and deposit user, untested yet

* small bug with downgrade tier id

* will add authorization so balance can be received for users

* balance history should be set now too

* will check balance over time again

* subususer can see rpc key balance if admin or owner

* stats also seems to work fine now with historical balance

* things seem to be building and working

* removed clone from  OpenRequestHandle

* removed influxdb from workspace members

* changed config files

* reran sea-orm generate entities, added a foreign key, should be proper now

* removed contract from commit

* made deposit contract optional

* added topic in polygon dev

* changed deposit contract to deposit factory contract

* added selfrelation on user_tier

* added payment required

* changed chain id to u64

* add wss in polygon llamarpc

* removed origin and method from the table

* added onchain transactions naming (and forgot to add a migration before)

* changed foreign key to be the referrer (id), not the code itself

* forgot to add id as the target foreign key

* WIP adding cache to update role

* fixed merge conflicts

---------

Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
Co-authored-by: Bryan Stitt <bryan@stitthappens.com>
 2023-05-12 09:45:15 -07:00
Bryan Stitt
36cc884112 Merge remote-tracking branch 'origin/main' into devel 2023-05-11 20:05:47 -07:00
Bryan Stitt
fc8ca4ba4f less max lag 2023-05-11 17:04:33 -07:00
Bryan Stitt
2a242fe37f cargo upgrade --incompatible 2023-05-11 13:25:55 -07:00
Bryan Stitt
6e8536d477 add support for http basic auth 2023-05-11 13:21:12 -07:00
Rory Trent
ec11e210ee
Peak ewma (#63) 
* use peak-ewma instead of head for latency calculation

* Implement some suggested changes from PR

* move latency to new package in workspace root

* fix unit tests which now require peak_latency on Web3Rpc

* Switch to atomics for peak-ewma

This change is to avoid locking from tokio::sync::watch.

* add decay calculation to latency reads in peak-ewma

* Add some tests for peak-ewma

* Sensible latency defaults and not blocking on full

* Cleanup and a couple additional comments
 2023-05-11 13:09:15 -07:00
Bryan Stitt
96a9a39789 add support for http basic auth 2023-05-11 13:07:38 -07:00
Bryan Stitt
0531e2f8dd add 'quota usage' rate limit substring 2023-05-08 11:25:27 -06:00
Bryan Stitt
4d64bf9580 cargo upgrade 2023-05-08 10:44:01 -06:00
Bryan Stitt
e7da6cd3d6 clone for now. this will change to use refs soon 2023-05-08 10:42:03 -06:00
Bryan Stitt
c2e3d71080 if protected rpcs, only use protected rpcs 2023-05-08 10:39:06 -06:00
Bryan Stitt
8916533a2a set archive_request to true on transaction retrying 2023-05-08 10:38:38 -06:00
Bryan Stitt
aff53ce36a automatic retry for eth_getTransactionReceipt and eth_getTransactionByHash 
thanks for the report Lefteris @ Rotki
 2023-05-08 10:38:23 -06:00
Bryan Stitt
ffb554d0b3 online editor isn't actually nice when it misses syntax errors :( 2023-04-29 18:50:26 -07:00
Bryan Stitt
91acbcaf75
Fix sort order 2023-04-29 18:38:31 -07:00
Bryan Stitt
93f243696a
simple match statement 2023-04-27 11:15:45 -07:00
Bryan Stitt
a236f7364d
DRY stats (#58) 
* DRY functions for saving stats and batch tsdb stats

* move comments
 2023-04-25 07:42:00 -07:00
Bryan Stitt
cc3c9462a3 cargo upgrade 2023-04-25 05:39:09 -07:00
Bryan Stitt
1fab1b1d46 oops. lower this log level 2023-04-24 11:21:53 -07:00
Bryan Stitt
56fdf48129
Send transactions to protected and public rpcs (#57) 
* move protected transactions into their own function and dry stats sending

* cargo upgrade

* comments

* time to live instead of time to idle

* minor workaround for eth_chainId

* cargo upgrade
 2023-04-24 11:00:12 -07:00
Bryan Stitt
7a087415a2 hash instead of block_hash 2023-04-20 19:55:18 -07:00
Bryan Stitt
4d2c0dae53 no sasl 2023-04-20 18:08:53 -07:00
Bryan Stitt
684b14c8a7 enable sasl for kafka 2023-04-20 14:48:15 -07:00
Bryan Stitt
77a317fdf8 default window to 60 seconds 2023-04-20 13:35:59 -07:00
Bryan Stitt
4ce372e57b lint 2023-04-20 10:38:10 -07:00
Bryan Stitt
03c4990dc3 remove commented out code 2023-04-20 10:36:56 -07:00
Bryan Stitt
9929e59f04 lower log level 2023-04-20 09:17:18 -07:00
Bryan Stitt
30eb6e91bf cargo upgrade 2023-04-19 21:09:17 -07:00
Bryan Stitt
03ef27baa4 Merge branch 'main' into devel 2023-04-19 15:31:08 -07:00
Bryan Stitt
b0653c0e9f add log for connecting to kafka 2023-04-19 15:30:01 -07:00
Bryan Stitt
10d2d46512 more packages for kafka+ssl 2023-04-19 15:22:27 -07:00
Bryan Stitt
f5775e5577 add security.protocol to search_kafka script 2023-04-19 14:28:28 -07:00
Bryan Stitt
decdac4b6d set protocol 2023-04-19 10:42:57 -07:00
Bryan Stitt
db3b157a47 really. shut up. i mean it 2023-04-18 21:19:55 -07:00
Bryan Stitt
9118117fe7 tell ethers to shut up 2023-04-18 20:50:35 -07:00
Bryan Stitt
4889c3e1ce Merge branch 'main' into devel 2023-04-18 20:38:43 -07:00
Bryan Stitt
e921d02eb2 remove stale todo 2023-04-17 17:10:09 -07:00
Bryan Stitt
5ff28943da fix incorrect context 2023-04-14 03:44:32 -07:00
Bryan Stitt
2652f88f52 attach more contexts 2023-04-14 03:43:49 -07:00
Bryan Stitt
fffd645acf remove copypasta comment 2023-04-14 02:10:49 -07:00
Bryan Stitt
c2710858e1 only warn if rpc_configs is too short 2023-04-14 00:41:51 -07:00
Bryan Stitt
df19619b77 add context to apply_server_configs 2023-04-14 00:38:49 -07:00
Bryan Stitt
8ed71e1cf1 more goerli fixes 2023-04-14 00:36:46 -07:00
Bryan Stitt
3621d71037 if not watching heads, send to any server 2023-04-14 00:15:27 -07:00
Bryan Stitt
f3435bc6e0 add bundler_4337_rpcs 2023-04-14 00:04:35 -07:00
Bryan Stitt
d035049c8f add /backups_needed endpoint for easy alerts 2023-04-13 17:15:01 -07:00
Bryan Stitt
3c5f973107 more stats 2023-04-11 12:04:16 -07:00
Bryan Stitt
b1120e635a change some error codes jsonrpc error handling 2023-04-10 22:29:02 -07:00
Bryan Stitt
b5ed0c4710 sometimes watch_for_block instead of returning an error 2023-04-10 22:28:31 -07:00
Bryan Stitt
ed5ca0575c logging improvements 2023-04-10 16:05:53 -07:00
Bryan Stitt
29c015508a fix warnings in cargo doc 2023-04-10 15:06:40 -07:00
Bryan Stitt
7250a0c155 upgrade ethers to 2.0.2 2023-04-10 15:01:36 -07:00
Bryan Stitt
55924ff840 cargo upgrade the easy things 2023-04-10 14:27:06 -07:00
Bryan Stitt
767920c055 remove log based debug lines 2023-04-10 14:19:13 -07:00
Bryan Stitt
b460f1701a rename revert_handler to error_handler 2023-04-05 15:34:28 -07:00
Bryan Stitt
072119901e remove stale comment 2023-04-05 15:26:39 -07:00
Bryan Stitt
7994c894bb cargo clippy and fmt 2023-04-05 15:15:20 -07:00
Bryan Stitt
1a160a8c7d Merge remote-tracking branch 'rorytrent/Web3ProxyError' into devel 2023-04-05 15:08:57 -07:00
Bryan Stitt
2b30422b84 fixes for NULL and UNIQUE to work together 2023-04-05 14:55:37 -07:00
Bryan Stitt
e90175c638 remove unused atomics 2023-04-05 14:37:59 -07:00
Bryan Stitt
09dacb45e2 remove stale comment 2023-04-05 14:37:48 -07:00
Bryan Stitt
2e934a41bf cargo upgrade 2023-04-05 12:30:52 -07:00
Bryan Stitt
05db94177d cargo fmt 2023-04-05 12:19:03 -07:00
yenicelik
0069e76040 latest changes from bryan 2023-04-04 14:40:22 +02:00
Rory Neithinger
f0d8a0c8c9 Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-04-01 00:04:44 -07:00
Rory Neithinger
ffdf25787f Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-03-31 23:23:30 -07:00
yenicelik
0bb3a2dc06 simple changes around bryans comments 2023-03-31 12:43:41 +01:00
yenicelik
345fa6c513 git status 2023-03-30 13:42:56 +01:00
yenicelik
05463c7ee5 migration seems to work for the most part 2023-03-30 12:54:01 +01:00
yenicelik
7390bb7910 non-lossy data transfer 2023-03-26 15:53:17 +02:00
yenicelik
cef26e66cc will create a PR for some feedback 2023-03-25 19:26:23 +01:00
yenicelik
53c7541fed seems to wait until everything is flushed. I will have to debug some stuff that isnt saved properly in the mysql (and then check influx) 2023-03-25 17:56:45 +01:00
Bryan Stitt
2205a472fd add optional tokio-console and improve reconnect logic 2023-03-22 18:43:13 -07:00
Bryan Stitt
7b223efa4d improve reconnect logic 2023-03-22 16:16:15 -07:00
Bryan Stitt
86b4f39a75 string hostname 2023-03-22 15:36:40 -07:00
Bryan Stitt
b4ca0fb62d better log when servers skipped 2023-03-22 15:19:09 -07:00
Bryan Stitt
c0afc295ca add hostname to status 2023-03-22 15:18:54 -07:00
Bryan Stitt
a69737db98 serialize best_rpcs 2023-03-22 14:23:14 -07:00
Bryan Stitt
8eff48611c more direct consensus finding code 
this hopefully has less bugs. speed isn't super important since this isn't on the host path.
 2023-03-22 13:10:15 -07:00
Bryan Stitt
9fae137e45 shorten log message 2023-03-22 02:06:26 -07:00
Rory Neithinger
e9c1d019fc fix some incorrect error codes and log levels 2023-03-21 22:52:46 -07:00
Rory Neithinger
b012f18b2b Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-03-21 22:40:39 -07:00
Bryan Stitt
5465d927b4 remove a debug log 2023-03-21 21:22:16 -07:00
Bryan Stitt
d992bb2dc9 skip, don't error 2023-03-21 16:00:34 -07:00
yenicelik
1d72a3cd44 will continue after bryan response 2023-03-21 12:49:36 +01:00
yenicelik
64505102ef creating a CLI endpoint for the migration 2023-03-21 12:07:21 +01:00
yenicelik
9beddc43a4 will get and inser some migration data 2023-03-21 10:11:35 +01:00
yenicelik
fbe97c12b2 will continue mtr 2023-03-20 22:00:49 +01:00
Rory Neithinger
fe2a760c8a more conversions to Web3ProxyError 2023-03-20 13:45:21 -07:00
Rory Neithinger
847e961eb0 remove more anyhows from app module 
the only ones left should be in the top level spawn functions.
 2023-03-20 13:07:24 -07:00
Rory Neithinger
60c1a6d382 remove anyhow from rest of frontend module 2023-03-20 12:47:57 -07:00
Rory Neithinger
3479bf9d06 add context type which preserves status code 2023-03-20 11:38:54 -07:00
yenicelik
0a6ccf28b5 will continue with migration 2023-03-20 12:22:30 +01:00
Rory Neithinger
1493d73386 better error handling for ws 2023-03-19 19:14:46 -07:00
Rory Neithinger
beac7ee017 better error handling for proxy_web3_rpc() 2023-03-19 18:52:28 -07:00
Rory Neithinger
c32d12b5e0 better error handling for ip_is_authorized() 2023-03-19 15:50:25 -07:00
yenicelik
c925ec9c27 about to merge stats into it 2023-03-18 10:15:21 +01:00
Rory Neithinger
f3fc4924dc Refactor FrontendErrorResponse into Web3ProxyError 
Renamed FrontendResponse to Web3ProxyResponse and introduced
a new generic type alias Web3ProxyResult.

Fixed a few noisy cargo warnings.
 2023-03-16 22:21:20 -07:00
yenicelik
ca41cb5c9e will push this for short review 2023-03-14 20:08:16 +01:00
yenicelik
65ca628cc7 will modify query so we can make queries more easily 2023-03-14 16:32:19 +01:00
yenicelik
9417961d78 for some reason values output is always zero 2023-03-12 16:09:20 +01:00
yenicelik
8c8c183e65 trying to debug returning multiple files 2023-03-10 20:26:15 +01:00
Bryan Stitt
b0c5d2b0eb remove debugging headers 2023-03-10 09:45:55 -08:00
Bryan Stitt
684c37ef6c cargo upgrade 2023-03-10 09:45:47 -08:00
Bryan Stitt
89853e24e5 all strings for json 2023-03-09 22:53:01 -08:00
Bryan Stitt
babd215e69 add an endpoint for debugging headers 2023-03-09 22:51:23 -08:00
Bryan Stitt
4203c61a59 use pretty address in logs 2023-03-09 11:25:57 -08:00
Bryan Stitt
7b9bcf6881 include head block in the log 2023-03-09 11:22:40 -08:00
Bryan Stitt
b480e92259 don't panic if param is not set 2023-03-09 09:32:30 -08:00
Bryan Stitt
d7a843cdc5 lint 2023-03-09 09:32:17 -08:00
Bryan Stitt
df8cc3b954 fix log order 2023-03-07 16:16:32 -08:00
Bryan Stitt
6717999ced actually use backup rpcs 2023-03-07 16:02:22 -08:00
Bryan Stitt
02a7c0be1e add todo 2023-03-07 16:02:22 -08:00
David
d25eed0bd2
hotfix for admin address changes (#39) 2023-03-07 15:44:22 -08:00
Bryan Stitt
7b1b997c72 include backtrace in page 2023-03-07 13:44:27 -08:00
yenicelik
f1f34fbcb0 will get back to this after fixing admin db issues 2023-03-07 22:40:34 +01:00
Bryan Stitt
b2016190e1 change_admin_status script fixes 2023-03-06 04:44:54 +00:00
yenicelik
ffd63444b2 tests seem to run successfully, should double check the huge merge 2023-03-04 22:31:39 +01:00
yenicelik
5a54fc5e99 lets modify web3 proxyd manually copy pasta 2023-03-04 21:19:39 +01:00
Bryan Stitt
c7dcc4aac3 actually return a json result from influx 2023-03-04 20:54:24 +01:00
Bryan Stitt
163bbbafca FromDataPoint needs Default 2023-03-04 20:54:24 +01:00
Bryan Stitt
5b48d471bb fix influxdb derive macro 
https://github.com/aprimadi/influxdb2/issues/17
 2023-03-04 20:54:24 +01:00
Bryan Stitt
dbd7860416 wip influxdb2 2023-03-04 20:54:04 +01:00
Bryan Stitt
3f217b930a cargo upgrade 2023-03-04 20:49:21 +01:00
Bryan Stitt
eb4d05a520 stats v2 
rebased all my commits and squashed them down to one
 2023-03-04 20:48:49 +01:00
Bryan Stitt
36e4183373 todos 2023-03-03 11:19:25 -08:00
Bryan Stitt
94089eb72b combine comparison 2023-03-03 07:34:04 -08:00
Bryan Stitt
4c18657387 more kafka things 2023-03-03 06:58:45 -08:00
Bryan Stitt
2eb8df22ec kafka helper script 2023-03-03 02:54:52 -08:00
Bryan Stitt
1869e73131 actually add the script 2023-03-02 22:50:14 -08:00
Bryan Stitt
cbdd6d7c1e oops. didn't mean to commit this script yet 2023-03-02 21:12:35 -08:00
Bryan Stitt
586b772cdf apt needs '--yes' 2023-03-02 20:58:07 -08:00
Bryan Stitt
3b61e009f8 group cfgs 2023-03-02 17:51:41 -08:00
Bryan Stitt
3098791ad9 add optional kafka feature 2023-03-03 01:39:50 +00:00
Bryan Stitt
a54e33f598 head lag instead of head block 2023-03-02 19:10:30 +00:00
Bryan Stitt
e6ca8a5253 add head_block, head_latency, and request_latency to popularity_contest table 2023-03-02 19:08:47 +00:00
Bryan Stitt
c7b74e1795 sort popularity_contest rows descending by request counts 2023-03-02 18:33:17 +00:00
Bryan Stitt
d1b955275d clean up wait_for_ functions 2023-03-02 18:27:32 +00:00
Bryan Stitt
907a147afa wait_for_provider helper function 2023-03-02 18:14:17 +00:00
Bryan Stitt
0d07e20d7c make it compile 2023-03-02 17:38:42 +00:00
Bryan Stitt
258e889f69 wait for a provider 2023-03-02 16:52:28 +00:00
Bryan Stitt
499d50b5fc handle empty private_rpcs 2023-03-01 23:21:09 +00:00
Bryan Stitt
c8bc0d288f high log level 2023-03-01 23:04:20 +00:00
Bryan Stitt
fdba8e7864 test all_connections with Nones 2023-03-01 22:50:44 +00:00
Bryan Stitt
f68886fafd add simple test for all connections and include more rpcs 2023-03-01 22:32:01 +00:00
Bryan Stitt
baf07f7de5 sync sort should include backup 2023-03-01 21:35:50 +00:00
Bryan Stitt
2e9803de72 remove arbitrary sleep timer 2023-03-01 20:56:00 +00:00
Bryan Stitt
b87c988439 lint 2023-03-01 11:23:59 -08:00
Bryan Stitt
1da12f5844 bump version to match number of migrations 2023-02-28 23:35:20 -08:00
Bryan Stitt
1ffccbe229
Merge pull request #36 from yenicelik/19-admin-imitate 
Admin endpoint (Godmode / Imitate User)
 2023-02-28 23:32:05 -08:00
Bryan Stitt
20851af51b log seconds instead of milliseconds 2023-02-28 11:09:49 -08:00
Bryan Stitt
d83a1f0169 use Web3Rpc instead of the name as a key 2023-02-28 11:01:34 -08:00
Bryan Stitt
26970c2d88 move wait on new connection 2023-02-27 13:40:13 -08:00
Bryan Stitt
c5e2402d6e disconnect is noisy but it works 2023-02-27 13:29:07 -08:00
Bryan Stitt
11ee0aafe9 inotify is a pain cross platform. just check file hashes 2023-02-27 13:13:18 -08:00
Bryan Stitt
6067369ee3 add inotify and rpc disconnect 2023-02-27 12:50:57 -08:00
David
27a85c946f
Merge branch 'main' into 19-admin-imitate 2023-02-27 10:56:29 +01:00
yenicelik
fb5c9c8770 typo login should have been logout 2023-02-27 10:56:06 +01:00
Bryan Stitt
475e521918 cache getLogs with blockhash 2023-02-26 23:08:00 -08:00
Bryan Stitt
f8f5e7a1c8 add thread for config file watching and run clippy lint 2023-02-26 23:08:00 -08:00
Bryan Stitt
bf79d677b0 by_name needs to a lock 2023-02-26 20:00:13 -08:00
Bryan Stitt
e96f09a9c4 refactors to make configs partially reloadable 2023-02-25 23:52:33 -08:00
Bryan Stitt
c9e5661c5b wip 2023-02-25 09:55:28 -08:00
Bryan Stitt
c47c9136a1 cargo upgrade 2023-02-25 00:47:16 -08:00
Bryan Stitt
580965ef83 get instead of remove 2023-02-24 23:31:10 -08:00
Bryan Stitt
91b853d21e archive instead of Some(u64::MAX) in the logs 2023-02-21 21:14:49 -08:00
Bryan Stitt
0ab7738393 per-chain rpc rate limits 2023-02-21 21:10:23 -08:00
Bryan Stitt
6c33b1ae1e block number instead of hash during health check 2023-02-21 20:25:54 -08:00
yenicelik
933f71d2d9 should be ok now again 2023-02-19 21:54:23 +01:00
yenicelik
a7161790f1 pull from origin, about to push 2023-02-19 21:45:58 +01:00
yenicelik
f783ff66cf fix some errors that occurred during refactor 2023-02-19 21:43:38 +01:00
yenicelik
fc1fdaaaf1 continue rebase 2023-02-19 21:36:54 +01:00
Bryan Stitt
df87a41d5b theres a perf regression in here somewhere 2023-02-19 21:34:42 +01:00
Bryan Stitt
678cd8067b lower log level 2023-02-19 21:34:42 +01:00
Bryan Stitt
30d4da6594 the macro is easier 2023-02-19 21:34:42 +01:00
Bryan Stitt
0c4ec8ea03 properly encode number responses as hex 2023-02-19 21:34:42 +01:00
Bryan Stitt
85608dbf25 fix params in health check query 2023-02-19 21:34:42 +01:00
Bryan Stitt
cfff3ac532 comment 2023-02-19 21:34:42 +01:00
Bryan Stitt
e1f803e91a make it work 2023-02-19 21:34:42 +01:00
Bryan Stitt
1b7050d294 yes health, no channel 2023-02-19 21:34:42 +01:00
Bryan Stitt
688cc62460 remove health for now? 2023-02-19 21:34:42 +01:00
Bryan Stitt
d50c869df0 dont hold a lock open 2023-02-19 21:34:42 +01:00
Bryan Stitt
6eff5e3bf1 health check and latency recordings 2023-02-19 21:34:42 +01:00
Bryan Stitt
83c1d7051d undo a rename in the json 2023-02-19 21:34:42 +01:00
Bryan Stitt
88da33c7e4 i think it works 2023-02-19 21:34:42 +01:00