Commit Graph

925 Commits

Author SHA1 Message Date
Bryan Stitt 4d64bf9580 cargo upgrade 2023-05-08 10:44:01 -06:00
Bryan Stitt e7da6cd3d6 clone for now. this will change to use refs soon 2023-05-08 10:42:03 -06:00
Bryan Stitt c2e3d71080 if protected rpcs, only use protected rpcs 2023-05-08 10:39:06 -06:00
Bryan Stitt 8916533a2a set archive_request to true on transaction retrying 2023-05-08 10:38:38 -06:00
Bryan Stitt aff53ce36a automatic retry for eth_getTransactionReceipt and eth_getTransactionByHash
thanks for the report Lefteris @ Rotki
2023-05-08 10:38:23 -06:00
Bryan Stitt ffb554d0b3 online editor isn't actually nice when it misses syntax errors :( 2023-04-29 18:50:26 -07:00
Bryan Stitt 91acbcaf75
Fix sort order 2023-04-29 18:38:31 -07:00
Bryan Stitt 93f243696a
simple match statement 2023-04-27 11:15:45 -07:00
Bryan Stitt a236f7364d
DRY stats (#58)
* DRY functions for saving stats and batch tsdb stats

* move comments
2023-04-25 07:42:00 -07:00
Bryan Stitt cc3c9462a3 cargo upgrade 2023-04-25 05:39:09 -07:00
Bryan Stitt 1fab1b1d46 oops. lower this log level 2023-04-24 11:21:53 -07:00
Bryan Stitt 56fdf48129
Send transactions to protected and public rpcs (#57)
* move protected transactions into their own function and dry stats sending

* cargo upgrade

* comments

* time to live instead of time to idle

* minor workaround for eth_chainId

* cargo upgrade
2023-04-24 11:00:12 -07:00
Bryan Stitt 7a087415a2 hash instead of block_hash 2023-04-20 19:55:18 -07:00
Bryan Stitt 4d2c0dae53 no sasl 2023-04-20 18:08:53 -07:00
Bryan Stitt 684b14c8a7 enable sasl for kafka 2023-04-20 14:48:15 -07:00
Bryan Stitt 77a317fdf8 default window to 60 seconds 2023-04-20 13:35:59 -07:00
Bryan Stitt 4ce372e57b lint 2023-04-20 10:38:10 -07:00
Bryan Stitt 03c4990dc3 remove commented out code 2023-04-20 10:36:56 -07:00
Bryan Stitt 9929e59f04 lower log level 2023-04-20 09:17:18 -07:00
Bryan Stitt 30eb6e91bf cargo upgrade 2023-04-19 21:09:17 -07:00
Bryan Stitt 03ef27baa4 Merge branch 'main' into devel 2023-04-19 15:31:08 -07:00
Bryan Stitt b0653c0e9f add log for connecting to kafka 2023-04-19 15:30:01 -07:00
Bryan Stitt 10d2d46512 more packages for kafka+ssl 2023-04-19 15:22:27 -07:00
Bryan Stitt f5775e5577 add security.protocol to search_kafka script 2023-04-19 14:28:28 -07:00
Bryan Stitt decdac4b6d set protocol 2023-04-19 10:42:57 -07:00
Bryan Stitt db3b157a47 really. shut up. i mean it 2023-04-18 21:19:55 -07:00
Bryan Stitt 9118117fe7 tell ethers to shut up 2023-04-18 20:50:35 -07:00
Bryan Stitt 4889c3e1ce Merge branch 'main' into devel 2023-04-18 20:38:43 -07:00
Bryan Stitt e921d02eb2 remove stale todo 2023-04-17 17:10:09 -07:00
Bryan Stitt 5ff28943da fix incorrect context 2023-04-14 03:44:32 -07:00
Bryan Stitt 2652f88f52 attach more contexts 2023-04-14 03:43:49 -07:00
Bryan Stitt fffd645acf remove copypasta comment 2023-04-14 02:10:49 -07:00
Bryan Stitt c2710858e1 only warn if rpc_configs is too short 2023-04-14 00:41:51 -07:00
Bryan Stitt df19619b77 add context to apply_server_configs 2023-04-14 00:38:49 -07:00
Bryan Stitt 8ed71e1cf1 more goerli fixes 2023-04-14 00:36:46 -07:00
Bryan Stitt 3621d71037 if not watching heads, send to any server 2023-04-14 00:15:27 -07:00
Bryan Stitt f3435bc6e0 add bundler_4337_rpcs 2023-04-14 00:04:35 -07:00
Bryan Stitt d035049c8f add /backups_needed endpoint for easy alerts 2023-04-13 17:15:01 -07:00
Bryan Stitt 3c5f973107 more stats 2023-04-11 12:04:16 -07:00
Bryan Stitt b1120e635a change some error codes jsonrpc error handling 2023-04-10 22:29:02 -07:00
Bryan Stitt b5ed0c4710 sometimes watch_for_block instead of returning an error 2023-04-10 22:28:31 -07:00
Bryan Stitt ed5ca0575c logging improvements 2023-04-10 16:05:53 -07:00
Bryan Stitt 29c015508a fix warnings in cargo doc 2023-04-10 15:06:40 -07:00
Bryan Stitt 7250a0c155 upgrade ethers to 2.0.2 2023-04-10 15:01:36 -07:00
Bryan Stitt 55924ff840 cargo upgrade the easy things 2023-04-10 14:27:06 -07:00
Bryan Stitt 767920c055 remove log based debug lines 2023-04-10 14:19:13 -07:00
Bryan Stitt b460f1701a rename revert_handler to error_handler 2023-04-05 15:34:28 -07:00
Bryan Stitt 072119901e remove stale comment 2023-04-05 15:26:39 -07:00
Bryan Stitt 7994c894bb cargo clippy and fmt 2023-04-05 15:15:20 -07:00
Bryan Stitt 1a160a8c7d Merge remote-tracking branch 'rorytrent/Web3ProxyError' into devel 2023-04-05 15:08:57 -07:00
Bryan Stitt 2b30422b84 fixes for NULL and UNIQUE to work together 2023-04-05 14:55:37 -07:00
Bryan Stitt e90175c638 remove unused atomics 2023-04-05 14:37:59 -07:00
Bryan Stitt 09dacb45e2 remove stale comment 2023-04-05 14:37:48 -07:00
Bryan Stitt 2e934a41bf cargo upgrade 2023-04-05 12:30:52 -07:00
Bryan Stitt 05db94177d cargo fmt 2023-04-05 12:19:03 -07:00
yenicelik 0069e76040 latest changes from bryan 2023-04-04 14:40:22 +02:00
Rory Neithinger f0d8a0c8c9 Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-04-01 00:04:44 -07:00
Rory Neithinger ffdf25787f Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-03-31 23:23:30 -07:00
yenicelik 0bb3a2dc06 simple changes around bryans comments 2023-03-31 12:43:41 +01:00
yenicelik 345fa6c513 git status 2023-03-30 13:42:56 +01:00
yenicelik 05463c7ee5 migration seems to work for the most part 2023-03-30 12:54:01 +01:00
yenicelik 7390bb7910 non-lossy data transfer 2023-03-26 15:53:17 +02:00
yenicelik cef26e66cc will create a PR for some feedback 2023-03-25 19:26:23 +01:00
yenicelik 53c7541fed seems to wait until everything is flushed. I will have to debug some stuff that isnt saved properly in the mysql (and then check influx) 2023-03-25 17:56:45 +01:00
Bryan Stitt 2205a472fd add optional tokio-console and improve reconnect logic 2023-03-22 18:43:13 -07:00
Bryan Stitt 7b223efa4d improve reconnect logic 2023-03-22 16:16:15 -07:00
Bryan Stitt 86b4f39a75 string hostname 2023-03-22 15:36:40 -07:00
Bryan Stitt b4ca0fb62d better log when servers skipped 2023-03-22 15:19:09 -07:00
Bryan Stitt c0afc295ca add hostname to status 2023-03-22 15:18:54 -07:00
Bryan Stitt a69737db98 serialize best_rpcs 2023-03-22 14:23:14 -07:00
Bryan Stitt 8eff48611c more direct consensus finding code
this hopefully has less bugs. speed isn't super important since this isn't on the host path.
2023-03-22 13:10:15 -07:00
Bryan Stitt 9fae137e45 shorten log message 2023-03-22 02:06:26 -07:00
Rory Neithinger e9c1d019fc fix some incorrect error codes and log levels 2023-03-21 22:52:46 -07:00
Rory Neithinger b012f18b2b Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-03-21 22:40:39 -07:00
Bryan Stitt 5465d927b4 remove a debug log 2023-03-21 21:22:16 -07:00
Bryan Stitt d992bb2dc9 skip, don't error 2023-03-21 16:00:34 -07:00
yenicelik 1d72a3cd44 will continue after bryan response 2023-03-21 12:49:36 +01:00
yenicelik 64505102ef creating a CLI endpoint for the migration 2023-03-21 12:07:21 +01:00
yenicelik 9beddc43a4 will get and inser some migration data 2023-03-21 10:11:35 +01:00
yenicelik fbe97c12b2 will continue mtr 2023-03-20 22:00:49 +01:00
Rory Neithinger fe2a760c8a more conversions to Web3ProxyError 2023-03-20 13:45:21 -07:00
Rory Neithinger 847e961eb0 remove more anyhows from app module
the only ones left should be in the top level spawn functions.
2023-03-20 13:07:24 -07:00
Rory Neithinger 60c1a6d382 remove anyhow from rest of frontend module 2023-03-20 12:47:57 -07:00
Rory Neithinger 3479bf9d06 add context type which preserves status code 2023-03-20 11:38:54 -07:00
yenicelik 0a6ccf28b5 will continue with migration 2023-03-20 12:22:30 +01:00
Rory Neithinger 1493d73386 better error handling for ws 2023-03-19 19:14:46 -07:00
Rory Neithinger beac7ee017 better error handling for proxy_web3_rpc() 2023-03-19 18:52:28 -07:00
Rory Neithinger c32d12b5e0 better error handling for ip_is_authorized() 2023-03-19 15:50:25 -07:00
yenicelik c925ec9c27 about to merge stats into it 2023-03-18 10:15:21 +01:00
Rory Neithinger f3fc4924dc Refactor FrontendErrorResponse into Web3ProxyError
Renamed FrontendResponse to Web3ProxyResponse and introduced
a new generic type alias Web3ProxyResult.

Fixed a few noisy cargo warnings.
2023-03-16 22:21:20 -07:00
yenicelik ca41cb5c9e will push this for short review 2023-03-14 20:08:16 +01:00
yenicelik 65ca628cc7 will modify query so we can make queries more easily 2023-03-14 16:32:19 +01:00
yenicelik 9417961d78 for some reason values output is always zero 2023-03-12 16:09:20 +01:00
yenicelik 8c8c183e65 trying to debug returning multiple files 2023-03-10 20:26:15 +01:00
Bryan Stitt b0c5d2b0eb remove debugging headers 2023-03-10 09:45:55 -08:00
Bryan Stitt 684c37ef6c cargo upgrade 2023-03-10 09:45:47 -08:00
Bryan Stitt 89853e24e5 all strings for json 2023-03-09 22:53:01 -08:00
Bryan Stitt babd215e69 add an endpoint for debugging headers 2023-03-09 22:51:23 -08:00
Bryan Stitt 4203c61a59 use pretty address in logs 2023-03-09 11:25:57 -08:00
Bryan Stitt 7b9bcf6881 include head block in the log 2023-03-09 11:22:40 -08:00
Bryan Stitt b480e92259 don't panic if param is not set 2023-03-09 09:32:30 -08:00
Bryan Stitt d7a843cdc5 lint 2023-03-09 09:32:17 -08:00
Bryan Stitt df8cc3b954 fix log order 2023-03-07 16:16:32 -08:00
Bryan Stitt 6717999ced actually use backup rpcs 2023-03-07 16:02:22 -08:00
Bryan Stitt 02a7c0be1e add todo 2023-03-07 16:02:22 -08:00
David d25eed0bd2
hotfix for admin address changes (#39) 2023-03-07 15:44:22 -08:00
Bryan Stitt 7b1b997c72 include backtrace in page 2023-03-07 13:44:27 -08:00
yenicelik f1f34fbcb0 will get back to this after fixing admin db issues 2023-03-07 22:40:34 +01:00
Bryan Stitt b2016190e1 change_admin_status script fixes 2023-03-06 04:44:54 +00:00
yenicelik ffd63444b2 tests seem to run successfully, should double check the huge merge 2023-03-04 22:31:39 +01:00
yenicelik 5a54fc5e99 lets modify web3 proxyd manually copy pasta 2023-03-04 21:19:39 +01:00
Bryan Stitt c7dcc4aac3 actually return a json result from influx 2023-03-04 20:54:24 +01:00
Bryan Stitt 163bbbafca FromDataPoint needs Default 2023-03-04 20:54:24 +01:00
Bryan Stitt 5b48d471bb fix influxdb derive macro
https://github.com/aprimadi/influxdb2/issues/17
2023-03-04 20:54:24 +01:00
Bryan Stitt dbd7860416 wip influxdb2 2023-03-04 20:54:04 +01:00
Bryan Stitt 3f217b930a cargo upgrade 2023-03-04 20:49:21 +01:00
Bryan Stitt eb4d05a520 stats v2
rebased all my commits and squashed them down to one
2023-03-04 20:48:49 +01:00
Bryan Stitt 36e4183373 todos 2023-03-03 11:19:25 -08:00
Bryan Stitt 94089eb72b combine comparison 2023-03-03 07:34:04 -08:00
Bryan Stitt 4c18657387 more kafka things 2023-03-03 06:58:45 -08:00
Bryan Stitt 2eb8df22ec kafka helper script 2023-03-03 02:54:52 -08:00
Bryan Stitt 1869e73131 actually add the script 2023-03-02 22:50:14 -08:00
Bryan Stitt cbdd6d7c1e oops. didn't mean to commit this script yet 2023-03-02 21:12:35 -08:00
Bryan Stitt 586b772cdf apt needs '--yes' 2023-03-02 20:58:07 -08:00
Bryan Stitt 3b61e009f8 group cfgs 2023-03-02 17:51:41 -08:00
Bryan Stitt 3098791ad9 add optional kafka feature 2023-03-03 01:39:50 +00:00
Bryan Stitt a54e33f598 head lag instead of head block 2023-03-02 19:10:30 +00:00
Bryan Stitt e6ca8a5253 add head_block, head_latency, and request_latency to popularity_contest table 2023-03-02 19:08:47 +00:00
Bryan Stitt c7b74e1795 sort popularity_contest rows descending by request counts 2023-03-02 18:33:17 +00:00
Bryan Stitt d1b955275d clean up wait_for_ functions 2023-03-02 18:27:32 +00:00
Bryan Stitt 907a147afa wait_for_provider helper function 2023-03-02 18:14:17 +00:00
Bryan Stitt 0d07e20d7c make it compile 2023-03-02 17:38:42 +00:00
Bryan Stitt 258e889f69 wait for a provider 2023-03-02 16:52:28 +00:00
Bryan Stitt 499d50b5fc handle empty private_rpcs 2023-03-01 23:21:09 +00:00
Bryan Stitt c8bc0d288f high log level 2023-03-01 23:04:20 +00:00
Bryan Stitt fdba8e7864 test all_connections with Nones 2023-03-01 22:50:44 +00:00
Bryan Stitt f68886fafd add simple test for all connections and include more rpcs 2023-03-01 22:32:01 +00:00
Bryan Stitt baf07f7de5 sync sort should include backup 2023-03-01 21:35:50 +00:00
Bryan Stitt 2e9803de72 remove arbitrary sleep timer 2023-03-01 20:56:00 +00:00
Bryan Stitt b87c988439 lint 2023-03-01 11:23:59 -08:00
Bryan Stitt 1da12f5844 bump version to match number of migrations 2023-02-28 23:35:20 -08:00
Bryan Stitt 1ffccbe229
Merge pull request #36 from yenicelik/19-admin-imitate
Admin endpoint (Godmode / Imitate User)
2023-02-28 23:32:05 -08:00
Bryan Stitt 20851af51b log seconds instead of milliseconds 2023-02-28 11:09:49 -08:00
Bryan Stitt d83a1f0169 use Web3Rpc instead of the name as a key 2023-02-28 11:01:34 -08:00
Bryan Stitt 26970c2d88 move wait on new connection 2023-02-27 13:40:13 -08:00
Bryan Stitt c5e2402d6e disconnect is noisy but it works 2023-02-27 13:29:07 -08:00
Bryan Stitt 11ee0aafe9 inotify is a pain cross platform. just check file hashes 2023-02-27 13:13:18 -08:00
Bryan Stitt 6067369ee3 add inotify and rpc disconnect 2023-02-27 12:50:57 -08:00
David 27a85c946f
Merge branch 'main' into 19-admin-imitate 2023-02-27 10:56:29 +01:00
yenicelik fb5c9c8770 typo login should have been logout 2023-02-27 10:56:06 +01:00
Bryan Stitt 475e521918 cache getLogs with blockhash 2023-02-26 23:08:00 -08:00
Bryan Stitt f8f5e7a1c8 add thread for config file watching and run clippy lint 2023-02-26 23:08:00 -08:00
Bryan Stitt bf79d677b0 by_name needs to a lock 2023-02-26 20:00:13 -08:00
Bryan Stitt e96f09a9c4 refactors to make configs partially reloadable 2023-02-25 23:52:33 -08:00
Bryan Stitt c9e5661c5b wip 2023-02-25 09:55:28 -08:00
Bryan Stitt c47c9136a1 cargo upgrade 2023-02-25 00:47:16 -08:00
Bryan Stitt 580965ef83 get instead of remove 2023-02-24 23:31:10 -08:00
Bryan Stitt 91b853d21e archive instead of Some(u64::MAX) in the logs 2023-02-21 21:14:49 -08:00
Bryan Stitt 0ab7738393 per-chain rpc rate limits 2023-02-21 21:10:23 -08:00
Bryan Stitt 6c33b1ae1e block number instead of hash during health check 2023-02-21 20:25:54 -08:00
yenicelik 933f71d2d9 should be ok now again 2023-02-19 21:54:23 +01:00
yenicelik a7161790f1 pull from origin, about to push 2023-02-19 21:45:58 +01:00
yenicelik f783ff66cf fix some errors that occurred during refactor 2023-02-19 21:43:38 +01:00
yenicelik fc1fdaaaf1 continue rebase 2023-02-19 21:36:54 +01:00
Bryan Stitt df87a41d5b theres a perf regression in here somewhere 2023-02-19 21:34:42 +01:00
Bryan Stitt 678cd8067b lower log level 2023-02-19 21:34:42 +01:00
Bryan Stitt 30d4da6594 the macro is easier 2023-02-19 21:34:42 +01:00
Bryan Stitt 0c4ec8ea03 properly encode number responses as hex 2023-02-19 21:34:42 +01:00
Bryan Stitt 85608dbf25 fix params in health check query 2023-02-19 21:34:42 +01:00
Bryan Stitt cfff3ac532 comment 2023-02-19 21:34:42 +01:00
Bryan Stitt e1f803e91a make it work 2023-02-19 21:34:42 +01:00
Bryan Stitt 1b7050d294 yes health, no channel 2023-02-19 21:34:42 +01:00
Bryan Stitt 688cc62460 remove health for now? 2023-02-19 21:34:42 +01:00
Bryan Stitt d50c869df0 dont hold a lock open 2023-02-19 21:34:42 +01:00
Bryan Stitt 6eff5e3bf1 health check and latency recordings 2023-02-19 21:34:42 +01:00
Bryan Stitt 83c1d7051d undo a rename in the json 2023-02-19 21:34:42 +01:00
Bryan Stitt 88da33c7e4 i think it works 2023-02-19 21:34:42 +01:00
Bryan Stitt 658f658a0b add time to idle to more caches 2023-02-19 21:34:42 +01:00
Bryan Stitt 45fe524efb cargo upgrade 2023-02-19 21:34:42 +01:00
Bryan Stitt 0e8ebfa823 getting close 2023-02-19 21:34:42 +01:00
Bryan Stitt a519427fcf fixed one test. still more to fix though 2023-02-19 21:34:42 +01:00
Bryan Stitt da33ec32eb well it compiles. doesnt work yet. but it compiles 2023-02-19 21:34:42 +01:00
Bryan Stitt 53757621ef variable rename 2023-02-19 21:34:42 +01:00
Bryan Stitt 54b4aa8522 cargo upgrade 2023-02-19 21:34:42 +01:00
Bryan Stitt 4daf984b4b use http and ws providers in a single config 2023-02-19 21:34:42 +01:00
Bryan Stitt 5a3f586d99 include to_block more places 2023-02-19 21:34:42 +01:00
Bryan Stitt 8af87845c9 better sorting of connections 2023-02-19 21:34:42 +01:00
yenicelik 11b787e8b1 removed one comment (not needed to remove redis) 2023-02-19 21:34:42 +01:00
yenicelik 38ac23ca06 added admin trail when someone tries to login for godmode 2023-02-19 21:34:42 +01:00
yenicelik 85bec3aaf0 currently also have to specify the admin user calling the request. also added a tiny manual test to imitate user. will add trails next 2023-02-19 21:34:42 +01:00
yenicelik 7bf1d6e95a redis is not saving raw bearer tokens, so i will remove this from here 2023-02-19 21:34:42 +01:00
yenicelik 2e1fd5cd3c still struggingling with login 2023-02-19 21:34:42 +01:00
yenicelik 2ac550f60a will wait for how to login as a third party user 2023-02-19 21:34:42 +01:00
yenicelik b1f18460d1 changes from review. will test now 2023-02-19 21:34:42 +01:00
yenicelik de8d665e40 should next write some simple end2end test 2023-02-19 21:34:42 +01:00
yenicelik f701109aed should write some tests next 2023-02-19 21:34:42 +01:00
yenicelik 1250b7dba2 continue rebase 2023-02-19 21:34:39 +01:00
yenicelik 61bd7d6e35 also removing login from cache. should add tests for all these cases 2023-02-19 21:33:39 +01:00
yenicelik bbe9061402 continue rebase 2023-02-19 21:33:33 +01:00
yenicelik 6420dc6aa6 tiny refactoring and added admin entities. some bugs coming up 2023-02-19 21:27:59 +01:00