Commit Graph

302 Commits

Author SHA1 Message Date
Bryan Stitt 8278fe006b cleanup 2022-10-19 21:34:05 +00:00
Bryan Stitt e7e949a53e user_login_get does not need params 2022-10-19 18:51:14 +00:00
Bryan Stitt 8a23efd048 optional params like this do not work. need hashmap 2022-10-19 18:50:16 +00:00
Bryan Stitt e9abcf883f query params and grouping for user stats 2022-10-19 18:38:00 +00:00
Bryan Stitt 3793bdff67 aggregate stats endpoint 2022-10-19 00:57:22 +00:00
Bryan Stitt f6c2d29d0e stricter configs 2022-10-18 23:27:33 +00:00
Bryan Stitt 69a090522b more todo 2022-10-17 23:16:09 +00:00
Bryan Stitt 848af3d8b3 more docs 2022-10-17 21:47:58 +00:00
Bryan Stitt 76c8f1ef96 i think it works 2022-10-11 21:31:34 +00:00
Bryan Stitt 552f3dbffc proper sizes for caches and emit all stats 2022-10-11 19:58:25 +00:00
Bryan Stitt 8f3d31869f less locks and fix some stats 2022-10-11 17:34:25 +00:00
Bryan Stitt cfd26940a9 this might be too many columns 2022-10-11 05:13:00 +00:00
Bryan Stitt da2bd0b0a5 wait on invalidating 2022-10-10 05:44:04 +00:00
Bryan Stitt 825370b5d9 stats almost work
just need to update the migration to match our new ideas for columns.

should also make the shutdown smarter so that nothing gets lost
2022-10-10 05:35:25 +00:00
Bryan Stitt 25aa68a5bf add per-user rpc accounting 2022-10-10 04:15:49 +00:00
Bryan Stitt ecd3dc29fc shorter strings 2022-10-07 02:21:34 +00:00
Bryan Stitt 44adffcd32 no pipe needed 2022-10-07 02:21:24 +00:00
Bryan Stitt c8fbf46885 stats in redis that actually work
we should still investigate a real time series db, but stats in redis is much simpler for now
2022-10-07 02:15:53 +00:00
Bryan Stitt 875ae457ef emit our first stats-- cache hits and misses 2022-10-03 20:02:05 +00:00
Bryan Stitt 25d34da98d add stat emitter 2022-10-03 18:08:01 +00:00
Bryan Stitt a7b5f25467 unwrap less 2022-09-30 04:18:18 +00:00
Bryan Stitt 9cc34f7aa8 move warning 2022-09-28 17:01:11 +00:00
Bryan Stitt 2b03372e6b error instead of todo 2022-09-28 16:35:18 +00:00
Bryan Stitt 53f9551180 finish adding semaphores 2022-09-28 03:35:55 +00:00
Bryan Stitt 0c7a2b05ec start adding semaphores 2022-09-27 02:01:45 +00:00
Bryan Stitt a199f13426 not everything needs to be under /rpc 2022-09-25 16:37:45 +00:00
Bryan Stitt 28dcfca47b add sensitive-headers 2022-09-25 16:35:01 +00:00
Bryan Stitt 7ed7f96a2f very permissive cors policy 2022-09-25 04:26:13 +00:00
Bryan Stitt 2c69b0ce9a fix method and param for saving reverts 2022-09-24 07:36:25 +00:00
Bryan Stitt 3074d6f76f left pad addresses 2022-09-24 07:18:33 +00:00
Bryan Stitt a80503ac48 better logging on save reverts checks 2022-09-24 07:04:11 +00:00
Bryan Stitt 7da8864a1d improve --address flag on user_create 2022-09-24 07:03:42 +00:00
Bryan Stitt bbaa32e7b5 release 0.2.0 2022-09-24 05:54:38 +00:00
Bryan Stitt 5df2469d53 ULID or UUID. Prefer ULID 2022-09-24 05:53:45 +00:00
Bryan Stitt b254cb7d26 check for bearer token on /rpc 2022-09-24 04:31:06 +00:00
Bryan Stitt 81deb1103c login needs its own rate limiter 2022-09-24 03:59:21 +00:00
Bryan Stitt c12e25a210 no need for this clone 2022-09-24 02:48:23 +00:00
Bryan Stitt 8459dcd1f1 let the frontend handle their own cookies 2022-09-24 02:47:44 +00:00
Bryan Stitt dbd8ea2429 add logout endpoint and prefix with /rpc 2022-09-23 21:49:18 +00:00
Bryan Stitt 961ccf7cf2 ip, origin, referer, and user agent checks 2022-09-23 05:22:33 +00:00
Bryan Stitt d55aea2d98 even flatter 2022-09-22 22:42:44 +00:00
Bryan Stitt 7f7be6aa69 flatter 2022-09-22 22:34:43 +00:00
Bryan Stitt 987ed181cd comments 2022-09-22 22:16:28 +00:00
Bryan Stitt 6d23071640 less clones, but still some 2022-09-22 22:14:24 +00:00
Bryan Stitt 8d011e0cd1 pass db conn through 2022-09-22 22:10:28 +00:00
Bryan Stitt 3854312674 we need to clone :'( 2022-09-22 21:51:52 +00:00
Bryan Stitt 2ed2408ed3 even more authorized_request 2022-09-22 21:04:03 +00:00
Bryan Stitt 43846a7051 pass authorized_request through a bunch of places 2022-09-22 20:27:14 +00:00
Bryan Stitt 6905e9fd46 create a struct for authenticated requests that we need for per-key stats 2022-09-22 19:57:21 +00:00
Bryan Stitt 8481f6d44c wip 2022-09-21 23:50:55 +00:00
Bryan Stitt 339bd41f50 change rpm to u64 and create RevertLogs table 2022-09-21 19:55:43 +00:00
Bryan Stitt 37a1aa554b revert error saving and extract blockHash from requests 2022-09-21 04:48:21 +00:00
Bryan Stitt 6054c3f340 we do need fetch_sub 2022-09-21 02:48:02 +00:00
Bryan Stitt d98b6aeff3 split error counts 2022-09-20 06:59:10 +00:00
Bryan Stitt abe5001792 atomic and less expires 2022-09-20 06:56:24 +00:00
Bryan Stitt 739947792a instrument more. add max_wait to wait_for_request_handle 2022-09-20 06:00:27 +00:00
Bryan Stitt dfd6aed6e6 no timeouts here, we already have a timeout on requests 2022-09-20 03:26:12 +00:00
Bryan Stitt e35a4119e7 context is really handy 2022-09-20 01:37:15 +00:00
Bryan Stitt 6ae24b1ff9 use cache's expiration and race-free get_with
when this was a dashmap, we needed our own expiration and parallel requests would do the same query.

with moka, we can use their expiration code and get_with
2022-09-20 01:33:39 +00:00
Bryan Stitt 90fed885bc add ttl to user_cache 2022-09-19 23:09:31 +00:00
Bryan Stitt 8785cb6d2d no idle timeout or max lifetime 2022-09-19 22:41:53 +00:00
Bryan Stitt b2adc73b50 upgrade tokio-stream 2022-09-19 22:25:45 +00:00
Bryan Stitt 3e3a9fcf64 larger max_capacity now that there is a weigher 2022-09-19 22:24:56 +00:00
Bryan Stitt a7761a0430 add a weigher function to sized caches 2022-09-19 22:17:35 +00:00
Bryan Stitt b6275aff1e no need for an atomic bool 2022-09-19 22:17:24 +00:00
Bryan Stitt 28fa424c2a start weighers 2022-09-17 02:30:06 +00:00
Bryan Stitt 05af0551c9 use ahash
this might actually be slower. do real benchmarks
2022-09-17 02:17:20 +00:00
Bryan Stitt 3d4bfbfde0 use ahash. allow no redis 2022-09-17 01:19:11 +00:00
Bryan Stitt 12b6d01434 work in progress 2022-09-15 17:57:24 +00:00
Bryan Stitt 7a1433e3c9 no longer need to use total difficulty on ETH 2.0 2022-09-14 19:39:08 +00:00
Bryan Stitt 31227d8c0c drop in flight. throughput is enough 2022-09-14 07:08:48 +00:00
Bryan Stitt 731a255d2f add old block to log and more todos 2022-09-14 06:57:24 +00:00
Bryan Stitt ea8f6091f9 improve (but disable) rate limits 2022-09-14 06:41:34 +00:00
Bryan Stitt ed505cdd23 fix check for http providers
even the websocket providers have an http_client set. maybe drop that?
2022-09-14 06:38:53 +00:00
Bryan Stitt 1730b8c7a5 switch to deadpool 2022-09-14 06:18:13 +00:00
Bryan Stitt 789672be43 try fixing warning about missing blocks 2022-09-14 05:26:46 +00:00
Bryan Stitt a5ccec76c7 silence some warnings and make private rpcs an option 2022-09-14 04:27:18 +00:00
Bryan Stitt 1d4d5844d0 reconnect -> connect 2022-09-14 03:32:38 +00:00
Bryan Stitt 612f5a48c1 minor polish to error 2022-09-14 03:01:58 +00:00
Bryan Stitt 5a16b9eed8 fix reconnect for http clients 2022-09-14 02:11:48 +00:00
Bryan Stitt 7eaf6f3540 retrying reconnect 2022-09-14 01:43:09 +00:00
Bryan Stitt 73a2fcaf72 add FAQ 2022-09-14 01:42:48 +00:00
Bryan Stitt 519c312e8d reconnect needs to clear more state 2022-09-12 23:00:10 +00:00
Bryan Stitt dc30bdcef3 cargo upgrade 2022-09-12 14:36:26 +00:00
Bryan Stitt 05e94ff81c send UNAUTHORIZED to unknown keys 2022-09-12 14:33:55 +00:00
Bryan Stitt 7ff319e9b0 reconnect sooner 2022-09-12 14:33:19 +00:00
Bryan Stitt 88661fa865 return UnknownKey instead of error 2022-09-10 03:46:02 +00:00
Bryan Stitt d98886db14 simple status json and prometheus 2022-09-10 02:59:07 +00:00
Bryan Stitt 501f2b0b24 improve logging 2022-09-10 00:58:33 +00:00
Bryan Stitt 879c6e49f2 dry errors so that rate limits dont log so much 2022-09-10 00:12:14 +00:00
Bryan Stitt c8da98d12e even more metrics 2022-09-09 03:53:16 +00:00
Bryan Stitt cae034afb3 better metrics and spawn 2022-09-08 21:01:36 +00:00
Bryan Stitt cfca16319b lower uncle log level 2022-09-07 20:24:45 +00:00
Bryan Stitt 2c41cad452 more counts to the status page 2022-09-07 04:11:47 +00:00
Bryan Stitt c9b2c0c0d2 dry cache code 2022-09-07 03:54:16 +00:00
Bryan Stitt 5eef5173a1 disable redis rate limits 2022-09-06 22:55:17 +00:00
Bryan Stitt e4d25b207d better default connection counts 2022-09-06 20:50:37 +00:00
Bryan Stitt c89295ef7e we should use retry_at 2022-09-06 20:33:25 +00:00
Bryan Stitt c34e8ef383 improve rate limiting and request counters 2022-09-06 20:13:07 +00:00
Bryan Stitt 98265424bb set response id in another spot. this is too fragile 2022-09-06 17:56:49 +00:00
Bryan Stitt 041dd5f317 more logs and beginning of a health check 2022-09-06 16:49:07 +00:00
Bryan Stitt 83a7b03dea improve no-op subscription
can still be improved more
2022-09-06 13:14:15 +00:00
Bryan Stitt bb96757452 None instead of Block::default() more places 2022-09-06 12:29:37 +00:00
Bryan Stitt e8977d203b use None more 2022-09-06 03:26:23 +00:00
Bryan Stitt 4502344027 add head_block_id to connection serializer 2022-09-05 16:39:46 +00:00
Bryan Stitt 8080794a8b skip dupes earlier 2022-09-05 16:29:21 +00:00
Bryan Stitt 068c05cf4f improve fork logic again 2022-09-05 16:25:21 +00:00
Bryan Stitt 2092e74fd3 fix variable names and log to match 2022-09-05 15:49:53 +00:00
Bryan Stitt e261886c60 user_address change not made yet 2022-09-05 06:29:27 +00:00
Bryan Stitt a7cb3d00df small cleanup 2022-09-05 06:13:36 +00:00
Bryan Stitt 6e12edd555 use sized Caches 2022-09-05 05:53:58 +00:00
Bryan Stitt 62d747c1b3 Vec instead of IndexSet 2022-09-05 03:40:13 +00:00
Bryan Stitt 593eb461b8 more dashmap 2022-09-05 01:52:59 +00:00
Bryan Stitt 1c2f3e1445 dry user data caching 2022-09-03 19:43:19 +00:00
Bryan Stitt 8225285bb8 bug fix and better logs 2022-09-03 02:59:30 +00:00
Bryan Stitt 6f0ae1ec35 slower but correct save_block 2022-09-02 21:35:03 +00:00
Bryan Stitt 010669cf81 handle a missing block 2022-09-02 20:46:39 +00:00
Bryan Stitt ac6296c5ac better connection pool sizing 2022-09-02 20:16:20 +00:00
Bryan Stitt b259b56dee upgrade deps 2022-09-02 20:00:59 +00:00
Bryan Stitt 2606844c61 almost done with heaviest chain tracking 2022-09-02 05:40:56 +00:00
Bryan Stitt 6450a4cd42 more fork detection work 2022-09-01 05:58:55 +00:00
Bryan Stitt e5e137f76c fix deadlock 2022-08-30 20:01:42 +00:00
Bryan Stitt 11c66636bb rewrite cannonical block 2022-08-27 23:49:41 +00:00
Bryan Stitt 93fe878748 more use login things 2022-08-27 05:42:25 +00:00
Bryan Stitt 5e239c05c8 add min_sum_soft_limit to config 2022-08-27 03:13:23 +00:00
Bryan Stitt 8703532ed7 better logs and minor cleanup 2022-08-27 02:14:53 +00:00
Bryan Stitt 9fabb8e1e1 add min_synced_rpcs to config 2022-08-27 00:33:45 +00:00
Bryan Stitt 5719397466 i think it works 2022-08-26 23:44:25 +00:00
Bryan Stitt 81254a24be wip 2022-08-26 17:26:17 +00:00
Bryan Stitt 101104ac5d upgrade 2022-08-26 05:26:27 +00:00
Bryan Stitt 04dc716250 move more things into their own files 2022-08-24 00:59:05 +00:00
Bryan Stitt 72312a686b names instead of urls
urls might have secrets in them, and we dont want those in logs
2022-08-24 00:32:16 +00:00
Bryan Stitt 59eb9a889f rename and lints 2022-08-24 00:14:49 +00:00
Bryan Stitt 2cc1b8e297 even more files 2022-08-24 00:11:49 +00:00
Bryan Stitt 4d357977e9 split Web3Connections across multiple files 2022-08-23 23:56:47 +00:00
Bryan Stitt 3496c828b8 rearrange code 2022-08-23 23:13:56 +00:00
Bryan Stitt 8a49128eec comments 2022-08-23 20:50:59 +00:00
Bryan Stitt 31d30b0eed use an enum and database roles for auth checks 2022-08-23 20:42:58 +00:00
Bryan Stitt e65edc7795 add stub verify_auth_token function 2022-08-23 19:08:47 +00:00
Bryan Stitt 8f5e4a200e create bearer and save bearer separately 2022-08-23 18:56:19 +00:00
Bryan Stitt 0ca7b828b3 check ip rate limit on user post 2022-08-23 18:53:14 +00:00
Bryan Stitt 2989b7e91c dry redis connections and use bearer tokens 2022-08-23 18:48:27 +00:00
Bryan Stitt 94bc6fef8c remove unnecessary enum and try_into 2022-08-21 09:47:38 +00:00
Bryan Stitt bda666eb6c delete unused code after the rate limit refactor 2022-08-21 09:44:53 +00:00
Bryan Stitt b16aa8d813 dry rate_limit_by_x 2022-08-21 09:39:38 +00:00
Bryan Stitt 748674fe7a forgot to git add this 2022-08-21 08:30:49 +00:00
Bryan Stitt 634dc5572d upgrade 2022-08-21 08:30:43 +00:00
Bryan Stitt 5af834d710 it compiles 2022-08-21 08:27:52 +00:00
Bryan Stitt 9a2fe46105 multiple ways to sign 2022-08-19 20:18:12 +00:00
Bryan Stitt f06304e202 temp fix for routing to eth_getTransactionByHash and eth_getTransactionReceipt 2022-08-17 22:19:34 +00:00