Commit Graph

1438 Commits

Author SHA1 Message Date
Bryan Stitt 633abc0d60 make wait_for_best_rpc work for private/4337/groups without head subscriptions 2023-09-02 10:59:07 -07:00
Bryan Stitt 2cf688cc68 fix one syntax error 2023-09-02 10:38:26 -07:00
Bryan Stitt 5ce1b5bace add web3_bundlerVersion with the same cost as web3_clientVersion 2023-09-02 10:30:42 -07:00
Bryan Stitt 8d51a1b4dd remove stale comment 2023-09-02 10:20:50 -07:00
Bryan Stitt bd77a6cac4 cargo upgrade and update 2023-09-02 10:20:30 -07:00
Bryan Stitt 428dfdd835 improve 4337 bundler support and retries 2023-09-02 10:19:43 -07:00
Bryan Stitt a487ce5149 skip backups unless backups are needed for ranked_rpcs to exist 2023-09-01 17:01:59 -07:00
Bryan Stitt 57f5d5b483 cargo upgrade and update 2023-09-01 16:58:02 -07:00
Bryan Stitt f415315626 check more possible empty values 2023-09-01 16:56:20 -07:00
Bryan Stitt 7530605e5b v1.42.14 2023-08-30 21:06:09 -07:00
Bryan Stitt 484610b80f cargo upgrade and update 2023-08-30 21:05:34 -07:00
Bryan Stitt 0ff3ccd479 eth = 0.0000004. everything else to match polygon (0.0000005333) 2023-08-30 21:02:31 -07:00
Bryan Stitt ec533c8b30 cargo upgrade and update 2023-08-30 20:59:06 -07:00
Bryan Stitt df12b3dc73 cargo upgrade and update 2023-08-24 14:37:04 -07:00
Bryan Stitt e76bef272f put rate limit things into data 2023-08-24 13:25:14 -07:00
Bryan Stitt a32e2ddcad cargo upgrade 2023-08-22 13:27:19 -07:00
Bryan Stitt 2acca1fbbc unimplemented compute units for alchemy_ method 2023-08-22 13:27:19 -07:00
Bryan Stitt 8b4aba592c cargo fmt 2023-08-22 10:49:04 -07:00
Bryan Stitt d036806a61 cargo upgrade and cargo update 2023-08-22 10:42:59 -07:00
Bryan Stitt 372c3c60fd shorter cache and timeout on /status and related endpoints 2023-08-22 10:42:02 -07:00
Bryan Stitt 99d405f7db Revert "remove caching the /status things"
This reverts commit c5827b6f37.
2023-08-22 10:37:12 -07:00
Bryan Stitt c5827b6f37 remove caching the /status things 2023-08-21 15:45:23 -07:00
Bryan Stitt dbda653812 trim trailing slashes isn't working as expected. put the routes back for now 2023-08-21 15:18:57 -07:00
Bryan Stitt eed03d2bfe cargo upgrade 2023-08-18 16:18:30 -07:00
Bryan Stitt 24058e04f1 bump version to 1.42.10 2023-08-18 16:01:32 -07:00
Bryan Stitt d2aa67e8ee test more methods 2023-08-17 21:55:58 -07:00
Bryan Stitt 10a8620044 fix method not found jsonrpc error codes 2023-08-17 15:32:22 -07:00
Bryan Stitt 6be05bab4b cargo upgrade 2023-08-17 13:49:45 -07:00
Bryan Stitt e0f700189b fix error code on method not implemented and error_response user_response bools 2023-08-17 13:44:26 -07:00
Bryan Stitt 36eb520f84 rename block_ to block_and_rpc_ where applicable 2023-08-16 13:20:22 -07:00
Bryan Stitt 5b9ef30e64 include user_error_response in aggregate stats 2023-08-15 17:08:11 -07:00
Bryan Stitt 54341c4654 learn to cache more 2023-08-15 16:37:53 -07:00
Bryan Stitt 9bd18250fe bump 1.42.6 2023-08-15 16:21:13 -07:00
Bryan Stitt 70d75d5acb exit early when empty params array 2023-08-15 16:20:25 -07:00
Bryan Stitt b742a25729 dont return max depth so we can do cannonical block faster 2023-08-15 16:12:40 -07:00
Bryan Stitt aa71a406bb cargo upgrade 2023-08-15 15:39:13 -07:00
Bryan Stitt 805cb6ad10 ignore alchemy_ methods 2023-08-08 17:40:26 -07:00
Bryan Stitt 891243ed46 cargo upgrade 2023-08-08 17:40:17 -07:00
Bryan Stitt cee4603b55 add check_balance command 2023-08-08 16:18:07 -07:00
Bryan Stitt 3f932b9d57 also bump cli version 2023-08-08 15:58:21 -07:00
Bryan Stitt a844c5f624 remove accidental todos 2023-08-08 15:52:11 -07:00
Bryan Stitt 8ea587120b handle functions that need the block num instead of the hash 2023-08-08 15:50:34 -07:00
Bryan Stitt 5f07213c8d new nightly supposedly compiles faster 2023-08-08 14:14:34 -07:00
Bryan Stitt d56d39c82f cargo upgrade 2023-08-07 15:40:59 -07:00
Bryan Stitt c38c27a3d1 lint 2023-08-07 15:25:14 -07:00
Bryan Stitt bc8040f61d more yields and less timeouts 2023-08-04 18:54:16 -07:00
Bryan Stitt aed98ac034 try yielding around all selects 2023-08-04 18:37:46 -07:00
Bryan Stitt 5257329559 cargo upgrade 2023-08-04 16:19:43 -07:00
Bryan Stitt 0bc523ab9f improve hard limits
rate limit per server or per endpoint and with any period
2023-08-03 12:24:28 -07:00
Bryan Stitt 690601643d improve eth_sendRawTransaction and other timeouts 2023-08-03 11:39:48 -07:00
Bryan Stitt 74224977b7 Revert "Revert "remove max_tries for now""
This reverts commit 5fbcb75157.
2023-08-03 09:15:31 -07:00
Bryan Stitt ec0b733c8e cache 16 MiB responses
we should really just get this from config
2023-08-03 01:19:12 -07:00
Bryan Stitt ca802751c1 if block fails, try other rpcs 2023-08-03 00:29:13 -07:00
Bryan Stitt 5747caec71 break if no max_wait 2023-08-03 00:23:44 -07:00
Bryan Stitt 5fbcb75157 Revert "remove max_tries for now"
This reverts commit e0d7d11398.
2023-08-03 00:22:45 -07:00
Bryan Stitt ab6d3a1d9b split command line into separte module 2023-08-03 00:16:54 -07:00
Bryan Stitt e0d7d11398 remove max_tries for now
i think this is causing problematic looping
2023-08-02 22:07:28 -07:00
Bryan Stitt 8aac4779bf clean up some selects 2023-08-02 21:27:56 -07:00
Bryan Stitt e70eeb11e0 dont check flush 2023-08-02 15:25:04 -07:00
Bryan Stitt d25884dfbb dont cache responses larger than 2MB 2023-08-02 15:22:26 -07:00
Bryan Stitt 79542c94ae improve tests 2023-08-01 12:09:24 -07:00
Bryan Stitt 7cd91af3a8 don't cache nulls 2023-07-31 14:26:07 -07:00
Bryan Stitt 2885bc6ef2 cargo upgrade 2023-07-31 12:58:20 -07:00
Bryan Stitt ca92b3646c cache eth_feeHistory and eth_gasPrice 2023-07-28 10:41:23 -07:00
Bryan Stitt 1185b2d0f9 cache eth_maxPriorityFeePerGas with the head block 2023-07-28 10:38:12 -07:00
Bryan Stitt a696fb92f6 disable CheckEmailInput for now 2023-07-27 13:52:12 -07:00
Bryan Stitt 32a73a0c53 cargo upgrade --incompatible 2023-07-27 12:29:10 -07:00
Bryan Stitt ebfb4c6386 cargo upgrade 2023-07-27 12:29:10 -07:00
Bryan Stitt 9c458c9ee0 add grant_credits_to_address script 2023-07-27 12:29:10 -07:00
Bryan Stitt f9dd827439 add a log message about how many users are being upgraded 2023-07-27 12:29:10 -07:00
Bryan Stitt 67fd5a2188 change log levels 2023-07-26 16:42:49 -07:00
Bryan Stitt d2f0d45dd2 reduce out of funds tier to match free config and bump to version 1 2023-07-26 16:40:08 -07:00
Bryan Stitt 788ebff29f improve logging during payment parsing 2023-07-26 16:39:17 -07:00
Bryan Stitt 2b14842879 check the payment factory address against the log address 2023-07-26 13:22:13 -07:00
Bryan Stitt 4d8836538d drop old fk 2023-07-26 09:33:15 -07:00
Bryan Stitt b1f447c5c8
add mass_grant_credits helper script and other prep for premium (#196)
* add mass_grant_credits helper script

* include error in the rate limit log

* allow granting 0 credits

* migration to change default user_tier to premium
2023-07-26 07:33:00 -07:00
Bryan Stitt b80f994e90 timestamps instead of tags for cardinality reduction 2023-07-25 21:45:01 -07:00
talking-llama e238203bbc
Add OtterScan Methods and corresponding CUs (#195)
Co-authored-by: Rory Neithinger <rorytrent@gmail.com>
2023-07-25 21:06:03 -07:00
Bryan Stitt c831a810b4 error level for anyhow errors 2023-07-25 15:15:03 -07:00
Bryan Stitt 05e618f74c
Anon rpc accounting (#194)
* cargo generate migrations didnt work without this

* anon rpc accounting

* Revert "anon rpc accounting"

This reverts commit f1b8cbcdeca0d5f9545da32f11dda6a90854f97a.

* 0 for anon in mysql

* only set approximate_balance_remaining on owned stats

* only set self.approximate_balance_remaining when adding if some

* todo complete
2023-07-24 22:02:33 -07:00
Bryan Stitt 9b361cb849 cargo upgrade 2023-07-24 21:59:37 -07:00
Bryan Stitt 439ac49495 jsonrpc formatted errors for JsonRejection 2023-07-24 16:25:58 -07:00
Bryan Stitt ba1ce91322 remove assert that doesnt work well with interval flushes 2023-07-22 02:41:27 -07:00
Bryan Stitt 22400e9c8b needs a string, not a ulid 2023-07-22 02:32:54 -07:00
Bryan Stitt e01b3c2806 watch for SIGTERM 2023-07-22 02:30:49 -07:00
Bryan Stitt 4dfe092e08 fix tests 2023-07-22 02:30:39 -07:00
Bryan Stitt 12f3f940a0 include the window in the tsdb logs 2023-07-22 01:56:13 -07:00
Bryan Stitt 35b03cffad lint 2023-07-22 01:41:07 -07:00
Bryan Stitt 80d50bc2c0 need unique names in tests 2023-07-22 01:38:54 -07:00
Bryan Stitt 4091e05f3d influx id from config. default to the hostname or fallback to ulid 2023-07-22 00:26:02 -07:00
Bryan Stitt 7c4de5ea8b give the databases longer to start 2023-07-21 23:34:54 -07:00
Bryan Stitt df027527e9 remove origin (pt 2) 2023-07-21 22:47:43 -07:00
Bryan Stitt 252b3001c1 increment a wrapping counter every time we save stats 2023-07-21 22:47:22 -07:00
Bryan Stitt 39141c1e52 comments 2023-07-21 22:14:31 -07:00
Bryan Stitt 185ce7a3b4 flush tsdb every 10 2023-07-21 16:39:27 -07:00
Bryan Stitt 88b2618edf add to the running total everywhere 2023-07-21 16:38:02 -07:00
Bryan Stitt b08a279b60 add another field to try to handle improper de-duplication 2023-07-21 16:29:10 -07:00
Bryan Stitt 0ef8bb50e8 drop x at the end of most tests 2023-07-21 16:18:33 -07:00
Bryan Stitt 620f2d9d00 30 second flushes for db too 2023-07-21 16:12:43 -07:00
Bryan Stitt 0a3698de0f 30 seconds for tsdb everywhere 2023-07-21 15:51:32 -07:00
Bryan Stitt 95bf1db01e save tsdb less often 2023-07-21 15:25:12 -07:00
Bryan Stitt f15f1027cf stat buffer count total requests and print on exit 2023-07-21 15:25:02 -07:00
yenicelik f7cfd39d93 Revert "Merge branch 'devel' of github.com:llamanodes/web3-proxy into devel"
This reverts commit 4fc745b52b, reversing
changes made to bc6d18041d.
2023-07-21 18:03:55 -04:00
yenicelik 4fc745b52b Merge branch 'devel' of github.com:llamanodes/web3-proxy into devel 2023-07-21 18:03:03 -04:00
yenicelik bc6d18041d added single proxy test 2023-07-21 18:02:07 -04:00
Bryan Stitt 489cd7c326 drop Origin (it was none everywhere already anyways) 2023-07-21 14:50:38 -07:00
Bryan Stitt d4518cd2bf spans on the stat add function 2023-07-21 14:27:43 -07:00
Bryan Stitt dc4b6d501c move active_premium opt_in check 2023-07-21 13:58:09 -07:00
Bryan Stitt 044d19e3ec more info in Balance's Debug 2023-07-21 13:57:47 -07:00
Bryan Stitt e6666d92d8 don't unwrap if config cant send 2023-07-21 13:28:08 -07:00
Bryan Stitt a68d0d60b7 only health check if theres a block and rpc sender 2023-07-21 13:28:00 -07:00
Bryan Stitt 7fa88e7251 keep the cache size consistent 2023-07-20 22:34:32 -07:00
Bryan Stitt 5b9eb080a2 include influx in test_sum_credits_used 2023-07-20 22:21:46 -07:00
Bryan Stitt f5036f6cd9 set tiers to fix tests 2023-07-20 22:07:46 -07:00
Bryan Stitt c674871c98 only create owned key if active premium 2023-07-20 21:13:55 -07:00
Bryan Stitt e222572d09 rpc_key_user_id should be None for global stats 2023-07-20 21:13:24 -07:00
Bryan Stitt 0b1cfeffed use normalize path layer to trim trailing slash 2023-07-20 20:05:37 -07:00
Bryan Stitt 03bfd7148f test_two_buffers needs docker 2023-07-20 20:05:22 -07:00
Bryan Stitt cb5c1c6582 no limits on rpc_secret_key_cache and user_balance_cache 2023-07-20 19:04:32 -07:00
Bryan Stitt 766e1939d5 cargo upgrade 2023-07-20 18:23:04 -07:00
Bryan Stitt 2611085c00 add simple stat buffer test 2023-07-20 18:22:25 -07:00
Bryan Stitt 7678cb7389 caches aren't actualy optional on buffer 2023-07-20 18:21:49 -07:00
Bryan Stitt 3334dc5c0b mark as premium if no downgrade tier id 2023-07-20 14:54:42 -07:00
Bryan Stitt ae8a12047e tarpit strange methods 2023-07-20 14:32:52 -07:00
Bryan Stitt e7905b4344 Revert "First pass implementation of async serialize for Web3Rps (#192)"
This reverts commit 269aa32260.
2023-07-19 23:12:49 -07:00
Bryan Stitt 6c6a2406b7 cargo upgrade 2023-07-19 22:55:53 -07:00
Bryan Stitt ac2d9049be improve flushing and buffering of stats 2023-07-19 22:55:14 -07:00
Bryan Stitt 90ffb5254a flush all stats 2023-07-19 18:41:40 -07:00
Bryan Stitt e6dc64f369 add timeout as a the tower layer 2023-07-19 18:41:26 -07:00
Bryan Stitt 1b8ae5dee0 cargo upgrade 2023-07-18 09:54:56 -07:00
Bryan Stitt a97cb6a0c5 lint 2023-07-18 09:50:51 -07:00
Bryan Stitt 600c1bafb4
David/multiple proxy test (#187)
* add test_multiple_proxies_stats_add_up

* make a premium user using both proxies

* added a couple clones, must add constraints now to run for multiple-proxies (check arithmetic)

* lint and code review

* fix comment

* fix tests (now fails at todo)

* will introduce endpoint to fetch rpc stats from mysql

* added influxdb to tests, should next do asserst in stats collected by influx, and mysql for multi-proxy

* created test where influx and mysql have separate data, should help with debugging

* forgot to drop influx

* tests pass except multi-proxy

* test passes, will check out nothing broke elswhere

* go back to numbers

* some linting

* linting

* removed redundant info!

* responding to PR comments

* ULID as instance-hash for the tag in influx (for anti-dup)

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-07-17 13:31:41 -07:00
Bryan Stitt fe672b5189 dont disable on_failure. the log breadcrumbs can be helpful still 2023-07-14 23:04:24 -07:00
Bryan Stitt 140dc04a8b dont put anon stats in the db (for now)
the indexes need to be fixed. UNIQUE+NULL do not play nice
2023-07-14 22:19:18 -07:00
Bryan Stitt 264508c454 treat serde errors as rate limits (because all of them seem to be) 2023-07-14 21:43:37 -07:00
Bryan Stitt 042e644865 comment 2023-07-14 18:38:13 -07:00
Bryan Stitt 61126bb2ac one more Cow 2023-07-14 18:35:40 -07:00
Bryan Stitt 020dff0233 lint 2023-07-14 18:31:42 -07:00
Bryan Stitt efa189d101 less anyhow and fix tests 2023-07-14 18:31:00 -07:00
Bryan Stitt 0046a02a4a
Global db (#191)
* dont migrate on start. just connect

* it compiles

* that log is loud

* get transaction from a local clone

* import the trait

* put the test app into a thread instead of tokio::spawn

* fix one test

* try db before rpcs

* db connection is too slow. need to wait for it

* do db setup once while spawning
2023-07-14 18:30:01 -07:00
Bryan Stitt 269aa32260
First pass implementation of async serialize for Web3Rps (#192)
Co-authored-by: Rory Neithinger <rorytrent@gmail.com>
2023-07-14 18:29:48 -07:00
Bryan Stitt fe45813e4c remove info logging 2023-07-14 01:21:50 -07:00
Bryan Stitt 37e3d0be2b you cant do a runtime in a drop like this apparently 2023-07-14 01:19:27 -07:00
Bryan Stitt 6371206315 rate limit by ip if unknown key 2023-07-14 00:23:05 -07:00
Bryan Stitt f217bf33b3 disable db stats for now 2023-07-13 23:45:04 -07:00
David 31f840432d
Add DateCreated fields to admin and on-chain deposit (#190)
* tests seem to be passing

* bump version

* bump version
2023-07-13 19:25:14 -07:00
Bryan Stitt 3c2d06324e faster rpc. backups always lose 2023-07-13 18:14:13 -07:00
Bryan Stitt 45af241429 less db transactions 2023-07-13 15:44:07 -07:00
Bryan Stitt 5d5e65ed40 close the db connection on exit 2023-07-13 15:44:07 -07:00
Bryan Stitt a893a41c90 rename get to connect 2023-07-13 15:44:07 -07:00