Commit Graph

1135 Commits

Author SHA1 Message Date
yenicelik 4c157cfcf7 made paymentrequired error code instead of introducing a new one 2023-06-25 15:05:27 -04:00
yenicelik 935681fde7 stats subuser + premium access logic 2023-06-25 14:29:54 -04:00
Bryan Stitt fee1a423a1 simpler type when no args 2023-06-24 22:30:57 -07:00
Bryan Stitt 83123f5cdd wait up to a second for it to sync 2023-06-24 21:51:58 -07:00
Bryan Stitt e7baeb1ba4 rearrange so maybe time works better 2023-06-24 21:41:18 -07:00
Bryan Stitt c2917f1093 try sleeping 2023-06-24 21:35:36 -07:00
Bryan Stitt ba702c7647 tests pass local, but jenkins isn't as fast 2023-06-24 21:27:11 -07:00
Bryan Stitt b5ae11b7f7 lint 2023-06-24 21:21:51 -07:00
Bryan Stitt a440d51377 test fixture 2023-06-24 21:17:08 -07:00
Bryan Stitt 8cfbbfe1b3 use default derive helpers 2023-06-24 20:39:46 -07:00
Bryan Stitt efe1e30791 better handling of 0 port and fix test port 2023-06-24 20:38:06 -07:00
Bryan Stitt ab3d4a91a8 json body instead of params 2023-06-24 18:17:51 -07:00
Bryan Stitt 0c10994835 why wont that import? 2023-06-24 16:45:07 -07:00
Bryan Stitt 94d0a25aaf test logs 2023-06-24 16:43:11 -07:00
Bryan Stitt 38b266156a consistent route name 2023-06-24 15:49:26 -07:00
Bryan Stitt 1d6e1ef9af polish error logs 2023-06-24 15:24:46 -07:00
Bryan Stitt fef03f089f less verbose db_conn and db_replica helpers 2023-06-24 11:11:07 -07:00
Bryan Stitt 9115419ec0 handle addresses consistently 2023-06-24 10:20:24 -07:00
Bryan Stitt 39a505c0ff POST instead of GET and use more txs 2023-06-24 09:59:08 -07:00
Bryan Stitt 8f76d9320d support EIP1271 login 2023-06-24 09:48:31 -07:00
Bryan Stitt 91cbce6ce8 start adding stat for CU 2023-06-24 00:48:56 -07:00
Bryan Stitt 4e6a80caa2 cargo upgrade 2023-06-24 00:41:48 -07:00
Bryan Stitt 30828ccae2 prioritize block. queries to syncing nodes ends up being slower 2023-06-24 00:41:30 -07:00
Bryan Stitt ac4b7e46b4 actually import info level 2023-06-23 22:54:02 -07:00
Bryan Stitt 6edf1a5c1b info on new block 2023-06-23 22:52:42 -07:00
Bryan Stitt 89330cfba0 upstream ethers again 2023-06-23 22:23:51 -07:00
Bryan Stitt fd661689a3
Tracing (#144)
* tracing

* add tracing and features to match

* use just one hostname crate

* cargo upgrade

* set up pretty logger and sentry layer
2023-06-23 16:28:45 -07:00
Bryan Stitt 3fac4248d6 Revert "just use Ulid instead of RpcSecretKey type"
It did work

This reverts commit 377e8356a5.
2023-06-23 01:29:11 -07:00
Bryan Stitt 377e8356a5 just use Ulid instead of RpcSecretKey type 2023-06-23 01:28:47 -07:00
Bryan Stitt 68ab9b6d1d always serialize RpcSecretKey as a ULID 2023-06-23 01:07:54 -07:00
Bryan Stitt 6b6fac396f lower default web3_proxy::rpcs::consensus level 2023-06-22 22:45:43 -07:00
Bryan Stitt b1b8346db9
#123 role to rpc keys (#142)
* added roles to rpc keys

* dont unwrap

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-06-22 12:56:19 -07:00
Bryan Stitt c79b787e13
allow setting referral code if not already set (#141) 2023-06-22 12:41:05 -07:00
Bryan Stitt 376b9f474d use short syntax for json data in logs 2023-06-22 12:10:23 -07:00
Bryan Stitt f593667169 fix lifetimes 2023-06-22 10:16:58 -07:00
Bryan Stitt 66212b5cf5 cargo upgrade --incompatible -p itertools 2023-06-22 09:26:57 -07:00
Bryan Stitt dffff65950 allow no referrer_record 2023-06-22 09:25:12 -07:00
Bryan Stitt c560a59ef0 add more headers for debug endpoints and use more refs 2023-06-21 22:11:26 -07:00
Bryan Stitt 095a505017 cargo upgrade 2023-06-21 17:57:53 -07:00
Bryan Stitt 662bbb296f add header debug endpoint 2023-06-21 11:28:33 -07:00
Bryan Stitt 63c4f9e591 allow no ip
this shouldn't be needed, but is a workaround while we debug staging
2023-06-21 11:28:22 -07:00
Bryan Stitt 7f4da8d43c lint 2023-06-21 10:40:55 -07:00
Bryan Stitt d962b749a2 use find_with_related, not find_also_related
find_also_related is 1:1, but we want 1:Many
2023-06-21 10:37:01 -07:00
Bryan Stitt 1cbb58e8e0 comment 2023-06-21 10:25:10 -07:00
Bryan Stitt 06ce95c386 let serde_json do more of the work 2023-06-21 10:18:36 -07:00
Bryan Stitt 5504062f47 dont error if no referrals 2023-06-21 10:18:06 -07:00
Bryan Stitt 8f1b6fe6c0 lint 2023-06-21 10:02:02 -07:00
Bryan Stitt 4dad743fc3 tuple, not list 2023-06-21 10:01:51 -07:00
Bryan Stitt f75f3f8d2f serialize caches in a list 2023-06-21 10:00:05 -07:00
Bryan Stitt 3c2f95de10
Improve max wait and timeouts (#137)
* pass max wait with more functions

* move rpc_head_str higher up to use it in all the logs

* refresh consensus at double the block time

* new pricing that also includes archive check

* lint

* don't clone method

* put caches into one list

they serialize with their name, so it will still be easy to tell them apart

* more consistent waiting. still waits more than I want right now

* clean up default max waits

* more timeouts
2023-06-20 14:22:14 -07:00
Bryan Stitt 78efad5453 cargo upgrade strum 2023-06-19 17:48:46 -07:00
Bryan Stitt 6d25c41faf
Compute units (#136)
* wip

* add basic compute unit calculator

* calculate costs with compute units
2023-06-19 17:47:38 -07:00
Bryan Stitt b7935f77fc spawn recording latency 2023-06-19 17:46:58 -07:00
Bryan Stitt 6038351cb8 change default tracking in prep for premium-only keys 2023-06-19 13:00:57 -07:00
Bryan Stitt b35cd58a76 rate limit user_balance_post by bearer OR ip 2023-06-19 10:21:48 -07:00
Bryan Stitt c3ae4ded2b move data to the right places 2023-06-18 21:42:16 -07:00
Bryan Stitt 58cc129837 forgot to convert tier_sec_size from ms. 1 was a very large default 2023-06-18 17:57:21 -07:00
Bryan Stitt ba410cf126 better log message 2023-06-18 16:41:01 -07:00
Bryan Stitt 89009dcd42 use web3_context for more errors 2023-06-18 16:40:17 -07:00
Bryan Stitt af8543c3bf some defaults are needed maybe? 2023-06-18 16:36:23 -07:00
Bryan Stitt 2f78e64c5d create empty balance entry if none exists 2023-06-18 16:25:25 -07:00
Bryan Stitt 6deeabe1e1 pretty print errors 2023-06-18 16:14:06 -07:00
Bryan Stitt 9dd0bae40a text instead of string payload 2023-06-18 16:13:55 -07:00
Bryan Stitt 825ba006f1 move floats and durations around for latency tracking 2023-06-18 10:46:22 -07:00
Bryan Stitt 2f0e6103ac Revert "tier more important than backup"
This reverts commit bca22537c6.
2023-06-18 10:09:21 -07:00
Bryan Stitt bca22537c6 tier more important than backup
we filter backups when they aren't needed, so this should be okay
2023-06-18 10:06:26 -07:00
Bryan Stitt e54299beca rolling median for choosing tier 2023-06-18 09:53:36 -07:00
Bryan Stitt 8b22c9861b Revert "temp fix while filtering is improved"
This reverts commit 5367c51195.
2023-06-17 12:15:40 -07:00
Bryan Stitt 5367c51195 temp fix while filtering is improved 2023-06-17 12:09:04 -07:00
Bryan Stitt 3b65074f38 cleaner duration multiplication 2023-06-17 11:33:33 -07:00
Bryan Stitt eab86c7ee8 automatic max_head_block_age 2023-06-17 10:13:40 -07:00
Bryan Stitt bf13e86a04 warn if no max_head_block_age 2023-06-17 09:19:05 -07:00
Bryan Stitt 3b9314a76f comments 2023-06-17 08:44:18 -07:00
Bryan Stitt 9b9e5341cb lint 2023-06-17 08:44:07 -07:00
Bryan Stitt 6bfa5a69ca latest_balance is part of AuthorizationChecks now 2023-06-17 08:43:51 -07:00
Bryan Stitt 8ffd7d2e1b enable more advanced cost calculation 2023-06-16 23:55:41 -07:00
Bryan Stitt ba67151706 lint 2023-06-16 23:48:16 -07:00
Bryan Stitt 745870d119 drop a bunch of log levels 2023-06-16 23:46:20 -07:00
Bryan Stitt 74d2333039 try this version of influxdb2 2023-06-16 23:41:12 -07:00
David d6513cde74
Forgot to handle Option unwrap (#132)
* made the query much faster

* aggregate queries should be fast now too

* forgot to handle an option unwrap
2023-06-16 23:18:51 -07:00
Bryan Stitt 86f9e7358a
move balance onto AuthorizationChecks (#131)
* move balance onto AuthorizationChecks

* todone

* bigger default

* flat cost for testing
2023-06-16 23:14:43 -07:00
Bryan Stitt 5d9365449f simplify max item weight by having the weigher send MAX for big things 2023-06-16 16:14:11 -07:00
Bryan Stitt 1ec0f14144 upgrade to influxdb 1e5577e14150797584f5ed0ea7aba0bd68f0a678 2023-06-16 14:48:03 -07:00
Bryan Stitt cb71de7d14 pin influxdb2 to 6d3cf64d61967f41fafb57b3f4553206a8f3f2a3 2023-06-16 13:30:35 -07:00
Bryan Stitt d9a865fd1b cargo upgrade 2023-06-16 10:58:09 -07:00
Bryan Stitt 3928496be5 start tiers at 1 2023-06-16 10:52:13 -07:00
Bryan Stitt af7759e491 by_name needs a lock to prevent races during disconnects 2023-06-16 10:40:02 -07:00
Bryan Stitt 7c876ac561 delete stale code
this was rewritten and put into its own crate
2023-06-16 10:02:14 -07:00
Bryan Stitt 7707729f77 Merge remote-tracking branch 'origin/main' into devel 2023-06-16 10:01:17 -07:00
David 4772166e97
Make aggregate queries faster as well (#130)
* made the query much faster

* aggregate queries should be fast now too
2023-06-16 08:27:46 -07:00
Bryan Stitt fe8675ddf2 cargo upgrade 2023-06-16 00:48:22 -07:00
Bryan Stitt a083bc652d
Handle uncle transactions (#129)
* add more columns to handle uncled transactions

* handle payment uncles

* put relations back

* include all the new columns

* lower log levels

* improve block caching

if we have a block with a number, its canonical. uncles don't get returned

* improve disconnect logic

* lint

* clear first changed for new_top_config_receiver

* better logs around config changing

* i guess we do want one apply top_config at the start

* check correct variable for data limits
2023-06-16 00:46:27 -07:00
Bryan Stitt 6040ca297f fix eth_unsubscribe 2023-06-15 11:43:24 -07:00
Bryan Stitt 49c60ac1b5 improve websocket error handling 2023-06-15 09:50:21 -07:00
David 5859cd8a8d
made the query much faster (#127) 2023-06-15 09:38:51 -07:00
Bryan Stitt 957ba161b5 move table data around 2023-06-13 12:08:47 -07:00
Bryan Stitt b8f429d70a improvements to popularity_contesst 2023-06-13 11:51:19 -07:00
Bryan Stitt f5a1ac274a refactor send_head_block_result 2023-06-13 10:00:08 -07:00
Bryan Stitt 1473ccb085 gentler parsing of new head blocks 2023-06-13 09:31:22 -07:00
Bryan Stitt 602b0117c4 panic if influxdb_bucket is not set when influxdb_host is set 2023-06-13 09:30:41 -07:00
Bryan Stitt 45543be256 add more caches to /status 2023-06-12 23:03:38 -07:00
Bryan Stitt a7fb12bd72 less random is fine for shuffling 2023-06-12 22:42:22 -07:00
Bryan Stitt 55e8471b19 interval instead of watch_blocks 2023-06-12 22:26:10 -07:00
Bryan Stitt 6324d9c7d9 short circuit if no requests 2023-06-12 21:54:33 -07:00
Bryan Stitt 18835e09da allow empty batches 2023-06-12 21:51:08 -07:00
Bryan Stitt c192e03fa2 count all the tiers at once 2023-06-12 21:03:38 -07:00
Bryan Stitt bba7ccf7eb change divisor for tier calculation 2023-06-12 20:44:52 -07:00
Bryan Stitt c771479d94 use WyRand instead of Xoshiro 2023-06-12 19:46:36 -07:00
Bryan Stitt 1a8f799969
we need our own Reconnect (#124)
* first pass at handling different return data limits

* put ws_provider in an arcswap

* add min max_latency

* add min max_latency

* subscribe with reconnect

* better logging around reconnect

* select on both watches

* subscribe to the correct watch
2023-06-12 18:58:51 -07:00
Bryan Stitt 62bda636b8 return the response instead of panicing 2023-06-12 16:38:57 -07:00
Bryan Stitt c456432cc3 cargo upgrade 2023-06-12 11:44:51 -07:00
Bryan Stitt c3a7d1646c lint 2023-06-12 11:41:10 -07:00
Bryan Stitt 70d5c09d3e clean up redirect url 2023-06-12 11:38:34 -07:00
Bryan Stitt f1176685dd pause time on more tests 2023-06-12 11:08:33 -07:00
David 093ca19454
added subuser RPC keys into key access control, as well as fetching RPC keys (#115) 2023-06-12 08:59:23 -07:00
David 0cc4557e8d
Use rpc id instead of key (#118)
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.

* Revert "added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc."

This reverts commit 022a13cfd9ca24aad9e05f75489396936f14784a.

* mixed up branches
2023-06-11 19:14:02 -07:00
David 81aa4788b4
David/60 add basic email verification (#119)
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.

* added rpc key logic to use db numbers instead of UUIDs

* Revert "added rpc key logic to use db numbers instead of UUIDs"

This reverts commit 056771d02725f90ed5f1745b0372e13e630a19ac.
2023-06-11 19:10:10 -07:00
yenicelik 0b26ae3bad made modify-subuser post 2023-06-11 18:59:27 +02:00
yenicelik 2333b8bbb6 influx query is not 20-40% faster 2023-06-11 17:57:02 +02:00
Bryan Stitt 4a9c1a0ce6 move user registration into a dedicated function 2023-06-09 18:31:47 -07:00
Bryan Stitt 71d3d63524 put ip rate limiting back 2023-06-09 16:35:44 -07:00
Bryan Stitt 41950c886c improvements for web3-this-then-that 2023-06-09 16:35:14 -07:00
Bryan Stitt bcf4f33543 add payment_factory_address to /status 2023-06-09 14:02:29 -07:00
Bryan Stitt bb900c61a5 todo 2023-06-09 13:31:46 -07:00
Bryan Stitt 8bf0ee473d loud log on rollback 2023-06-09 13:30:00 -07:00
Bryan Stitt bd87fcb13c move sort and shuffle for loadbalancing into proper functions 2023-06-09 13:09:58 -07:00
Bryan Stitt 1f3040f6c7 automated tiers 2023-06-09 12:21:55 -07:00
Bryan Stitt b6ff14210d cargo upgrade 2023-06-08 22:35:14 -07:00
David d91713e09e
this brings back balance back to the stats request (#112)
* removed bloom filter temporarily, added some fixes with decimals in payment.rs

* balance is inside the influx query again

* fixed aggregate
2023-06-08 15:57:53 -07:00
Bryan Stitt 68f73ec0b1 more things should be BadRequest instead of 500 2023-06-08 13:42:45 -07:00
Bryan Stitt 50ec653557 lower log level 2023-06-08 11:46:38 -07:00
Bryan Stitt 74280bc329 lint 2023-06-08 11:33:16 -07:00
Bryan Stitt 334b1c21b3 improve logs 2023-06-08 11:30:48 -07:00
Bryan Stitt 1d05bc5e89 lower log levels 2023-06-08 11:26:09 -07:00
David 3f76b08364
Test balance and referral accounting (#111)
* removed bloom filter temporarily, added some fixes with decimals in payment.rs

* balance accounting for a single user (paid and free tier) seems to be ok

* fixed some balance accounting

* compute_cost reduced back to 0
2023-06-08 10:08:29 -07:00
Bryan Stitt c37b320286 add cache names and actually serialize 2023-06-07 17:55:34 -07:00
Bryan Stitt 5744b459ee lint 2023-06-07 17:43:10 -07:00
Bryan Stitt 6cf738261b fix test 2023-06-07 17:42:34 -07:00
Bryan Stitt bfaff81626 cargo upgrade 2023-06-07 17:31:29 -07:00
Bryan Stitt 5c1569f9c2 clippy lint 2023-06-07 17:31:15 -07:00
Bryan Stitt 40bee30730 back to moka 2023-06-07 17:30:05 -07:00
David de7d8919d3
Change balance to record total spend and total deposits (#109)
* lets test total deposit and total spent

* removed referrer from cache for performance reasons
2023-06-07 14:45:57 -07:00
Bryan Stitt 4f7144abc6 Arc instead of Box for better caching 2023-06-07 13:57:38 -07:00
Bryan Stitt c2a3a4a0a9 Merge remote-tracking branch 'origin/main' into devel 2023-06-07 13:29:12 -07:00
Bryan Stitt 91d288315f add queue lengths to status page 2023-06-07 13:05:15 -07:00
Bryan Stitt b50579ea3a add block caches to web3rpcs serializer 2023-06-07 12:04:39 -07:00
Bryan Stitt 8249b0aefe cache isnt on main yet 2023-06-07 11:18:36 -07:00
Bryan Stitt aa57dd2fcf serialize the caches on /status 2023-06-07 10:58:30 -07:00
Bryan Stitt f0a9f03076 smaller caches 2023-06-07 10:57:43 -07:00
Bryan Stitt c4b2ce5266 smaller caches 2023-06-07 10:57:01 -07:00