Commit Graph

470 Commits

Author SHA1 Message Date
Bryan Stitt
5d207fb2c6
Auto tier change (#184)
* lint

* change user to premium on admin credits

todo: change on other deposits

* set tier more places

* BadRequest instead of 500

* insert existing users too

* add the premium file
2023-07-12 10:24:16 -07:00
Bryan Stitt
df2f3d340f
More balance tests (#182)
* fix popularity contest

* more info in the Debug for Web3Rpc

* add frontend_requests and cache_misses to the Balance query

* add more to balance and stats flushing and improved test coverage

* it compiles

* deserializer for Ulid to Uuid

I think a wrapper type on Ulid that implements sea_orm::Value is probably better

* rename variable to match struct name

* add deserializer for Address -> Vec<u8>

* sql sum returns a Decimal. need to convert to u64

* assert more

* one log and assert more

* log more

* use a helper to get the user's rpc provider

* this should be 2 now that we have a public and authed call

* this should be zero. the public has the cache miss

* instrument cu calcs

* trace the value we took, not the default that replaced it

* move usd_per_chain into config

* remove some extra logging

* use Arc::into_inner to maybe avoid a race

* off by 1

* pass paid credits used instead of returning it

this lets us use it to write to our user balance cache first. importantly, this keeps us from holding a write lock while writing to mysql

* no cache misses expected in this test

* actually check the admin

* put the balance checks back now that the rest of the test works

* archive request is being set incorrectly

* wow howd we manage flipping the greater than sign on archive depth

* move latest_balance and premium_credits_used to before any stats are emitted

* lint

* and build undoes the linting. fun

i didnt even want to lint them in the first place, so this is fine

* missed incrementing total_spent when not incrementing total_spent_paid_credits

* use the credits on self

* use the credits on self (pt 2)

* fix type for 10 cu query

* convert the requestmetadata on the other side of the channel

* logs

* viewing stats is allowed even without a balance

* move paid_credits_used to AuthorizationChecks

* wip

* test_sum_credits_used finally passes

* UserBalanceCache::get_or_insert

* re-enable rpc_secret_key_cache

* move invalidate to a helper function

and always call it **after** the db is commited

* fix PartialEq and Eq on RpcSecretKey

* cargo upgrade
2023-07-12 00:35:07 -07:00
Bryan Stitt
0f7e370c92 drop flume. just use tokio
i dont know if this is the problem, but this will also get compile times down
2023-07-10 23:08:06 -07:00
Bryan Stitt
2dfc5dcdf3 set http keep alive to a longer time 2023-07-10 18:37:25 -07:00
Bryan Stitt
a7119f6b5b
split balance queries up (#177)
* wip

* tests pass, referee_bonus changed to one_time_referee_bonus

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-07-10 14:03:05 -07:00
Bryan Stitt
9ef08c80e1 remove parking_lot 2023-07-10 11:05:07 -07:00
Bryan Stitt
9ee6b18172 no more lock update needed
all these things are idempodent now
2023-07-09 22:51:01 -07:00
Bryan Stitt
a43fda0c32
Bryan devel (#176)
* refactor more balance logic

* refactor more balance checks

* doc running full test suite
2023-07-09 21:37:50 -07:00
Bryan Stitt
a560d47c48 lint 2023-07-09 20:13:03 -07:00
Bryan Stitt
b527f5d0d6
David/remove balance logic non destructive (#173)
* WIP will work on locks

* added concurrent test for referral, numbers rn still add up

* will change balance table to be a view instead of a table

* merged in devel. will proceed with migrations for rpc_stats_v2 and referral

* WIP

* WIP

* WIP compiling

* gotta add the balance_v2 view

* about to add balance view

* stupid missing closing param

* compiles again, now i will just have to implement the raw sql

* need to paste the real query and play with it

* ok now on to testing

* addresses many comments in PR

* app works for simple unauthorized access. will look into tests

* some tests pass, referral logic tests fail

* not sure why the test is failing

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-07-09 19:23:32 -07:00
Bryan Stitt
386363e295 add request id to tracing 2023-07-08 11:57:53 -07:00
Bryan Stitt
9e89304eb7 try_send instead of send 2023-07-07 21:58:00 -07:00
Bryan Stitt
b234265458
Premium tier changes (#170)
* lower premium concurrency in preparation for key+IP limits

* include the ip in the user semaphore

* 3, not 5

this is our current limit for free

* per user_id+ip rate limiting
2023-07-07 15:15:41 -07:00
Bryan Stitt
651244fbca lower log levels 2023-07-07 14:14:03 -07:00
Bryan Stitt
512bcf305b lint 2023-07-07 14:06:35 -07:00
David
f6337ce940
Update payment_stripe.rs 2023-07-07 08:37:47 -04:00
Bryan Stitt
cf3d984548
switch the lock type (#165) 2023-07-06 14:54:52 -07:00
yenicelik
af45f78ff5 added test for admin deposit endpoint, fixed bugs 2023-07-06 08:19:33 -04:00
David
659fce0e33
Deposit history for stripe and admin (endpoints) (#160)
* added endpoints for stripe and admin deposits

* modified comments
2023-07-05 19:26:03 -07:00
Bryan Stitt
eb7b98fdbe
Split errors (#158)
* add migration for splitting errors

* fix type from u32 to u64

* update entities to match migrations

* no migration needed. these are only in stats

* add user_error_response to influx

* only if detailed

* set error_response and user_error_response

* 0 cost error responses

* only 33 migrations now

* put macros back

* get the stat buffer sender to the TestApp helper

* fixes
2023-07-05 18:18:10 -07:00
Bryan Stitt
21b59bb953 rename stripe_api_key to stripe_whsec_key 2023-07-05 11:26:39 -07:00
Bryan Stitt
a64e32b085 remove bearer token semaphore. it isn't really necesary
we can come back to rate limiting authenticated endpoints later
2023-07-03 16:54:25 -07:00
Bryan Stitt
f0cdb3b234 flatten routes 2023-07-03 15:59:06 -07:00
Bryan Stitt
d613969c3e Merge remote-tracking branch 'origin/devel' into david/test-admin 2023-07-03 14:07:43 -07:00
Bryan Stitt
ee6b725a1b lower log level on payload and headers 2023-07-03 13:34:16 -07:00
Bryan Stitt
d9c6da74c2 fix case on currency check 2023-07-03 13:30:37 -07:00
Bryan Stitt
d8512d50de just pass the whole string through
based on this test (which i think async-stripe forked): https://docs.rs/stripe-rust/0.12.3/src/stripe/resources/event.rs.html#355

and then based on the struct that they deserialize into: https://docs.rs/async-stripe/latest/src/stripe/resources/generated/event.rs.html#16
2023-07-03 12:57:52 -07:00
Bryan Stitt
b13e5b2440 move more logs to tracing and possible stripe payment fix
maybe RawValue does need to_string instead of just get
2023-07-03 12:17:30 -07:00
Bryan Stitt
377cd8bcf3 one more stripe option
need to check if case matters
2023-07-01 09:12:50 -07:00
Bryan Stitt
7044edc63c check multiple posible stripe headers 2023-07-01 09:10:53 -07:00
yenicelik
17c42243d0 wont shutdown 2023-06-30 22:24:48 -04:00
Bryan Stitt
c26d57fe5e test log in and out 2023-06-30 12:53:21 -07:00
Bryan Stitt
581af0d596
stripe integration first mvp (replaces #140) (#154)
* stripe integration first mvp

* addressing some points from #140

* stripe endpoint needs testing

* addressed some comments

* added Json(payload)

* update toolchain

* lint (well, most of it)

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-06-29 14:32:05 -07:00
Bryan Stitt
f6c8172a13 always do connectinfo for now 2023-06-29 13:12:35 -07:00
Bryan Stitt
280e2075d3
Fix Issue 65 Pt. 2 (#153)
* change subuser balance logic and add message to access denied errors

* clearer import

* comments

* comments
2023-06-28 23:00:34 -07:00
Bryan Stitt
ebceb0d7d6 descriptive text on NotImplemented errors 2023-06-28 21:47:06 -07:00
Bryan Stitt
fba41b9bc2 lots of tracing improvements 2023-06-28 21:30:00 -07:00
Bryan Stitt
3cfbc5baa4 improve cache keys and allow failures while parsing params 2023-06-28 17:42:49 -07:00
Bryan Stitt
cdea61cb6b one list for ranked rpcs 2023-06-27 15:55:29 -07:00
Bryan Stitt
6f7783d260 add block stats to top of status data 2023-06-26 21:11:16 -07:00
yenicelik
88617dc72b added more tracing for balance 2023-06-25 17:25:35 -04:00
Bryan Stitt
efe1e30791 better handling of 0 port and fix test port 2023-06-24 20:38:06 -07:00
Bryan Stitt
ab3d4a91a8 json body instead of params 2023-06-24 18:17:51 -07:00
Bryan Stitt
38b266156a consistent route name 2023-06-24 15:49:26 -07:00
Bryan Stitt
1d6e1ef9af polish error logs 2023-06-24 15:24:46 -07:00
Bryan Stitt
fef03f089f less verbose db_conn and db_replica helpers 2023-06-24 11:11:07 -07:00
Bryan Stitt
9115419ec0 handle addresses consistently 2023-06-24 10:20:24 -07:00
Bryan Stitt
39a505c0ff POST instead of GET and use more txs 2023-06-24 09:59:08 -07:00
Bryan Stitt
8f76d9320d support EIP1271 login 2023-06-24 09:48:31 -07:00
Bryan Stitt
fd661689a3
Tracing (#144)
* tracing

* add tracing and features to match

* use just one hostname crate

* cargo upgrade

* set up pretty logger and sentry layer
2023-06-23 16:28:45 -07:00
Bryan Stitt
3fac4248d6 Revert "just use Ulid instead of RpcSecretKey type"
It did work

This reverts commit 377e8356a57d80dd17b746dd05a62b475d9e7d07.
2023-06-23 01:29:11 -07:00
Bryan Stitt
377e8356a5 just use Ulid instead of RpcSecretKey type 2023-06-23 01:28:47 -07:00
Bryan Stitt
68ab9b6d1d always serialize RpcSecretKey as a ULID 2023-06-23 01:07:54 -07:00
Bryan Stitt
b1b8346db9
#123 role to rpc keys (#142)
* added roles to rpc keys

* dont unwrap

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-06-22 12:56:19 -07:00
Bryan Stitt
c79b787e13
allow setting referral code if not already set (#141) 2023-06-22 12:41:05 -07:00
Bryan Stitt
f593667169 fix lifetimes 2023-06-22 10:16:58 -07:00
Bryan Stitt
dffff65950 allow no referrer_record 2023-06-22 09:25:12 -07:00
Bryan Stitt
c560a59ef0 add more headers for debug endpoints and use more refs 2023-06-21 22:11:26 -07:00
Bryan Stitt
662bbb296f add header debug endpoint 2023-06-21 11:28:33 -07:00
Bryan Stitt
63c4f9e591 allow no ip
this shouldn't be needed, but is a workaround while we debug staging
2023-06-21 11:28:22 -07:00
Bryan Stitt
7f4da8d43c lint 2023-06-21 10:40:55 -07:00
Bryan Stitt
d962b749a2 use find_with_related, not find_also_related
find_also_related is 1:1, but we want 1:Many
2023-06-21 10:37:01 -07:00
Bryan Stitt
1cbb58e8e0 comment 2023-06-21 10:25:10 -07:00
Bryan Stitt
06ce95c386 let serde_json do more of the work 2023-06-21 10:18:36 -07:00
Bryan Stitt
5504062f47 dont error if no referrals 2023-06-21 10:18:06 -07:00
Bryan Stitt
3c2f95de10
Improve max wait and timeouts (#137)
* pass max wait with more functions

* move rpc_head_str higher up to use it in all the logs

* refresh consensus at double the block time

* new pricing that also includes archive check

* lint

* don't clone method

* put caches into one list

they serialize with their name, so it will still be easy to tell them apart

* more consistent waiting. still waits more than I want right now

* clean up default max waits

* more timeouts
2023-06-20 14:22:14 -07:00
Bryan Stitt
6d25c41faf
Compute units (#136)
* wip

* add basic compute unit calculator

* calculate costs with compute units
2023-06-19 17:47:38 -07:00
Bryan Stitt
6038351cb8 change default tracking in prep for premium-only keys 2023-06-19 13:00:57 -07:00
Bryan Stitt
b35cd58a76 rate limit user_balance_post by bearer OR ip 2023-06-19 10:21:48 -07:00
Bryan Stitt
89009dcd42 use web3_context for more errors 2023-06-18 16:40:17 -07:00
Bryan Stitt
af8543c3bf some defaults are needed maybe? 2023-06-18 16:36:23 -07:00
Bryan Stitt
2f78e64c5d create empty balance entry if none exists 2023-06-18 16:25:25 -07:00
Bryan Stitt
ba67151706 lint 2023-06-16 23:48:16 -07:00
Bryan Stitt
745870d119 drop a bunch of log levels 2023-06-16 23:46:20 -07:00
Bryan Stitt
86f9e7358a
move balance onto AuthorizationChecks (#131)
* move balance onto AuthorizationChecks

* todone

* bigger default

* flat cost for testing
2023-06-16 23:14:43 -07:00
Bryan Stitt
a083bc652d
Handle uncle transactions (#129)
* add more columns to handle uncled transactions

* handle payment uncles

* put relations back

* include all the new columns

* lower log levels

* improve block caching

if we have a block with a number, its canonical. uncles don't get returned

* improve disconnect logic

* lint

* clear first changed for new_top_config_receiver

* better logs around config changing

* i guess we do want one apply top_config at the start

* check correct variable for data limits
2023-06-16 00:46:27 -07:00
Bryan Stitt
6040ca297f fix eth_unsubscribe 2023-06-15 11:43:24 -07:00
Bryan Stitt
49c60ac1b5 improve websocket error handling 2023-06-15 09:50:21 -07:00
Bryan Stitt
45543be256 add more caches to /status 2023-06-12 23:03:38 -07:00
Bryan Stitt
18835e09da allow empty batches 2023-06-12 21:51:08 -07:00
Bryan Stitt
c771479d94 use WyRand instead of Xoshiro 2023-06-12 19:46:36 -07:00
Bryan Stitt
c3a7d1646c lint 2023-06-12 11:41:10 -07:00
Bryan Stitt
70d5c09d3e clean up redirect url 2023-06-12 11:38:34 -07:00
David
093ca19454
added subuser RPC keys into key access control, as well as fetching RPC keys (#115) 2023-06-12 08:59:23 -07:00
David
0cc4557e8d
Use rpc id instead of key (#118)
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.

* Revert "added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc."

This reverts commit 022a13cfd9ca24aad9e05f75489396936f14784a.

* mixed up branches
2023-06-11 19:14:02 -07:00
David
81aa4788b4
David/60 add basic email verification (#119)
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.

* added rpc key logic to use db numbers instead of UUIDs

* Revert "added rpc key logic to use db numbers instead of UUIDs"

This reverts commit 056771d02725f90ed5f1745b0372e13e630a19ac.
2023-06-11 19:10:10 -07:00
yenicelik
0b26ae3bad made modify-subuser post 2023-06-11 18:59:27 +02:00
Bryan Stitt
4a9c1a0ce6 move user registration into a dedicated function 2023-06-09 18:31:47 -07:00
Bryan Stitt
71d3d63524 put ip rate limiting back 2023-06-09 16:35:44 -07:00
Bryan Stitt
41950c886c improvements for web3-this-then-that 2023-06-09 16:35:14 -07:00
Bryan Stitt
bcf4f33543 add payment_factory_address to /status 2023-06-09 14:02:29 -07:00
Bryan Stitt
1f3040f6c7 automated tiers 2023-06-09 12:21:55 -07:00
Bryan Stitt
68f73ec0b1 more things should be BadRequest instead of 500 2023-06-08 13:42:45 -07:00
Bryan Stitt
74280bc329 lint 2023-06-08 11:33:16 -07:00
Bryan Stitt
334b1c21b3 improve logs 2023-06-08 11:30:48 -07:00
Bryan Stitt
1d05bc5e89 lower log levels 2023-06-08 11:26:09 -07:00
David
3f76b08364
Test balance and referral accounting (#111)
* removed bloom filter temporarily, added some fixes with decimals in payment.rs

* balance accounting for a single user (paid and free tier) seems to be ok

* fixed some balance accounting

* compute_cost reduced back to 0
2023-06-08 10:08:29 -07:00
Bryan Stitt
c37b320286 add cache names and actually serialize 2023-06-07 17:55:34 -07:00
Bryan Stitt
5c1569f9c2 clippy lint 2023-06-07 17:31:15 -07:00
Bryan Stitt
40bee30730 back to moka 2023-06-07 17:30:05 -07:00