* add test_multiple_proxies_stats_add_up
* make a premium user using both proxies
* added a couple clones, must add constraints now to run for multiple-proxies (check arithmetic)
* lint and code review
* fix comment
* fix tests (now fails at todo)
* will introduce endpoint to fetch rpc stats from mysql
* added influxdb to tests, should next do asserst in stats collected by influx, and mysql for multi-proxy
* created test where influx and mysql have separate data, should help with debugging
* forgot to drop influx
* tests pass except multi-proxy
* test passes, will check out nothing broke elswhere
* go back to numbers
* some linting
* linting
* removed redundant info!
* responding to PR comments
* ULID as instance-hash for the tag in influx (for anti-dup)
---------
Co-authored-by: yenicelik <david.yenicelik@gmail.com>
* dont migrate on start. just connect
* it compiles
* that log is loud
* get transaction from a local clone
* import the trait
* put the test app into a thread instead of tokio::spawn
* fix one test
* try db before rpcs
* db connection is too slow. need to wait for it
* do db setup once while spawning
* lint
* change user to premium on admin credits
todo: change on other deposits
* set tier more places
* BadRequest instead of 500
* insert existing users too
* add the premium file
* fix popularity contest
* more info in the Debug for Web3Rpc
* add frontend_requests and cache_misses to the Balance query
* add more to balance and stats flushing and improved test coverage
* it compiles
* deserializer for Ulid to Uuid
I think a wrapper type on Ulid that implements sea_orm::Value is probably better
* rename variable to match struct name
* add deserializer for Address -> Vec<u8>
* sql sum returns a Decimal. need to convert to u64
* assert more
* one log and assert more
* log more
* use a helper to get the user's rpc provider
* this should be 2 now that we have a public and authed call
* this should be zero. the public has the cache miss
* instrument cu calcs
* trace the value we took, not the default that replaced it
* move usd_per_chain into config
* remove some extra logging
* use Arc::into_inner to maybe avoid a race
* off by 1
* pass paid credits used instead of returning it
this lets us use it to write to our user balance cache first. importantly, this keeps us from holding a write lock while writing to mysql
* no cache misses expected in this test
* actually check the admin
* put the balance checks back now that the rest of the test works
* archive request is being set incorrectly
* wow howd we manage flipping the greater than sign on archive depth
* move latest_balance and premium_credits_used to before any stats are emitted
* lint
* and build undoes the linting. fun
i didnt even want to lint them in the first place, so this is fine
* missed incrementing total_spent when not incrementing total_spent_paid_credits
* use the credits on self
* use the credits on self (pt 2)
* fix type for 10 cu query
* convert the requestmetadata on the other side of the channel
* logs
* viewing stats is allowed even without a balance
* move paid_credits_used to AuthorizationChecks
* wip
* test_sum_credits_used finally passes
* UserBalanceCache::get_or_insert
* re-enable rpc_secret_key_cache
* move invalidate to a helper function
and always call it **after** the db is commited
* fix PartialEq and Eq on RpcSecretKey
* cargo upgrade
* WIP will work on locks
* added concurrent test for referral, numbers rn still add up
* will change balance table to be a view instead of a table
* merged in devel. will proceed with migrations for rpc_stats_v2 and referral
* WIP
* WIP
* WIP compiling
* gotta add the balance_v2 view
* about to add balance view
* stupid missing closing param
* compiles again, now i will just have to implement the raw sql
* need to paste the real query and play with it
* ok now on to testing
* addresses many comments in PR
* app works for simple unauthorized access. will look into tests
* some tests pass, referral logic tests fail
* not sure why the test is failing
* invalidate cache for admin, will check for more possible targets
* user balance decrease test works
* all tests pass
* changed to active model
---------
Co-authored-by: yenicelik <david.yenicelik@gmail.com>
* WIP will work on locks
* added concurrent test for referral, numbers rn still add up
* will change balance table to be a view instead of a table
* merged in devel. will proceed with migrations for rpc_stats_v2 and referral
* WIP
* WIP
* WIP compiling
* gotta add the balance_v2 view
* about to add balance view
* stupid missing closing param
* compiles again, now i will just have to implement the raw sql
* need to paste the real query and play with it
* ok now on to testing
* addresses many comments in PR
* app works for simple unauthorized access. will look into tests
* some tests pass, referral logic tests fail
* not sure why the test is failing
* invalidate cache for admin, will check for more possible targets
* user balance decrease test works
* all tests pass
* WIP will work on locks
* added concurrent test for referral, numbers rn still add up
* will change balance table to be a view instead of a table
* merged in devel. will proceed with migrations for rpc_stats_v2 and referral
* WIP
* WIP
* WIP compiling
* gotta add the balance_v2 view
* about to add balance view
* stupid missing closing param
* compiles again, now i will just have to implement the raw sql
* need to paste the real query and play with it
* ok now on to testing
* addresses many comments in PR
* app works for simple unauthorized access. will look into tests
* some tests pass, referral logic tests fail
* not sure why the test is failing
---------
Co-authored-by: yenicelik <david.yenicelik@gmail.com>
* lower premium concurrency in preparation for key+IP limits
* include the ip in the user semaphore
* 3, not 5
this is our current limit for free
* per user_id+ip rate limiting
* working on test for referral
* refactored logic in tests to helper functions
* added getbalance helper
* added more helper functions to test for referral
* gotta be able to submit to rpc now
* adding provider instead
* referral tests are not passing, may be because we need for stats to be updated
* balance doesnt decrease
* will look into deposits in the meantime
* finding some first bugs
* seems to have fixed a bunch of stuff
* feedback, lint, and make it compile
---------
Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
* add migration for splitting errors
* fix type from u32 to u64
* update entities to match migrations
* no migration needed. these are only in stats
* add user_error_response to influx
* only if detailed
* set error_response and user_error_response
* 0 cost error responses
* only 33 migrations now
* put macros back
* get the stat buffer sender to the TestApp helper
* fixes
* pass max wait with more functions
* move rpc_head_str higher up to use it in all the logs
* refresh consensus at double the block time
* new pricing that also includes archive check
* lint
* don't clone method
* put caches into one list
they serialize with their name, so it will still be easy to tell them apart
* more consistent waiting. still waits more than I want right now
* clean up default max waits
* more timeouts
* add more columns to handle uncled transactions
* handle payment uncles
* put relations back
* include all the new columns
* lower log levels
* improve block caching
if we have a block with a number, its canonical. uncles don't get returned
* improve disconnect logic
* lint
* clear first changed for new_top_config_receiver
* better logs around config changing
* i guess we do want one apply top_config at the start
* check correct variable for data limits
* first pass at handling different return data limits
* put ws_provider in an arcswap
* add min max_latency
* add min max_latency
* subscribe with reconnect
* better logging around reconnect
* select on both watches
* subscribe to the correct watch
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.
* Revert "added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc."
This reverts commit 022a13cfd9ca24aad9e05f75489396936f14784a.
* mixed up branches
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.
* added rpc key logic to use db numbers instead of UUIDs
* Revert "added rpc key logic to use db numbers instead of UUIDs"
This reverts commit 056771d02725f90ed5f1745b0372e13e630a19ac.
* removed bloom filter temporarily, added some fixes with decimals in payment.rs
* balance accounting for a single user (paid and free tier) seems to be ok
* fixed some balance accounting
* compute_cost reduced back to 0
* tried to make the balance_entry atomic, asking for feedback
* added locked select everywhere and inside the txn, it should be atomic now
* forgot about overwrite (referee vs sender entities were the same, overwriting each other), fixed it
* will now trigger refreshing cache when balance thresholds are met
* will also invalidate cache if balance gets too low (to out of funds)
* added separate balance cache with shorter TTL. need to know if i should modify it (rn its not atomic), or if low TTL is good enough, and we just fetch the value every now and then from the Database
* removed UserTier struct bcs duplicate with database
* removed subuser premium requirement (which would have lead to a merge conflict later on)
* removed user_tier artefact
* replaced cache to use AtomicF64
* a bunch of small changes
* some changes
* will merge with devel
* changed AtomicF64 to RwLock<Decimal>
* changed AtomicF64 to RwLock<Decimal>
* downgrading user when at 0.1$ or if sum_credits_used is very low
* changed caches to be more aggressive in being non-empty
* replaced Arc::clone() by reference
* added stats page for referrals (shared, and received)
* removed referrals address, and return ok response if no referrals used
* merged from devel
* changed unwrap for context
* changes from PR
* tried to make the balance_entry atomic, asking for feedback
* added locked select everywhere and inside the txn, it should be atomic now
* forgot about overwrite (referee vs sender entities were the same, overwriting each other), fixed it
* will now trigger refreshing cache when balance thresholds are met
* Revert "will now trigger refreshing cache when balance thresholds are met"
This reverts commit b1215ddc43a1358f6086bddbd106996165431165.
* will not modify userTier in database anymore
* made payment atomic
* made updates mostly atomic, locking only the referral table (bcs more complex logic)
* latest changes
