Commit Graph

60 Commits

Author SHA1 Message Date
Bryan Stitt
05e618f74c
Anon rpc accounting (#194)
* cargo generate migrations didnt work without this

* anon rpc accounting

* Revert "anon rpc accounting"

This reverts commit f1b8cbcdeca0d5f9545da32f11dda6a90854f97a.

* 0 for anon in mysql

* only set approximate_balance_remaining on owned stats

* only set self.approximate_balance_remaining when adding if some

* todo complete
2023-07-24 22:02:33 -07:00
Bryan Stitt
ba1ce91322 remove assert that doesnt work well with interval flushes 2023-07-22 02:41:27 -07:00
Bryan Stitt
22400e9c8b needs a string, not a ulid 2023-07-22 02:32:54 -07:00
Bryan Stitt
4dfe092e08 fix tests 2023-07-22 02:30:39 -07:00
Bryan Stitt
12f3f940a0 include the window in the tsdb logs 2023-07-22 01:56:13 -07:00
Bryan Stitt
80d50bc2c0 need unique names in tests 2023-07-22 01:38:54 -07:00
Bryan Stitt
4091e05f3d influx id from config. default to the hostname or fallback to ulid 2023-07-22 00:26:02 -07:00
Bryan Stitt
7c4de5ea8b give the databases longer to start 2023-07-21 23:34:54 -07:00
Bryan Stitt
0ef8bb50e8 drop x at the end of most tests 2023-07-21 16:18:33 -07:00
Bryan Stitt
620f2d9d00 30 second flushes for db too 2023-07-21 16:12:43 -07:00
Bryan Stitt
95bf1db01e save tsdb less often 2023-07-21 15:25:12 -07:00
Bryan Stitt
f15f1027cf stat buffer count total requests and print on exit 2023-07-21 15:25:02 -07:00
yenicelik
bc6d18041d added single proxy test 2023-07-21 18:02:07 -04:00
Bryan Stitt
5b9eb080a2 include influx in test_sum_credits_used 2023-07-20 22:21:46 -07:00
Bryan Stitt
f5036f6cd9 set tiers to fix tests 2023-07-20 22:07:46 -07:00
Bryan Stitt
03bfd7148f test_two_buffers needs docker 2023-07-20 20:05:22 -07:00
Bryan Stitt
2611085c00 add simple stat buffer test 2023-07-20 18:22:25 -07:00
Bryan Stitt
7678cb7389 caches aren't actualy optional on buffer 2023-07-20 18:21:49 -07:00
Bryan Stitt
ac2d9049be improve flushing and buffering of stats 2023-07-19 22:55:14 -07:00
Bryan Stitt
90ffb5254a flush all stats 2023-07-19 18:41:40 -07:00
Bryan Stitt
a97cb6a0c5 lint 2023-07-18 09:50:51 -07:00
Bryan Stitt
600c1bafb4
David/multiple proxy test (#187)
* add test_multiple_proxies_stats_add_up

* make a premium user using both proxies

* added a couple clones, must add constraints now to run for multiple-proxies (check arithmetic)

* lint and code review

* fix comment

* fix tests (now fails at todo)

* will introduce endpoint to fetch rpc stats from mysql

* added influxdb to tests, should next do asserst in stats collected by influx, and mysql for multi-proxy

* created test where influx and mysql have separate data, should help with debugging

* forgot to drop influx

* tests pass except multi-proxy

* test passes, will check out nothing broke elswhere

* go back to numbers

* some linting

* linting

* removed redundant info!

* responding to PR comments

* ULID as instance-hash for the tag in influx (for anti-dup)

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-07-17 13:31:41 -07:00
Bryan Stitt
efa189d101 less anyhow and fix tests 2023-07-14 18:31:00 -07:00
Bryan Stitt
0046a02a4a
Global db (#191)
* dont migrate on start. just connect

* it compiles

* that log is loud

* get transaction from a local clone

* import the trait

* put the test app into a thread instead of tokio::spawn

* fix one test

* try db before rpcs

* db connection is too slow. need to wait for it

* do db setup once while spawning
2023-07-14 18:30:01 -07:00
Bryan Stitt
45af241429 less db transactions 2023-07-13 15:44:07 -07:00
Bryan Stitt
5d5e65ed40 close the db connection on exit 2023-07-13 15:44:07 -07:00
Bryan Stitt
dce3540ee3 less authorization and remove pending txs for now 2023-07-13 11:01:16 -07:00
Bryan Stitt
2b1829a882 move db and anvil functions to their proper places 2023-07-12 14:42:59 -07:00
Bryan Stitt
6b5437be2c split mysql and anvil into their own fixtures 2023-07-12 14:28:31 -07:00
Bryan Stitt
5d207fb2c6
Auto tier change (#184)
* lint

* change user to premium on admin credits

todo: change on other deposits

* set tier more places

* BadRequest instead of 500

* insert existing users too

* add the premium file
2023-07-12 10:24:16 -07:00
Bryan Stitt
b6cbf02ae7 test_sum_credits_used needs docker 2023-07-12 00:47:55 -07:00
Bryan Stitt
df2f3d340f
More balance tests (#182)
* fix popularity contest

* more info in the Debug for Web3Rpc

* add frontend_requests and cache_misses to the Balance query

* add more to balance and stats flushing and improved test coverage

* it compiles

* deserializer for Ulid to Uuid

I think a wrapper type on Ulid that implements sea_orm::Value is probably better

* rename variable to match struct name

* add deserializer for Address -> Vec<u8>

* sql sum returns a Decimal. need to convert to u64

* assert more

* one log and assert more

* log more

* use a helper to get the user's rpc provider

* this should be 2 now that we have a public and authed call

* this should be zero. the public has the cache miss

* instrument cu calcs

* trace the value we took, not the default that replaced it

* move usd_per_chain into config

* remove some extra logging

* use Arc::into_inner to maybe avoid a race

* off by 1

* pass paid credits used instead of returning it

this lets us use it to write to our user balance cache first. importantly, this keeps us from holding a write lock while writing to mysql

* no cache misses expected in this test

* actually check the admin

* put the balance checks back now that the rest of the test works

* archive request is being set incorrectly

* wow howd we manage flipping the greater than sign on archive depth

* move latest_balance and premium_credits_used to before any stats are emitted

* lint

* and build undoes the linting. fun

i didnt even want to lint them in the first place, so this is fine

* missed incrementing total_spent when not incrementing total_spent_paid_credits

* use the credits on self

* use the credits on self (pt 2)

* fix type for 10 cu query

* convert the requestmetadata on the other side of the channel

* logs

* viewing stats is allowed even without a balance

* move paid_credits_used to AuthorizationChecks

* wip

* test_sum_credits_used finally passes

* UserBalanceCache::get_or_insert

* re-enable rpc_secret_key_cache

* move invalidate to a helper function

and always call it **after** the db is commited

* fix PartialEq and Eq on RpcSecretKey

* cargo upgrade
2023-07-12 00:35:07 -07:00
Bryan Stitt
0f7e370c92 drop flume. just use tokio
i dont know if this is the problem, but this will also get compile times down
2023-07-10 23:08:06 -07:00
Bryan Stitt
3059d08674 error on max_head_block_age and keep the sender alive longer 2023-07-10 22:17:15 -07:00
Bryan Stitt
a560d47c48 lint 2023-07-09 20:13:03 -07:00
David
000a1326c3
New Balance Logic (#174)
* WIP will work on locks

* added concurrent test for referral, numbers rn still add up

* will change balance table to be a view instead of a table

* merged in devel. will proceed with migrations for rpc_stats_v2 and referral

* WIP

* WIP

* WIP compiling

* gotta add the balance_v2 view

* about to add balance view

* stupid missing closing param

* compiles again, now i will just have to implement the raw sql

* need to paste the real query and play with it

* ok now on to testing

* addresses many comments in PR

* app works for simple unauthorized access. will look into tests

* some tests pass, referral logic tests fail

* not sure why the test is failing

* invalidate cache for admin, will check for more possible targets

* user balance decrease test works

* all tests pass
2023-07-09 19:43:44 -07:00
Bryan Stitt
b527f5d0d6
David/remove balance logic non destructive (#173)
* WIP will work on locks

* added concurrent test for referral, numbers rn still add up

* will change balance table to be a view instead of a table

* merged in devel. will proceed with migrations for rpc_stats_v2 and referral

* WIP

* WIP

* WIP compiling

* gotta add the balance_v2 view

* about to add balance view

* stupid missing closing param

* compiles again, now i will just have to implement the raw sql

* need to paste the real query and play with it

* ok now on to testing

* addresses many comments in PR

* app works for simple unauthorized access. will look into tests

* some tests pass, referral logic tests fail

* not sure why the test is failing

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-07-09 19:23:32 -07:00
Bryan Stitt
651244fbca lower log levels 2023-07-07 14:14:03 -07:00
Bryan Stitt
512bcf305b lint 2023-07-07 14:06:35 -07:00
yenicelik
af45f78ff5 added test for admin deposit endpoint, fixed bugs 2023-07-06 08:19:33 -04:00
Bryan Stitt
e0c57a22ae derivative and serde didn't work together like we'd hoped 2023-07-06 03:51:39 -07:00
David
8d31680ff1
David/referral test new (#162)
* working on test for referral

* refactored logic in tests to helper functions

* added getbalance helper

* added more helper functions to test for referral

* gotta be able to submit to rpc now

* adding provider instead

* referral tests are not passing, may be because we need for stats to be updated

* balance doesnt decrease

* will look into deposits in the meantime

* finding some first bugs

* seems to have fixed a bunch of stuff

* feedback, lint, and make it compile

---------

Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
2023-07-05 22:18:39 -07:00
Bryan Stitt
3bbbdd5596
Flush stats (#161)
* create buffer if mysql OR influx is set up

* this one flush should error

* it helps to set up the db in the db migration test

* comment
2023-07-05 19:24:21 -07:00
Bryan Stitt
eb7b98fdbe
Split errors (#158)
* add migration for splitting errors

* fix type from u32 to u64

* update entities to match migrations

* no migration needed. these are only in stats

* add user_error_response to influx

* only if detailed

* set error_response and user_error_response

* 0 cost error responses

* only 33 migrations now

* put macros back

* get the stat buffer sender to the TestApp helper

* fixes
2023-07-05 18:18:10 -07:00
Bryan Stitt
a64e32b085 remove bearer token semaphore. it isn't really necesary
we can come back to rate limiting authenticated endpoints later
2023-07-03 16:54:25 -07:00
Bryan Stitt
865c6d498e db helper and fix test never timing out 2023-07-03 15:54:22 -07:00
Bryan Stitt
d613969c3e Merge remote-tracking branch 'origin/devel' into david/test-admin 2023-07-03 14:07:43 -07:00
yenicelik
6cf7e4d621 problem executing the last bit 2023-07-01 15:38:56 -04:00
yenicelik
458f7d6514 could call subcommand 2023-07-01 15:26:42 -04:00
Bryan Stitt
7044edc63c check multiple posible stripe headers 2023-07-01 09:10:53 -07:00