* working on test for referral
* refactored logic in tests to helper functions
* added getbalance helper
* added more helper functions to test for referral
* gotta be able to submit to rpc now
* adding provider instead
* referral tests are not passing, may be because we need for stats to be updated
* balance doesnt decrease
* will look into deposits in the meantime
* finding some first bugs
* seems to have fixed a bunch of stuff
* feedback, lint, and make it compile
---------
Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
* add migration for splitting errors
* fix type from u32 to u64
* update entities to match migrations
* no migration needed. these are only in stats
* add user_error_response to influx
* only if detailed
* set error_response and user_error_response
* 0 cost error responses
* only 33 migrations now
* put macros back
* get the stat buffer sender to the TestApp helper
* fixes
* pass max wait with more functions
* move rpc_head_str higher up to use it in all the logs
* refresh consensus at double the block time
* new pricing that also includes archive check
* lint
* don't clone method
* put caches into one list
they serialize with their name, so it will still be easy to tell them apart
* more consistent waiting. still waits more than I want right now
* clean up default max waits
* more timeouts
* add more columns to handle uncled transactions
* handle payment uncles
* put relations back
* include all the new columns
* lower log levels
* improve block caching
if we have a block with a number, its canonical. uncles don't get returned
* improve disconnect logic
* lint
* clear first changed for new_top_config_receiver
* better logs around config changing
* i guess we do want one apply top_config at the start
* check correct variable for data limits
* first pass at handling different return data limits
* put ws_provider in an arcswap
* add min max_latency
* add min max_latency
* subscribe with reconnect
* better logging around reconnect
* select on both watches
* subscribe to the correct watch
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.
* Revert "added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc."
This reverts commit 022a13cfd9ca24aad9e05f75489396936f14784a.
* mixed up branches
* added logic to check e-mails thru SMPT server, very basic, no validation e-mails etc.
* added rpc key logic to use db numbers instead of UUIDs
* Revert "added rpc key logic to use db numbers instead of UUIDs"
This reverts commit 056771d02725f90ed5f1745b0372e13e630a19ac.
* removed bloom filter temporarily, added some fixes with decimals in payment.rs
* balance accounting for a single user (paid and free tier) seems to be ok
* fixed some balance accounting
* compute_cost reduced back to 0
* tried to make the balance_entry atomic, asking for feedback
* added locked select everywhere and inside the txn, it should be atomic now
* forgot about overwrite (referee vs sender entities were the same, overwriting each other), fixed it
* will now trigger refreshing cache when balance thresholds are met
* will also invalidate cache if balance gets too low (to out of funds)
* added separate balance cache with shorter TTL. need to know if i should modify it (rn its not atomic), or if low TTL is good enough, and we just fetch the value every now and then from the Database
* removed UserTier struct bcs duplicate with database
* removed subuser premium requirement (which would have lead to a merge conflict later on)
* removed user_tier artefact
* replaced cache to use AtomicF64
* a bunch of small changes
* some changes
* will merge with devel
* changed AtomicF64 to RwLock<Decimal>
* changed AtomicF64 to RwLock<Decimal>
* downgrading user when at 0.1$ or if sum_credits_used is very low
* changed caches to be more aggressive in being non-empty
* replaced Arc::clone() by reference
* added stats page for referrals (shared, and received)
* removed referrals address, and return ok response if no referrals used
* merged from devel
* changed unwrap for context
* changes from PR
* tried to make the balance_entry atomic, asking for feedback
* added locked select everywhere and inside the txn, it should be atomic now
* forgot about overwrite (referee vs sender entities were the same, overwriting each other), fixed it
* will now trigger refreshing cache when balance thresholds are met
* Revert "will now trigger refreshing cache when balance thresholds are met"
This reverts commit b1215ddc43a1358f6086bddbd106996165431165.
* will not modify userTier in database anymore
* made payment atomic
* made updates mostly atomic, locking only the referral table (bcs more complex logic)
* latest changes
* will push it for feedback loop
* filtering also to only show rpc keys that the user is an owner of
* revert logs also returning rpc keys
* merged with devel
* wip
* AsRef finally works like i wanted
* actually return the block
* start adding async trait
* remove stale import
* include id in the error response when possible
* remove stale comments
* Revert "remove cache on /status and /health for now"
This reverts commit 166b0d810cd25b895560d50d00383a57e1269ad4.
* Revert "remove cache on backups needed, too"
This reverts commit 4597967def2f1daa71a277ec6731db4739ead640.
* add tests
* add max_item_weight
* comment
* add some helper functions, max weights, and tests
* now should also group by archive_needed (turns out this was wanted)
* influxdb query also groups by archive_needed now
* updated error_response
* updated error_response
* added admin balance increase, and also test
* slightly hacky, but does do an update expression
* added note column to entities
* removed redundant printlns
* flattened items
* removed user tier logic from admin increase balance completely. admin will have to manually increase the user tier
* quick cache and allocate less
* improve /status cache
* prepare to cache raw transaction hashes so we dont dos our backends
* simple benchmark for /health and /status
* mut not needed with atomics
* DRY all the status pages
* use u64 instead of bytes for subscriptions
* fix setting earliest_retry_at and improve logs
* Revert "use kanal instead of flume or tokio channels (#68)"
This reverts commit 510612d343fc51338a8a4282dcc229b50097835b.
* fix automatic retries
* put relaxed back
* convert error message time to seconds
* assert instead of debug_assert while we debug
* ns instead of seconds
* disable peak_latency for now
* null is the default
* cargo fmt
* comments
* remove request caching for now
* log on exit
* unit weigher for now
* make cache smaller. we need a weigher for prod. just debugging
* oops. we need async
* add todo
* no need for to_string on a RawValue
* add minor todo
* BadRequest instead of web3_context
* more bad request error codes
* use tokio-uring for the tcp listener
* clear block instead of panic
* clone earlier
* more watch channels instead of rwlocks
* drop uring for now (its single threaded) and combine get/post/put routes
* clean up iter vs into_iter and unnecessary collect
* arcswap instead of rwlock for Web3Rpcs.by_name
* cargo upgrade
* uuid fast-rng and alphabetize
* if protected rpcs, only use protected rpcs
* listenfd
* make connectinfo optional
* try_get_with_by_ref instead of try_get_with
* anyhow ensure. and try_get_with_as_ref isn't actually needed
* fix feature flags
* more refs and less clone
* automatic retry for eth_getTransactionReceipt and eth_getTransactionByHash
thanks for the report Lefteris @ Rotki
* ArcSwap for provider
* set archive_request to true on transaction retrying
* merge durable stats
* Revert "ArcSwap for provider"
This reverts commit 166d77f204cde9fa7722c0cefecbb27008749d47.
* comments
* less clones
* more refs
* fix test
* add optional mimalloc feature
* remove stale dependency
* sort
* cargo upgrade
* lint constants
* add todo
* another todo
* lint
* anyhow::ensure instead of panic
* allow rpc_accounting_v2 entries for requests without an rpc key
* will implement balance topup endpoint
* will quickly fix other PR reviews
* merging from master
* will finish up godmoe
* will finish up login
* added logic to top up balance (first iteration)
* should implement additional columns soon (currency, amount, tx-hash), as well as a new table for spend
* updated migrations, will account for spend next
* get back to this later
* will merge PR from stats-v2
* stats v2
rebased all my commits and squashed them down to one
* cargo upgrade
* added migrtation for spend in accounting table. will run test-deposit next
* trying to get request from polygon
* first iteration /user/balance/:tx_hash works, needs to add accepted tokens next
* creating the referral code seems to work
* will now check if spending enough credits will lead to both parties receiving credits
* rpcstats takes care of accounting for spend data
* removed track spend from table
* Revert "removed track spend from table"
This reverts commit a50802d6ae75f786864c5ec42d0ceb2cb27124ed.
* Revert "rpcstats takes care of accounting for spend data"
This reverts commit 1cec728bf241e4cfd24351134637ed81c1a5a10b.
* removed rpc request table entity
* updated referral code to use ulid s
* credits used are aggregated
* added a bunch of fields to referrer
* added database logic whenever an aggregate stats is added. will have to iterate over this a couple times i think. go to (1) detecting accepted stables next, (2) fix influxdb bug and (3) start to write test
* removed track spend as this will occur in the database
* will first work on "balance", then referral. these should really be treated as two separate PRs (although already convoluted)
* balance logic initial commit
* breaking WIP, changing the RPC call logic functions
* will start testing next
* got rid of warnings & lint
* will proceed with subtracting / adding to balance
* added decimal points, balance tracking seems to work
* will beautify code a bit
* removed deprecated dependency, and added topic + deposit contract to app.yaml
* brownie test suite does not rely on local contract files it pulls all from polygonscan
* will continue with referral
* should perhaps (in a future revision) recordhow much the referees got for free. marking referrals seems to work rn
* user is upgraded to premium if they deposit more than 10$. we dont accept more than $10M in a single tx
* will start PR, referral seems to be fine so far, perhaps up to some numbers that still may need tweaking
* will start PR
* removed rogue comments, cleaned up payments a bit
* changes before PR
* apply stats
* added unique constraint
* some refactoring such that the user file is not too bloated
* compiling
* progress with subusers, creating a table entry seems to work
* good response type is there as well now, will work on getters from primary user and secondary user next
* subuser logic also seems fine now
* downgrade logic
* fixed bug influxdb does not support different types in same query (which makes sense)
* WIP temporary commit
* merging with PR
* Delete daemon.rs
there are multiple daemons now, so this was moved to `proxyd`
* will remove request clone to &mut
* multiple request handles for payment
* making requests still seem fine
* removed redundant commented out bits
* added deposit endpoint, added deposit amount and deposit user, untested yet
* small bug with downgrade tier id
* will add authorization so balance can be received for users
* balance history should be set now too
* will check balance over time again
* subususer can see rpc key balance if admin or owner
* stats also seems to work fine now with historical balance
* things seem to be building and working
* removed clone from OpenRequestHandle
* removed influxdb from workspace members
* changed config files
* reran sea-orm generate entities, added a foreign key, should be proper now
* removed contract from commit
* made deposit contract optional
* added topic in polygon dev
* changed deposit contract to deposit factory contract
* added selfrelation on user_tier
* added payment required
* changed chain id to u64
* add wss in polygon llamarpc
* removed origin and method from the table
* added onchain transactions naming (and forgot to add a migration before)
* changed foreign key to be the referrer (id), not the code itself
* forgot to add id as the target foreign key
* WIP adding cache to update role
* fixed merge conflicts
---------
Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
Co-authored-by: Bryan Stitt <bryan@stitthappens.com>
* use peak-ewma instead of head for latency calculation
* Implement some suggested changes from PR
* move latency to new package in workspace root
* fix unit tests which now require peak_latency on Web3Rpc
* Switch to atomics for peak-ewma
This change is to avoid locking from tokio::sync::watch.
* add decay calculation to latency reads in peak-ewma
* Add some tests for peak-ewma
* Sensible latency defaults and not blocking on full
* Cleanup and a couple additional comments
* move protected transactions into their own function and dry stats sending
* cargo upgrade
* comments
* time to live instead of time to idle
* minor workaround for eth_chainId
* cargo upgrade
