Commit Graph

52 Commits

Author SHA1 Message Date
Bryan Stitt
b527f5d0d6
David/remove balance logic non destructive (#173)
* WIP will work on locks

* added concurrent test for referral, numbers rn still add up

* will change balance table to be a view instead of a table

* merged in devel. will proceed with migrations for rpc_stats_v2 and referral

* WIP

* WIP

* WIP compiling

* gotta add the balance_v2 view

* about to add balance view

* stupid missing closing param

* compiles again, now i will just have to implement the raw sql

* need to paste the real query and play with it

* ok now on to testing

* addresses many comments in PR

* app works for simple unauthorized access. will look into tests

* some tests pass, referral logic tests fail

* not sure why the test is failing

---------

Co-authored-by: yenicelik <david.yenicelik@gmail.com>
2023-07-09 19:23:32 -07:00
Bryan Stitt
eb7b98fdbe
Split errors (#158)
* add migration for splitting errors

* fix type from u32 to u64

* update entities to match migrations

* no migration needed. these are only in stats

* add user_error_response to influx

* only if detailed

* set error_response and user_error_response

* 0 cost error responses

* only 33 migrations now

* put macros back

* get the stat buffer sender to the TestApp helper

* fixes
2023-07-05 18:18:10 -07:00
Bryan Stitt
a64e32b085 remove bearer token semaphore. it isn't really necesary
we can come back to rate limiting authenticated endpoints later
2023-07-03 16:54:25 -07:00
Bryan Stitt
280e2075d3
Fix Issue 65 Pt. 2 (#153)
* change subuser balance logic and add message to access denied errors

* clearer import

* comments

* comments
2023-06-28 23:00:34 -07:00
Bryan Stitt
8815b5c8cd lint 2023-06-25 21:26:28 -07:00
yenicelik
627c359645 mixed up directions 2023-06-25 21:19:13 -04:00
David
0f4d12e2a5
Merge pull request #149 from yenicelik/devel
Devel
2023-06-25 17:55:55 -04:00
Bryan Stitt
0aa94d9c87 don't escape newlines 2023-06-25 14:34:19 -07:00
yenicelik
992860a46d forgot to allow aggregate global stats 2023-06-25 16:18:09 -04:00
yenicelik
4c157cfcf7 made paymentrequired error code instead of introducing a new one 2023-06-25 15:05:27 -04:00
yenicelik
935681fde7 stats subuser + premium access logic 2023-06-25 14:29:54 -04:00
Bryan Stitt
1d6e1ef9af polish error logs 2023-06-24 15:24:46 -07:00
Bryan Stitt
fef03f089f less verbose db_conn and db_replica helpers 2023-06-24 11:11:07 -07:00
Bryan Stitt
fd661689a3
Tracing (#144)
* tracing

* add tracing and features to match

* use just one hostname crate

* cargo upgrade

* set up pretty logger and sentry layer
2023-06-23 16:28:45 -07:00
Bryan Stitt
745870d119 drop a bunch of log levels 2023-06-16 23:46:20 -07:00
Bryan Stitt
74d2333039 try this version of influxdb2 2023-06-16 23:41:12 -07:00
David
d6513cde74
Forgot to handle Option unwrap (#132)
* made the query much faster

* aggregate queries should be fast now too

* forgot to handle an option unwrap
2023-06-16 23:18:51 -07:00
David
4772166e97
Make aggregate queries faster as well (#130)
* made the query much faster

* aggregate queries should be fast now too
2023-06-16 08:27:46 -07:00
David
5859cd8a8d
made the query much faster (#127) 2023-06-15 09:38:51 -07:00
yenicelik
2333b8bbb6 influx query is not 20-40% faster 2023-06-11 17:57:02 +02:00
David
d91713e09e
this brings back balance back to the stats request (#112)
* removed bloom filter temporarily, added some fixes with decimals in payment.rs

* balance is inside the influx query again

* fixed aggregate
2023-06-08 15:57:53 -07:00
Bryan Stitt
68f73ec0b1 more things should be BadRequest instead of 500 2023-06-08 13:42:45 -07:00
Bryan Stitt
74280bc329 lint 2023-06-08 11:33:16 -07:00
Bryan Stitt
334b1c21b3 improve logs 2023-06-08 11:30:48 -07:00
Bryan Stitt
1d05bc5e89 lower log levels 2023-06-08 11:26:09 -07:00
Bryan Stitt
d64761403b Cow instead of String 2023-05-30 23:17:05 -07:00
Bryan Stitt
fd8bba4bdd most of these are for the whole app, not just the frontend 2023-05-30 21:26:11 -07:00
Bryan Stitt
cebe8ed1ea
Request refactor (#99)
* wip

* AsRef finally works like i wanted

* actually return the block

* start adding async trait

* remove stale import

* include id in the error response when possible

* remove stale comments
2023-05-30 16:32:34 -07:00
yenicelik
b6cab88c4e removed joined_on variable that was not used 2023-05-24 13:35:10 +02:00
yenicelik
47f07261ed detailed request must come with a bearer token 2023-05-24 13:34:50 +02:00
yenicelik
59634139c9 updated error_response 2023-05-21 20:27:52 -04:00
yenicelik
29678b025e updated error_response 2023-05-21 20:27:46 -04:00
yenicelik
46afe8b1e4 influxdb query also groups by archive_needed now 2023-05-21 17:13:15 -04:00
yenicelik
795548580a now should also group by archive_needed (turns out this was wanted) 2023-05-21 16:40:42 -04:00
Bryan Stitt
24439c5143 disable balance join for now 2023-05-18 00:43:33 -07:00
Bryan Stitt
8a097dabbe
Bryan devel 2023-05-12 (#67)
* add minor todo

* BadRequest instead of web3_context

* more bad request error codes

* use tokio-uring for the tcp listener

* clear block instead of panic

* clone earlier

* more watch channels instead of rwlocks

* drop uring for now (its single threaded) and combine get/post/put routes

* clean up iter vs into_iter and unnecessary collect

* arcswap instead of rwlock for Web3Rpcs.by_name

* cargo upgrade

* uuid fast-rng and alphabetize

* if protected rpcs, only use protected rpcs

* listenfd

* make connectinfo optional

* try_get_with_by_ref instead of try_get_with

* anyhow ensure. and try_get_with_as_ref isn't actually needed

* fix feature flags

* more refs and less clone

* automatic retry for eth_getTransactionReceipt and eth_getTransactionByHash

thanks for the report Lefteris @ Rotki

* ArcSwap for provider

* set archive_request to true on transaction retrying

* merge durable stats

* Revert "ArcSwap for provider"

This reverts commit 166d77f204cde9fa7722c0cefecbb27008749d47.

* comments

* less clones

* more refs

* fix test

* add optional mimalloc feature

* remove stale dependency

* sort

* cargo upgrade

* lint constants

* add todo

* another todo

* lint

* anyhow::ensure instead of panic

* allow rpc_accounting_v2 entries for requests without an rpc key
2023-05-12 15:15:32 -07:00
David
34ed450fab
User Balance + Referral Logic (#44)
* will implement balance topup endpoint

* will quickly fix other PR reviews

* merging from master

* will finish up godmoe

* will finish up login

* added logic to top up balance (first iteration)

* should implement additional columns soon (currency, amount, tx-hash), as well as a new table for spend

* updated migrations, will account for spend next

* get back to this later

* will merge PR from stats-v2

* stats v2

rebased all my commits and squashed them down to one

* cargo upgrade

* added migrtation for spend in accounting table. will run test-deposit next

* trying to get request from polygon

* first iteration /user/balance/:tx_hash works, needs to add accepted tokens next

* creating the referral code seems to work

* will now check if spending enough credits will lead to both parties receiving credits

* rpcstats takes care of accounting for spend data

* removed track spend from table

* Revert "removed track spend from table"

This reverts commit a50802d6ae75f786864c5ec42d0ceb2cb27124ed.

* Revert "rpcstats takes care of accounting for spend data"

This reverts commit 1cec728bf241e4cfd24351134637ed81c1a5a10b.

* removed rpc request table entity

* updated referral code to use ulid s

* credits used are aggregated

* added a bunch of fields to referrer

* added database logic whenever an aggregate stats is added. will have to iterate over this a couple times i think. go to (1) detecting accepted stables next, (2) fix influxdb bug and (3) start to write test

* removed track spend as this will occur in the database

* will first work on "balance", then referral. these should really be treated as two separate PRs (although already convoluted)

* balance logic initial commit

* breaking WIP, changing the RPC call logic functions

* will start testing next

* got rid of warnings & lint

* will proceed with subtracting / adding to balance

* added decimal points, balance tracking seems to work

* will beautify code a bit

* removed deprecated dependency, and added topic + deposit contract to app.yaml

* brownie test suite does not rely on local contract files it pulls all from polygonscan

* will continue with referral

* should perhaps (in a future revision) recordhow much the referees got for free. marking referrals seems to work rn

* user is upgraded to premium if they deposit more than 10$. we dont accept more than $10M in a single tx

* will start PR, referral seems to be fine so far, perhaps up to some numbers that still may need tweaking

* will start PR

* removed rogue comments, cleaned up payments a bit

* changes before PR

* apply stats

* added unique constraint

* some refactoring such that the user file is not too bloated

* compiling

* progress with subusers, creating a table entry seems to work

* good response type is there as well now, will work on getters from primary user and secondary user next

* subuser logic also seems fine now

* downgrade logic

* fixed bug influxdb does not support different types in same query (which makes sense)

* WIP temporary commit

* merging with PR

* Delete daemon.rs

there are multiple daemons now, so this was moved to `proxyd`

* will remove request clone to &mut

* multiple request handles for payment

* making requests still seem fine

* removed redundant commented out bits

* added deposit endpoint, added deposit amount and deposit user, untested yet

* small bug with downgrade tier id

* will add authorization so balance can be received for users

* balance history should be set now too

* will check balance over time again

* subususer can see rpc key balance if admin or owner

* stats also seems to work fine now with historical balance

* things seem to be building and working

* removed clone from  OpenRequestHandle

* removed influxdb from workspace members

* changed config files

* reran sea-orm generate entities, added a foreign key, should be proper now

* removed contract from commit

* made deposit contract optional

* added topic in polygon dev

* changed deposit contract to deposit factory contract

* added selfrelation on user_tier

* added payment required

* changed chain id to u64

* add wss in polygon llamarpc

* removed origin and method from the table

* added onchain transactions naming (and forgot to add a migration before)

* changed foreign key to be the referrer (id), not the code itself

* forgot to add id as the target foreign key

* WIP adding cache to update role

* fixed merge conflicts

---------

Co-authored-by: Bryan Stitt <bryan@llamanodes.com>
Co-authored-by: Bryan Stitt <bryan@stitthappens.com>
2023-05-12 09:45:15 -07:00
Bryan Stitt
03c4990dc3 remove commented out code 2023-04-20 10:36:56 -07:00
Bryan Stitt
9929e59f04 lower log level 2023-04-20 09:17:18 -07:00
Bryan Stitt
767920c055 remove log based debug lines 2023-04-10 14:19:13 -07:00
Bryan Stitt
7994c894bb cargo clippy and fmt 2023-04-05 15:15:20 -07:00
Rory Neithinger
b012f18b2b Merge branch 'stats_v3' of github.com:yenicelik/web3-proxy into Web3ProxyError 2023-03-21 22:40:39 -07:00
yenicelik
0a6ccf28b5 will continue with migration 2023-03-20 12:22:30 +01:00
Rory Neithinger
f3fc4924dc Refactor FrontendErrorResponse into Web3ProxyError
Renamed FrontendResponse to Web3ProxyResponse and introduced
a new generic type alias Web3ProxyResult.

Fixed a few noisy cargo warnings.
2023-03-16 22:21:20 -07:00
yenicelik
ca41cb5c9e will push this for short review 2023-03-14 20:08:16 +01:00
yenicelik
65ca628cc7 will modify query so we can make queries more easily 2023-03-14 16:32:19 +01:00
yenicelik
9417961d78 for some reason values output is always zero 2023-03-12 16:09:20 +01:00
yenicelik
8c8c183e65 trying to debug returning multiple files 2023-03-10 20:26:15 +01:00
Bryan Stitt
c7dcc4aac3 actually return a json result from influx 2023-03-04 20:54:24 +01:00
Bryan Stitt
163bbbafca FromDataPoint needs Default 2023-03-04 20:54:24 +01:00