web3-proxy/migration/src/m20230125_204810_stats_v2.rs

172 lines
6.2 KiB
Rust
Raw Normal View History

use sea_orm_migration::prelude::*;
#[derive(DeriveMigrationName)]
pub struct Migration;
#[async_trait::async_trait]
impl MigrationTrait for Migration {
async fn up(&self, manager: &SchemaManager) -> Result<(), DbErr> {
manager
.create_table(
Table::create()
.table(RpcAccountingV2::Table)
.col(
ColumnDef::new(RpcAccountingV2::Id)
.big_unsigned()
.not_null()
.auto_increment()
.primary_key(),
)
.col(
ColumnDef::new(RpcAccountingV2::RpcKeyId)
.big_unsigned()
2023-04-06 00:36:51 +03:00
.not_null()
.default(0),
)
User Balance + Referral Logic (#44) * will implement balance topup endpoint * will quickly fix other PR reviews * merging from master * will finish up godmoe * will finish up login * added logic to top up balance (first iteration) * should implement additional columns soon (currency, amount, tx-hash), as well as a new table for spend * updated migrations, will account for spend next * get back to this later * will merge PR from stats-v2 * stats v2 rebased all my commits and squashed them down to one * cargo upgrade * added migrtation for spend in accounting table. will run test-deposit next * trying to get request from polygon * first iteration /user/balance/:tx_hash works, needs to add accepted tokens next * creating the referral code seems to work * will now check if spending enough credits will lead to both parties receiving credits * rpcstats takes care of accounting for spend data * removed track spend from table * Revert "removed track spend from table" This reverts commit a50802d6ae75f786864c5ec42d0ceb2cb27124ed. * Revert "rpcstats takes care of accounting for spend data" This reverts commit 1cec728bf241e4cfd24351134637ed81c1a5a10b. * removed rpc request table entity * updated referral code to use ulid s * credits used are aggregated * added a bunch of fields to referrer * added database logic whenever an aggregate stats is added. will have to iterate over this a couple times i think. go to (1) detecting accepted stables next, (2) fix influxdb bug and (3) start to write test * removed track spend as this will occur in the database * will first work on "balance", then referral. these should really be treated as two separate PRs (although already convoluted) * balance logic initial commit * breaking WIP, changing the RPC call logic functions * will start testing next * got rid of warnings & lint * will proceed with subtracting / adding to balance * added decimal points, balance tracking seems to work * will beautify code a bit * removed deprecated dependency, and added topic + deposit contract to app.yaml * brownie test suite does not rely on local contract files it pulls all from polygonscan * will continue with referral * should perhaps (in a future revision) recordhow much the referees got for free. marking referrals seems to work rn * user is upgraded to premium if they deposit more than 10$. we dont accept more than $10M in a single tx * will start PR, referral seems to be fine so far, perhaps up to some numbers that still may need tweaking * will start PR * removed rogue comments, cleaned up payments a bit * changes before PR * apply stats * added unique constraint * some refactoring such that the user file is not too bloated * compiling * progress with subusers, creating a table entry seems to work * good response type is there as well now, will work on getters from primary user and secondary user next * subuser logic also seems fine now * downgrade logic * fixed bug influxdb does not support different types in same query (which makes sense) * WIP temporary commit * merging with PR * Delete daemon.rs there are multiple daemons now, so this was moved to `proxyd` * will remove request clone to &mut * multiple request handles for payment * making requests still seem fine * removed redundant commented out bits * added deposit endpoint, added deposit amount and deposit user, untested yet * small bug with downgrade tier id * will add authorization so balance can be received for users * balance history should be set now too * will check balance over time again * subususer can see rpc key balance if admin or owner * stats also seems to work fine now with historical balance * things seem to be building and working * removed clone from OpenRequestHandle * removed influxdb from workspace members * changed config files * reran sea-orm generate entities, added a foreign key, should be proper now * removed contract from commit * made deposit contract optional * added topic in polygon dev * changed deposit contract to deposit factory contract * added selfrelation on user_tier * added payment required * changed chain id to u64 * add wss in polygon llamarpc * removed origin and method from the table * added onchain transactions naming (and forgot to add a migration before) * changed foreign key to be the referrer (id), not the code itself * forgot to add id as the target foreign key * WIP adding cache to update role * fixed merge conflicts --------- Co-authored-by: Bryan Stitt <bryan@llamanodes.com> Co-authored-by: Bryan Stitt <bryan@stitthappens.com>
2023-05-12 19:45:15 +03:00
.foreign_key(
ForeignKeyCreateStatement::new()
.from_col(RpcAccountingV2::RpcKeyId)
.to_tbl(RpcKey::Table)
.to_col(RpcKey::Id),
)
.col(
ColumnDef::new(RpcAccountingV2::ChainId)
.big_unsigned()
.not_null(),
)
2023-04-06 00:36:51 +03:00
.col(
ColumnDef::new(RpcAccountingV2::Origin)
.string()
.not_null()
.default(""),
)
.col(
ColumnDef::new(RpcAccountingV2::PeriodDatetime)
.timestamp()
.not_null(),
)
2023-04-06 00:36:51 +03:00
.col(
ColumnDef::new(RpcAccountingV2::Method)
.string()
.not_null()
.default(""),
)
.col(
ColumnDef::new(RpcAccountingV2::ArchiveNeeded)
.boolean()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::ErrorResponse)
.boolean()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::FrontendRequests)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::BackendRequests)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::BackendRetries)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::NoServers)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::CacheMisses)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::CacheHits)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::SumRequestBytes)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::SumResponseMillis)
.big_unsigned()
.not_null(),
)
.col(
ColumnDef::new(RpcAccountingV2::SumResponseBytes)
.big_unsigned()
.not_null(),
)
// cannot use NULL columns for any of these because unique indexes allow duplicates on NULL
.index(
sea_query::Index::create()
.col(RpcAccountingV2::RpcKeyId)
.col(RpcAccountingV2::ChainId)
.col(RpcAccountingV2::Origin)
.col(RpcAccountingV2::PeriodDatetime)
.col(RpcAccountingV2::Method)
.col(RpcAccountingV2::ArchiveNeeded)
.col(RpcAccountingV2::ErrorResponse)
.unique(),
)
// cannot use a foreign key for RpcKeyId because the UNIQUE index uses 0 instead of NULL
.index(sea_query::Index::create().col(RpcAccountingV2::RpcKeyId))
.index(sea_query::Index::create().col(RpcAccountingV2::ChainId))
.index(sea_query::Index::create().col(RpcAccountingV2::Origin))
.index(sea_query::Index::create().col(RpcAccountingV2::PeriodDatetime))
.index(sea_query::Index::create().col(RpcAccountingV2::Method))
.index(sea_query::Index::create().col(RpcAccountingV2::ArchiveNeeded))
.index(sea_query::Index::create().col(RpcAccountingV2::ErrorResponse))
.to_owned(),
)
.await?;
Ok(())
}
async fn down(&self, manager: &SchemaManager) -> Result<(), DbErr> {
manager
.drop_table(Table::drop().table(RpcAccountingV2::Table).to_owned())
.await?;
Ok(())
}
}
User Balance + Referral Logic (#44) * will implement balance topup endpoint * will quickly fix other PR reviews * merging from master * will finish up godmoe * will finish up login * added logic to top up balance (first iteration) * should implement additional columns soon (currency, amount, tx-hash), as well as a new table for spend * updated migrations, will account for spend next * get back to this later * will merge PR from stats-v2 * stats v2 rebased all my commits and squashed them down to one * cargo upgrade * added migrtation for spend in accounting table. will run test-deposit next * trying to get request from polygon * first iteration /user/balance/:tx_hash works, needs to add accepted tokens next * creating the referral code seems to work * will now check if spending enough credits will lead to both parties receiving credits * rpcstats takes care of accounting for spend data * removed track spend from table * Revert "removed track spend from table" This reverts commit a50802d6ae75f786864c5ec42d0ceb2cb27124ed. * Revert "rpcstats takes care of accounting for spend data" This reverts commit 1cec728bf241e4cfd24351134637ed81c1a5a10b. * removed rpc request table entity * updated referral code to use ulid s * credits used are aggregated * added a bunch of fields to referrer * added database logic whenever an aggregate stats is added. will have to iterate over this a couple times i think. go to (1) detecting accepted stables next, (2) fix influxdb bug and (3) start to write test * removed track spend as this will occur in the database * will first work on "balance", then referral. these should really be treated as two separate PRs (although already convoluted) * balance logic initial commit * breaking WIP, changing the RPC call logic functions * will start testing next * got rid of warnings & lint * will proceed with subtracting / adding to balance * added decimal points, balance tracking seems to work * will beautify code a bit * removed deprecated dependency, and added topic + deposit contract to app.yaml * brownie test suite does not rely on local contract files it pulls all from polygonscan * will continue with referral * should perhaps (in a future revision) recordhow much the referees got for free. marking referrals seems to work rn * user is upgraded to premium if they deposit more than 10$. we dont accept more than $10M in a single tx * will start PR, referral seems to be fine so far, perhaps up to some numbers that still may need tweaking * will start PR * removed rogue comments, cleaned up payments a bit * changes before PR * apply stats * added unique constraint * some refactoring such that the user file is not too bloated * compiling * progress with subusers, creating a table entry seems to work * good response type is there as well now, will work on getters from primary user and secondary user next * subuser logic also seems fine now * downgrade logic * fixed bug influxdb does not support different types in same query (which makes sense) * WIP temporary commit * merging with PR * Delete daemon.rs there are multiple daemons now, so this was moved to `proxyd` * will remove request clone to &mut * multiple request handles for payment * making requests still seem fine * removed redundant commented out bits * added deposit endpoint, added deposit amount and deposit user, untested yet * small bug with downgrade tier id * will add authorization so balance can be received for users * balance history should be set now too * will check balance over time again * subususer can see rpc key balance if admin or owner * stats also seems to work fine now with historical balance * things seem to be building and working * removed clone from OpenRequestHandle * removed influxdb from workspace members * changed config files * reran sea-orm generate entities, added a foreign key, should be proper now * removed contract from commit * made deposit contract optional * added topic in polygon dev * changed deposit contract to deposit factory contract * added selfrelation on user_tier * added payment required * changed chain id to u64 * add wss in polygon llamarpc * removed origin and method from the table * added onchain transactions naming (and forgot to add a migration before) * changed foreign key to be the referrer (id), not the code itself * forgot to add id as the target foreign key * WIP adding cache to update role * fixed merge conflicts --------- Co-authored-by: Bryan Stitt <bryan@llamanodes.com> Co-authored-by: Bryan Stitt <bryan@stitthappens.com>
2023-05-12 19:45:15 +03:00
#[derive(Iden)]
enum RpcKey {
Table,
Id,
}
#[derive(Iden)]
enum RpcAccountingV2 {
Table,
Id,
RpcKeyId,
ChainId,
Origin,
PeriodDatetime,
Method,
ArchiveNeeded,
ErrorResponse,
FrontendRequests,
BackendRequests,
BackendRetries,
NoServers,
CacheMisses,
CacheHits,
SumRequestBytes,
SumResponseMillis,
SumResponseBytes,
}