Support multiple bridges in one monitor (#262)

CONFIGURATION.md

@@ -69,3 +69,4 @@ MONITOR_VALIDATOR_HOME_TX_LIMIT | Average gas usage of a transaction sent by a v
 MONITOR_VALIDATOR_FOREIGN_TX_LIMIT | Average gas usage of a transaction sent by a validator, it is used to estimate the number of transaction that can be paid by the validator. | integer
 MONITOR_TX_NUMBER_THRESHOLD | If estimated number of transaction is equal to or below this value, the monitor will report that the validator has less funds than it is required. | integer
 MONITOR_PORT | The port for the Monitor. | integer
+MONITOR_BRIDGE_NAME | The name to be used in the url path for the bridge | string

deployment-e2e/molecule/monitor/converge.yml

@@ -0,0 +1,3 @@
+---
+- import_playbook: ../../../deployment/site.yml
+- import_playbook: ./run-checks.yml

deployment-e2e/molecule/monitor/molecule.yml

@@ -29,10 +29,11 @@ provisioner:
       r: ["bug"]
   playbooks:
     prepare: ../prepare.yml
-    converge: ../../../deployment/site.yml
+    converge: ./converge.yml
   inventory:
     host_vars:
       monitor-host:
+        MONITOR_PORT: 3003
         syslog_server_port: "udp://127.0.0.1:514"
 verifier:
   name: testinfra

deployment-e2e/molecule/monitor/run-checks.yml

@@ -0,0 +1,7 @@
+---
+- name: Generate initial data for monitor
+  hosts: monitor
+  become: true
+  tasks:
+    - name: Run monitor checks
+      shell: /bin/bash -c 'cd /home/poadocker/bridge/monitor/scripts; ./getBridgeStats.sh >cronWorker.out 2>cronWorker.err'

deployment-e2e/molecule/monitor/tests/test_monitor.py

@@ -34,14 +34,14 @@ def test_logging(host, filename):
 
 def test_home_exists(host):
     assert host.run_test(
-        'curl -s http://localhost:3003 | '
+        'curl -s http://localhost:3003/bridge | '
         'grep -q -i "home"'
     )
 
 
 def test_foreign_exists(host):
     assert host.run_test(
-        'curl -s http://localhost:3003 | '
+        'curl -s http://localhost:3003/bridge | '
         'grep -q -i "foreign"'
     )
 
@@ -49,6 +49,6 @@ def test_foreign_exists(host):
 def test_no_error(host):
     assert host.run_expect(
         [1],
-        'curl -s http://localhost:3003 | '
+        'curl -s http://localhost:3003/bridge | '
         'grep -i -q "error"'
     )

deployment/CONFIGURATION.md

@@ -79,6 +79,10 @@ Example config for installing only UI:
    `cd group_vars`
    2. Note the  <bridge_name> and add it to the hosts.yml configuration. For example, if a bridge file is named sokol-kovan.yml, you would change the <bridge_name> value in hosts.yml to sokol-kovan.
    
+## Examples
+
+[Deploy a monitor for multiple bridges](./MONITOR.md)
+
 ## Administrator Configurations
 
 1. The `group_vars/<bridge_name>.yml` file contains the public bridge parameters. This file is prepared by administrators for each bridge. The validator only needs to add the required bridge name in the hosts.yml file to tell Ansible which file to use.

deployment/MONITOR.md

@@ -0,0 +1,104 @@
+## Deploy multiple bridge monitor on the same host
+
+If you want to deploy a monitor for different bridges, the [monitor variables](../monitor/.env.example) should be configured in `group_vars/<bridge_name>.yml` for each bridge.
+
+For example, let's say we are going to deploy a monitor for xDai bridge and for WETC bridge.
+
+#### Setup ansible configuration for xDai Bridge
+
+First we create `hosts.yml` file to deploy the monitor for xdai bridge
+```yaml
+---
+xdai:
+  children:
+    monitor:
+      hosts:
+        <host_ip_A>:
+          ansible_user: ubuntu
+```
+In `group_vars/xdai.yml`
+```
+---
+MONITOR_BRIDGE_NAME: "xdai"
+MONITOR_PORT: 3003
+
+COMMON_HOME_RPC_URL: "https://dai.poa.network"
+COMMON_HOME_BRIDGE_ADDRESS: "0x7301CFA0e1756B71869E93d4e4Dca5c7d0eb0AA6"
+COMMON_FOREIGN_RPC_URL: "https://mainnet.infura.io/v3/INFURA_KEY"
+COMMON_FOREIGN_BRIDGE_ADDRESS: "0x4aa42145Aa6Ebf72e164C9bBC74fbD3788045016"
+
+COMMON_HOME_GAS_PRICE_FALLBACK: 0
+COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL: "https://gasprice.poa.network/"
+COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE: "standard"
+COMMON_FOREIGN_GAS_PRICE_FALLBACK: 10000000000
+COMMON_FOREIGN_GAS_PRICE_FACTOR: 1
+
+MONITOR_HOME_START_BLOCK: 759
+MONITOR_FOREIGN_START_BLOCK: 6478417
+MONITOR_VALIDATOR_HOME_TX_LIMIT: 0
+MONITOR_VALIDATOR_FOREIGN_TX_LIMIT: 300000
+MONITOR_TX_NUMBER_THRESHOLD: 100
+```
+
+Run the playbook to deploy the monitor for xdai bridge
+```
+ansible-playbook -i hosts.yml site.yml
+```
+
+This command will deploy the monitor component and enable statistics for xdai bridge.
+
+#### Setup ansible configuration for WETC Bridge
+
+Update `hosts.yml` file to deploy the monitor for WETC Bridge
+```yaml
+---
+wetc:
+  children:
+    monitor:
+      hosts:
+        <host_ip_A>:
+          ansible_user: ubuntu
+```
+
+In `group_vars/wetc.yml`
+```
+---
+MONITOR_BRIDGE_NAME: "wetc"
+
+COMMON_HOME_RPC_URL: "https://ethereumclassic.network"
+COMMON_HOME_BRIDGE_ADDRESS: "0x073081832B4Ecdce79d4D6753565c85Ba4b3BeA9"
+COMMON_FOREIGN_RPC_URL: "https://mainnet.infura.io/v3/32e8e252699a4ac1b5dd5c1ef53cc301"
+COMMON_FOREIGN_BRIDGE_ADDRESS: "0x0cB781EE62F815bdD9CD4c2210aE8600d43e7040"
+
+COMMON_HOME_GAS_PRICE_SUPPLIER_URL: "https://gasprice-etc.poa.network/"
+COMMON_HOME_GAS_PRICE_SPEED_TYPE: "standard"
+COMMON_HOME_GAS_PRICE_FALLBACK: 15000000000
+COMMON_HOME_GAS_PRICE_FACTOR: 1
+
+COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL: "https://gasprice.poa.network/"
+COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE: "standard"
+COMMON_FOREIGN_GAS_PRICE_FALLBACK: 10000000000
+ORACLE_FOREIGN_GAS_PRICE_UPDATE_INTERVAL: 600000
+COMMON_FOREIGN_GAS_PRICE_FACTOR: 1
+
+MONITOR_HOME_START_BLOCK: 7703292
+MONITOR_FOREIGN_START_BLOCK: 7412459
+MONITOR_VALIDATOR_HOME_TX_LIMIT: 300000
+MONITOR_VALIDATOR_FOREIGN_TX_LIMIT: 300000
+MONITOR_TX_NUMBER_THRESHOLD: 100
+```
+Given that there is a monitor component deployed in the system, the `MONITOR_PORT` variable is not needed.
+
+Run the playbook to deploy the monitor for WETC Bridge
+```
+ansible-playbook -i hosts.yml site.yml
+```
+
+They playbook will detect that the monitor component is already deployed in the system, so it will only generate the configuration needed to enable the WETC Bridge statistics.
+
+##### Get Monitor results
+The monitor output will be available at `http://host_ip_A:MONITOR_PORT/MONITOR_BRIDGE_NAME`.
+
+Given that in `xdai.env` the variable `MONITOR_BRIDGE_NAME` is set to `xdai`, the results are in the url `http://host_ip_A:3003/xdai/`.
+
+Similar to the xdai case, in `wetc.env` the variable `MONITOR_BRIDGE_NAME` is set to `wetc`, so the results are in the url `http://host_ip_A:3003/wetc/`.

deployment/group_vars/dai.yml

@@ -42,6 +42,7 @@ UI_HOME_GAS_PRICE_UPDATE_INTERVAL: 600000
 UI_FOREIGN_GAS_PRICE_UPDATE_INTERVAL: 600000
 
 ## Monitor
+MONITOR_BRIDGE_NAME: "xdai"
 MONITOR_PORT: 3003
 MONITOR_HOME_START_BLOCK: 759
 MONITOR_FOREIGN_START_BLOCK: 6478417

deployment/group_vars/example.yml

@@ -44,9 +44,10 @@ UI_HOME_GAS_PRICE_UPDATE_INTERVAL: 600000
 UI_FOREIGN_GAS_PRICE_UPDATE_INTERVAL: 600000
 
 ## Monitor
+MONITOR_BRIDGE_NAME: "bridge"
 MONITOR_PORT: 3003
 MONITOR_HOME_START_BLOCK: 0
 MONITOR_FOREIGN_START_BLOCK: 0
 MONITOR_VALIDATOR_HOME_TX_LIMIT: 300000
 MONITOR_VALIDATOR_FOREIGN_TX_LIMIT: 300000
-MONITOR_LEFT_TX_THRESHOLD: 100
+MONITOR_TX_NUMBER_THRESHOLD: 100

deployment/group_vars/ultimate.yml

@@ -41,9 +41,10 @@ UI_FOREIGN_EXPLORER_ADDRESS_TEMPLATE: https://blockscout.com/eth/kovan/address/%
 UI_HOME_GAS_PRICE_UPDATE_INTERVAL: 600000
 UI_FOREIGN_GAS_PRICE_UPDATE_INTERVAL: 600000
 
-#montior
+#monitor
+MONITOR_BRIDGE_NAME: "bridge"
 MONITOR_HOME_START_BLOCK: 0
 MONITOR_FOREIGN_START_BLOCK: 0
 MONITOR_VALIDATOR_HOME_TX_LIMIT: 300000
 MONITOR_VALIDATOR_FOREIGN_TX_LIMIT: 300000
-MONITOR_LEFT_TX_THRESHOLD: 100
+MONITOR_TX_NUMBER_THRESHOLD: 100

deployment/group_vars/wetc.yml

@@ -43,6 +43,7 @@ UI_HOME_GAS_PRICE_UPDATE_INTERVAL: 600000
 UI_FOREIGN_GAS_PRICE_UPDATE_INTERVAL: 600000
 
 ## Monitor
+MONITOR_BRIDGE_NAME: "wetc"
 MONITOR_PORT: 3003
 MONITOR_HOME_START_BLOCK: 7703292
 MONITOR_FOREIGN_START_BLOCK: 7412459

deployment/roles/common/tasks/main.yml

@@ -1,4 +1,21 @@
 ---
-- include_tasks: dependencies.yml
+- name: Check if component is already deployed
-- include_tasks: repo.yml
+  shell: "test -f {{ bridge_path }}/{{ component }}/docker-compose.yml && echo 'true'"
-- include_tasks: logging.yml
+  ignore_errors: True
+  register: already_deployed
+  when: check_deployed is defined
+
+- name: Set if tasks should be skipped
+  set_fact: skip_task="{{ already_deployed.stdout | default('false') }}"
+
+- name: Include dependencies tasks
+  include_tasks: dependencies.yml
+  when: skip_task != true
+
+- name: Include repo tasks
+  include_tasks: repo.yml
+  when: skip_task != true
+
+- name: Include logging tasks
+  include_tasks: logging.yml
+  when: skip_task != true

deployment/roles/monitor/meta/main.yml

@@ -1,3 +1,3 @@
 ---
 dependencies:
-  - role: common
+  - { role: common, check_deployed: true, component: 'monitor' }

deployment/roles/monitor/tasks/cron.yml

@@ -6,7 +6,7 @@
     day: "{{ monitor_cron_schedule.split(' ')[2] }}"
     month: "{{ monitor_cron_schedule.split(' ')[3] }}"
     weekday: "{{ monitor_cron_schedule.split(' ')[4] }}"
-    job: "/bin/bash -c 'cd {{ bridge_path }}/monitor/scripts; ./checkDocker.sh >cronWorker.out 2>cronWorker.err'"
+    job: "/bin/bash -c 'cd {{ bridge_path }}/monitor/scripts; ./getBridgeStats.sh >cronWorker.out 2>cronWorker.err'"
 - name: Add cron entry
   cron:
     name: "RUN_MONITOR_CHECKS"

deployment/roles/monitor/tasks/main.yml

@@ -1,6 +1,18 @@
 ---
 - include_tasks: pre_config.yml
-- include_tasks: logging.yml
+
-- include_tasks: jumpbox.yml
+- name: Include logging tasks
-- include_tasks: servinstall.yml
+  include_tasks: logging.yml
-- include_tasks: cron.yml
+  when: skip_task != true
+
+- name: Include jumpbox tasks
+  include_tasks: jumpbox.yml
+  when: skip_task != true
+
+- name: Include servinstall tasks
+  include_tasks: servinstall.yml
+  when: skip_task != true
+
+- name: Include cron tasks
+  include_tasks: cron.yml
+  when: skip_task != true

deployment/roles/monitor/tasks/pre_config.yml

@@ -3,3 +3,9 @@
   template:
     src: .env.j2
     dest: "{{ bridge_path }}/monitor/.env"
+  when: skip_task != true
+
+- name: Install bridge config env
+  template:
+    src: config.env.j2
+    dest: "{{ bridge_path }}/monitor/configs/{{ MONITOR_BRIDGE_NAME }}.env"

deployment/roles/monitor/templates/.env.j2

@@ -1,24 +1 @@
-COMMON_HOME_RPC_URL={{ COMMON_HOME_RPC_URL }}
-COMMON_FOREIGN_RPC_URL={{ COMMON_FOREIGN_RPC_URL }}
-COMMON_HOME_BRIDGE_ADDRESS={{ COMMON_HOME_BRIDGE_ADDRESS }}
-COMMON_FOREIGN_BRIDGE_ADDRESS={{ COMMON_FOREIGN_BRIDGE_ADDRESS }}
-MONITOR_HOME_START_BLOCK={{ MONITOR_HOME_START_BLOCK }}
-MONITOR_FOREIGN_START_BLOCK={{ MONITOR_FOREIGN_START_BLOCK }}
-MONITOR_VALIDATOR_HOME_TX_LIMIT={{ MONITOR_VALIDATOR_HOME_TX_LIMIT }}
-{% if COMMON_HOME_GAS_PRICE_SUPPLIER_URL | default('') != '' %}
-COMMON_HOME_GAS_PRICE_SUPPLIER_URL={{ COMMON_HOME_GAS_PRICE_SUPPLIER_URL }}
-{% endif %}
-{% if COMMON_HOME_GAS_PRICE_SPEED_TYPE | default('') != '' %}
-COMMON_HOME_GAS_PRICE_SPEED_TYPE={{ COMMON_HOME_GAS_PRICE_SPEED_TYPE }}
-{% endif %}
-COMMON_HOME_GAS_PRICE_FALLBACK={{ COMMON_HOME_GAS_PRICE_FALLBACK }}
-{% if COMMON_HOME_GAS_PRICE_FACTOR | default('') != '' %}
-COMMON_HOME_GAS_PRICE_FACTOR={{ COMMON_HOME_GAS_PRICE_FACTOR }}
-{% endif %}
-MONITOR_VALIDATOR_FOREIGN_TX_LIMIT={{ MONITOR_VALIDATOR_FOREIGN_TX_LIMIT }}
-COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL={{ COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL }}
-COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE={{ COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE }}
-COMMON_FOREIGN_GAS_PRICE_FALLBACK={{ COMMON_FOREIGN_GAS_PRICE_FALLBACK }}
-COMMON_FOREIGN_GAS_PRICE_FACTOR={{ COMMON_FOREIGN_GAS_PRICE_FACTOR }}
-MONITOR_LEFT_TX_THRESHOLD={{ MONITOR_LEFT_TX_THRESHOLD }}
 MONITOR_PORT={{ MONITOR_PORT }}

deployment/roles/monitor/templates/config.env.j2

@@ -0,0 +1,34 @@
+MONITOR_BRIDGE_NAME={{ MONITOR_BRIDGE_NAME }}
+
+COMMON_HOME_RPC_URL={{ COMMON_HOME_RPC_URL }}
+COMMON_HOME_BRIDGE_ADDRESS={{ COMMON_HOME_BRIDGE_ADDRESS }}
+COMMON_FOREIGN_RPC_URL={{ COMMON_FOREIGN_RPC_URL }}
+COMMON_FOREIGN_BRIDGE_ADDRESS={{ COMMON_FOREIGN_BRIDGE_ADDRESS }}
+
+{% if COMMON_HOME_GAS_PRICE_SUPPLIER_URL | default('') != '' %}
+COMMON_HOME_GAS_PRICE_SUPPLIER_URL={{ COMMON_HOME_GAS_PRICE_SUPPLIER_URL }}
+{% endif %}
+{% if COMMON_HOME_GAS_PRICE_SPEED_TYPE | default('') != '' %}
+COMMON_HOME_GAS_PRICE_SPEED_TYPE={{ COMMON_HOME_GAS_PRICE_SPEED_TYPE }}
+{% endif %}
+COMMON_HOME_GAS_PRICE_FALLBACK={{ COMMON_HOME_GAS_PRICE_FALLBACK }}
+{% if COMMON_HOME_GAS_PRICE_FACTOR | default('') != '' %}
+COMMON_HOME_GAS_PRICE_FACTOR={{ COMMON_HOME_GAS_PRICE_FACTOR }}
+{% endif %}
+
+{% if COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL | default('') != '' %}
+COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL={{ COMMON_FOREIGN_GAS_PRICE_SUPPLIER_URL }}
+{% endif %}
+{% if COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE | default('') != '' %}
+COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE={{ COMMON_FOREIGN_GAS_PRICE_SPEED_TYPE }}
+{% endif %}
+COMMON_FOREIGN_GAS_PRICE_FALLBACK={{ COMMON_FOREIGN_GAS_PRICE_FALLBACK }}
+{% if COMMON_FOREIGN_GAS_PRICE_FACTOR | default('') != '' %}
+COMMON_FOREIGN_GAS_PRICE_FACTOR={{ COMMON_FOREIGN_GAS_PRICE_FACTOR }}
+{% endif %}
+
+MONITOR_HOME_START_BLOCK={{ MONITOR_HOME_START_BLOCK }}
+MONITOR_FOREIGN_START_BLOCK={{ MONITOR_FOREIGN_START_BLOCK }}
+MONITOR_VALIDATOR_HOME_TX_LIMIT={{ MONITOR_VALIDATOR_HOME_TX_LIMIT }}
+MONITOR_VALIDATOR_FOREIGN_TX_LIMIT={{ MONITOR_VALIDATOR_FOREIGN_TX_LIMIT }}
+MONITOR_TX_NUMBER_THRESHOLD={{ MONITOR_TX_NUMBER_THRESHOLD }}

deployment/roles/monitor/templates/tokenbridge-monitor.j2

@@ -18,6 +18,7 @@ start(){
     sudo -u "{{ compose_service_user }}" /usr/local/bin/docker-compose  down -v
     sudo -u "{{ compose_service_user }}" /usr/local/bin/docker-compose  rm -fv
     sudo -u "{{ compose_service_user }}" /usr/local/bin/docker-compose  up  --detach
+    sudo -u "{{ compose_service_user }}" /bin/bash -c 'cd scripts; ./getBridgeStats.sh >cronWorker.out 2>cronWorker.err'
 }
 
 stop(){

e2e-commons/components-envs/monitor-amb.env

@@ -16,3 +16,4 @@ COMMON_FOREIGN_GAS_PRICE_FALLBACK=1000000000
 COMMON_FOREIGN_GAS_PRICE_FACTOR=1
 MONITOR_TX_NUMBER_THRESHOLD=100
 MONITOR_PORT=3013
+MONITOR_BRIDGE_NAME=bridge

e2e-commons/components-envs/monitor-erc20-native.env

@@ -16,3 +16,4 @@ COMMON_FOREIGN_GAS_PRICE_FALLBACK=1000000000
 COMMON_FOREIGN_GAS_PRICE_FACTOR=1
 MONITOR_TX_NUMBER_THRESHOLD=100
 MONITOR_PORT=3012
+MONITOR_BRIDGE_NAME=bridge

e2e-commons/components-envs/monitor-erc20.env

@@ -16,3 +16,4 @@ COMMON_FOREIGN_GAS_PRICE_FALLBACK=1000000000
 COMMON_FOREIGN_GAS_PRICE_FACTOR=1
 MONITOR_TX_NUMBER_THRESHOLD=100
 MONITOR_PORT=3011
+MONITOR_BRIDGE_NAME=bridge

e2e-commons/components-envs/monitor.env

@@ -16,3 +16,4 @@ COMMON_FOREIGN_GAS_PRICE_FALLBACK=1000000000
 COMMON_FOREIGN_GAS_PRICE_FACTOR=1
 MONITOR_TX_NUMBER_THRESHOLD=100
 MONITOR_PORT=3010
+MONITOR_BRIDGE_NAME=bridge

e2e-commons/constants.json

@@ -20,7 +20,7 @@
     "foreign": "0x2B6871b9B02F73fa24F4864322CdC78604207769",
     "foreignToken": "0xdbeE25CbE97e4A5CC6c499875774dc7067E9426B",
     "ui": "http://localhost:3000",
-    "monitor": "http://monitor:3010"
+    "monitor": "http://monitor:3010/bridge"
   },
   "ercToErcBridge": {
     "home": "0x1feB40aD9420b186F019A717c37f5546165d411E",
@@ -28,7 +28,7 @@
     "homeToken": "0x792455a6bCb62Ed4C4362D323E0590654CA4765c",
     "foreignToken": "0x3C665A31199694Bf723fD08844AD290207B5797f",
     "ui": "http://localhost:3001",
-    "monitor": "http://monitor-erc20:3011"
+    "monitor": "http://monitor-erc20:3011/bridge"
   },
   "ercToNativeBridge": {
     "home": "0x488Af810997eD1730cB3a3918cD83b3216E6eAda",
@@ -37,14 +37,14 @@
     "halfDuplexToken": "0x89d24A6b4CcB1B6fAA2625fE562bDD9a23260359",
     "saiTop": "0x9b0ccf7C8994E19F39b2B4CF708e0A7DF65fA8a3",
     "ui": "http://localhost:3002",
-    "monitor": "http://monitor-erc20-native:3012"
+    "monitor": "http://monitor-erc20-native:3012/bridge"
   },
   "amb": {
     "home": "0x0AEe1FCD12dDFab6265F7f8956e6E012A9Fe4Aa0",
     "foreign": "0x0AEe1FCD12dDFab6265F7f8956e6E012A9Fe4Aa0",
     "homeBox": "0x6C4EaAb8756d53Bf599FFe2347FAFF1123D6C8A1",
     "foreignBox": "0x6C4EaAb8756d53Bf599FFe2347FAFF1123D6C8A1",
-    "monitor": "http://monitor-amb:3013"
+    "monitor": "http://monitor-amb:3013/bridge"
   },
   "homeRPC": {
     "URL": "http://parity1:8545",

monitor-e2e/utils.js

@@ -1,7 +1,7 @@
 const Web3 = require('web3')
 const { ERC677_BRIDGE_TOKEN_ABI, BRIDGE_VALIDATORS_ABI, FOREIGN_NATIVE_TO_ERC_ABI, BOX_ABI } = require('../commons')
 
-const waitUntil = async (predicate, step = 100, timeout = 10000) => {
+const waitUntil = async (predicate, step = 100, timeout = 20000) => {
   const stopTime = Date.now() + timeout
   while (Date.now() <= stopTime) {
     const result = await predicate()

monitor-e2e/wait-for-monitor.sh

@@ -3,7 +3,7 @@ FILES=(getBalances.json validators.json eventsStats.json alerts.json)
 check_files_exist() {
   rc=0
   for f in "${FILES[@]}"; do
-    command="test -f responses/$f"
+    command="test -f responses/bridge/$f"
     (docker-compose -f ../e2e-commons/docker-compose.yml exec monitor /bin/bash -c "$command") || rc=1
     (docker-compose -f ../e2e-commons/docker-compose.yml exec monitor-erc20 /bin/bash -c "$command") || rc=1
     (docker-compose -f ../e2e-commons/docker-compose.yml exec monitor-erc20-native /bin/bash -c "$command") || rc=1

monitor/.env.example

@@ -1,3 +1,5 @@
+MONITOR_BRIDGE_NAME=bridge
+
 COMMON_HOME_RPC_URL=https://sokol.poa.network
 COMMON_FOREIGN_RPC_URL=https://kovan.infura.io/mew
 COMMON_HOME_BRIDGE_ADDRESS=0xABb4C1399DcC28FBa3Beb76CAE2b50Be3e087353

monitor/README.md

@@ -126,7 +126,7 @@ Using Docker:
 docker-compose up -d
 ```
 
-- The application will run on `http://localhost:PORT`, where `PORT` is specified in your `.env` file.
+- The application will run on `http://localhost:MONITOR_PORT/MONITOR_BRIDGE_NAME`, where `MONITOR_PORT` and `MONITOR_BRIDGE_NAME` are specified in your `.env` file.
 - To enabled debug logging, set `DEBUG=1` variable in `.env`.
 
 ## Check balances of contracts and validators, get unprocessed events

monitor/checkWorker.js

@@ -1,13 +1,17 @@
-const fs = require('fs')
-const path = require('path')
 const Web3 = require('web3')
 const logger = require('./logger')('checkWorker')
 const { getBridgeMode } = require('../commons')
 const getBalances = require('./getBalances')
 const getShortEventStats = require('./getShortEventStats')
 const validators = require('./validators')
+const { writeFile, createDir } = require('./utils/file')
+
+const { COMMON_HOME_BRIDGE_ADDRESS, COMMON_HOME_RPC_URL, MONITOR_BRIDGE_NAME } = process.env
+
+const MONITOR_VALIDATOR_HOME_TX_LIMIT = Number(process.env.MONITOR_VALIDATOR_HOME_TX_LIMIT) || 0
+const MONITOR_VALIDATOR_FOREIGN_TX_LIMIT = Number(process.env.MONITOR_VALIDATOR_FOREIGN_TX_LIMIT) || 0
+const MONITOR_TX_NUMBER_THRESHOLD = Number(process.env.MONITOR_TX_NUMBER_THRESHOLD) || 100
 
-const { COMMON_HOME_BRIDGE_ADDRESS, COMMON_HOME_RPC_URL } = process.env
 const homeProvider = new Web3.providers.HttpProvider(COMMON_HOME_RPC_URL)
 const web3Home = new Web3(homeProvider)
 
@@ -15,6 +19,7 @@ const { HOME_ERC_TO_ERC_ABI } = require('../commons')
 
 async function checkWorker() {
   try {
+    createDir(`/responses/${MONITOR_BRIDGE_NAME}`)
     const homeBridge = new web3Home.eth.Contract(HOME_ERC_TO_ERC_ABI, COMMON_HOME_BRIDGE_ADDRESS)
     const bridgeMode = await getBridgeMode(homeBridge)
     logger.debug('Bridge mode:', bridgeMode)
@@ -26,12 +31,35 @@ async function checkWorker() {
     const foreign = Object.assign({}, balances.foreign, events.foreign)
     const status = Object.assign({}, balances, events, { home }, { foreign })
     if (!status) throw new Error('status is empty: ' + JSON.stringify(status))
-    fs.writeFileSync(path.join(__dirname, '/responses/getBalances.json'), JSON.stringify(status, null, 4))
+    writeFile(`/responses/${MONITOR_BRIDGE_NAME}/getBalances.json`, status)
 
     logger.debug('calling validators()')
     const vBalances = await validators(bridgeMode)
     if (!vBalances) throw new Error('vBalances is empty: ' + JSON.stringify(vBalances))
-    fs.writeFileSync(path.join(__dirname, '/responses/validators.json'), JSON.stringify(vBalances, null, 4))
+
+    vBalances.homeOk = true
+    vBalances.foreignOk = true
+
+    if (MONITOR_VALIDATOR_HOME_TX_LIMIT) {
+      for (const hv in vBalances.home.validators) {
+        if (vBalances.home.validators[hv].leftTx < MONITOR_TX_NUMBER_THRESHOLD) {
+          vBalances.homeOk = false
+          break
+        }
+      }
+    }
+
+    if (MONITOR_VALIDATOR_FOREIGN_TX_LIMIT) {
+      for (const hv in vBalances.foreign.validators) {
+        if (vBalances.foreign.validators[hv].leftTx < MONITOR_TX_NUMBER_THRESHOLD) {
+          vBalances.foreignOk = false
+          break
+        }
+      }
+    }
+
+    vBalances.ok = vBalances.homeOk && vBalances.foreignOk
+    writeFile(`/responses/${MONITOR_BRIDGE_NAME}/validators.json`, vBalances)
     logger.debug('Done')
   } catch (e) {
     logger.error(e)

monitor/checkWorker2.js

@@ -1,19 +1,28 @@
-const fs = require('fs')
-const path = require('path')
 const logger = require('./logger')('checkWorker2')
 const eventsStats = require('./eventsStats')
 const alerts = require('./alerts')
+const { writeFile, createDir } = require('./utils/file')
+
+const { MONITOR_BRIDGE_NAME } = process.env
 
 async function checkWorker2() {
   try {
+    createDir(`/responses/${MONITOR_BRIDGE_NAME}`)
     logger.debug('calling eventsStats()')
     const evStats = await eventsStats()
     if (!evStats) throw new Error('evStats is empty: ' + JSON.stringify(evStats))
-    fs.writeFileSync(path.join(__dirname, '/responses/eventsStats.json'), JSON.stringify(evStats, null, 4))
+    evStats.ok =
+      (evStats.onlyInHomeDeposits || evStats.home.deliveredMsgNotProcessedInForeign).length === 0 &&
+      (evStats.onlyInForeignDeposits || evStats.home.processedMsgNotDeliveredInForeign).length === 0 &&
+      (evStats.onlyInHomeWithdrawals || evStats.foreign.deliveredMsgNotProcessedInHome).length === 0 &&
+      (evStats.onlyInForeignWithdrawals || evStats.foreign.processedMsgNotDeliveredInHome).length === 0
+    writeFile(`/responses/${MONITOR_BRIDGE_NAME}/eventsStats.json`, evStats)
+
     logger.debug('calling alerts()')
     const _alerts = await alerts()
     if (!_alerts) throw new Error('alerts is empty: ' + JSON.stringify(_alerts))
-    fs.writeFileSync(path.join(__dirname, '/responses/alerts.json'), JSON.stringify(_alerts, null, 4))
+    _alerts.ok = !_alerts.executeAffirmations.mostRecentTxHash && !_alerts.executeSignatures.mostRecentTxHash
+    writeFile(`/responses/${MONITOR_BRIDGE_NAME}/alerts.json`, _alerts)
     logger.debug('Done x2')
   } catch (e) {
     logger.error(e)

monitor/checkWorker3.js

@@ -1,15 +1,18 @@
-const fs = require('fs')
-const path = require('path')
 const logger = require('./logger')('checkWorker3')
 const stuckTransfers = require('./stuckTransfers')
+const { writeFile, createDir } = require('./utils/file')
+
+const { MONITOR_BRIDGE_NAME } = process.env
 
 async function checkWorker3() {
   try {
+    createDir(`/responses/${MONITOR_BRIDGE_NAME}`)
     logger.debug('calling stuckTransfers()')
     const transfers = await stuckTransfers()
     // console.log(transfers)
     if (!transfers) throw new Error('transfers is empty: ' + JSON.stringify(transfers))
-    fs.writeFileSync(path.join(__dirname, '/responses/stuckTransfers.json'), JSON.stringify(transfers, null, 4))
+    transfers.ok = transfers.total.length === 0
+    writeFile(`/responses/${MONITOR_BRIDGE_NAME}/stuckTransfers.json`, transfers)
     logger.debug('Done')
   } catch (e) {
     logger.error('checkWorker3.js', e)

monitor/docker-compose.yml

@@ -10,5 +10,7 @@ services:
     env_file: ./.env
     environment: 
       - NODE_ENV=production
+    volumes:
+      - ./responses:/mono/monitor/responses
     restart: unless-stopped
-    entrypoint: "yarn check-and-start"
+    entrypoint: "yarn start"

monitor/index.js

@@ -1,47 +1,15 @@
 require('dotenv').config()
 const express = require('express')
-const fs = require('fs')
+const { readFile } = require('./utils/file')
-const { isV1Bridge } = require('./utils/serverUtils')
 
 const app = express()
+const bridgeRouter = express.Router({ mergeParams: true })
 
-const MONITOR_VALIDATOR_HOME_TX_LIMIT = Number(process.env.MONITOR_VALIDATOR_HOME_TX_LIMIT) || 0
+app.use('/:bridgeName', bridgeRouter)
-const MONITOR_VALIDATOR_FOREIGN_TX_LIMIT = Number(process.env.MONITOR_VALIDATOR_FOREIGN_TX_LIMIT) || 0
-const MONITOR_TX_NUMBER_THRESHOLD = Number(process.env.MONITOR_TX_NUMBER_THRESHOLD) || 100
-console.log('MONITOR_TX_NUMBER_THRESHOLD = ' + MONITOR_TX_NUMBER_THRESHOLD)
 
-async function readFile(path) {
+bridgeRouter.get('/', async (req, res, next) => {
   try {
-    const content = await fs.readFileSync(path)
+    const results = await readFile(`./responses/${req.params.bridgeName}/getBalances.json`)
-    const json = JSON.parse(content)
-    const timeDiff = Math.floor(Date.now() / 1000) - json.lastChecked
-    return Object.assign({}, json, { timeDiff })
-  } catch (e) {
-    console.error(e)
-    return {
-      error: 'please check your worker'
-    }
-  }
-}
-
-async function initV1routes(app) {
-  const exposeV1Routes = await isV1Bridge()
-  if (exposeV1Routes) {
-    app.get('/stuckTransfers', async (req, res, next) => {
-      try {
-        const results = await readFile('./responses/stuckTransfers.json')
-        results.ok = results.total.length === 0
-        res.json(results)
-      } catch (e) {
-        next(e)
-      }
-    })
-  }
-}
-
-app.get('/', async (req, res, next) => {
-  try {
-    const results = await readFile('./responses/getBalances.json')
     res.json(results)
   } catch (e) {
     // this will eventually be handled by your error handling middleware
@@ -49,31 +17,9 @@ app.get('/', async (req, res, next) => {
   }
 })
 
-app.get('/validators', async (req, res, next) => {
+bridgeRouter.get('/validators', async (req, res, next) => {
   try {
-    const results = await readFile('./responses/validators.json')
+    const results = await readFile(`./responses/${req.params.bridgeName}/validators.json`)
-    results.homeOk = true
-    results.foreignOk = true
-
-    if (MONITOR_VALIDATOR_HOME_TX_LIMIT) {
-      for (const hv in results.home.validators) {
-        if (results.home.validators[hv].leftTx < MONITOR_TX_NUMBER_THRESHOLD) {
-          results.homeOk = false
-          break
-        }
-      }
-    }
-
-    if (MONITOR_VALIDATOR_FOREIGN_TX_LIMIT) {
-      for (const hv in results.foreign.validators) {
-        if (results.foreign.validators[hv].leftTx < MONITOR_TX_NUMBER_THRESHOLD) {
-          results.foreignOk = false
-          break
-        }
-      }
-    }
-
-    results.ok = results.homeOk && results.foreignOk
     res.json(results)
   } catch (e) {
     // this will eventually be handled by your error handling middleware
@@ -81,15 +27,9 @@ app.get('/validators', async (req, res, next) => {
   }
 })
 
-// responses/eventsStats.json
+bridgeRouter.get('/eventsStats', async (req, res, next) => {
-app.get('/eventsStats', async (req, res, next) => {
   try {
-    const results = await readFile('./responses/eventsStats.json')
+    const results = await readFile(`./responses/${req.params.bridgeName}/eventsStats.json`)
-    results.ok =
-      (results.onlyInHomeDeposits || results.home.deliveredMsgNotProcessedInForeign).length === 0 &&
-      (results.onlyInForeignDeposits || results.home.processedMsgNotDeliveredInForeign).length === 0 &&
-      (results.onlyInHomeWithdrawals || results.foreign.deliveredMsgNotProcessedInHome).length === 0 &&
-      (results.onlyInForeignWithdrawals || results.foreign.processedMsgNotDeliveredInHome).length === 0
     res.json(results)
   } catch (e) {
     // this will eventually be handled by your error handling middleware
@@ -97,17 +37,23 @@ app.get('/eventsStats', async (req, res, next) => {
   }
 })
 
-app.get('/alerts', async (req, res, next) => {
+bridgeRouter.get('/alerts', async (req, res, next) => {
   try {
-    const results = await readFile('./responses/alerts.json')
+    const results = await readFile(`./responses/${req.params.bridgeName}/alerts.json`)
-    results.ok = !results.executeAffirmations.mostRecentTxHash && !results.executeSignatures.mostRecentTxHash
     res.json(results)
   } catch (e) {
     next(e)
   }
 })
 
-initV1routes(app)
+bridgeRouter.get('/stuckTransfers', async (req, res, next) => {
+  try {
+    const results = await readFile(`./responses/${req.params.bridgeName}/stuckTransfers.json`)
+    res.json(results)
+  } catch (e) {
+    next(e)
+  }
+})
 
 const port = process.env.MONITOR_PORT || 3003
 app.set('port', port)

monitor/scripts/checkDocker.sh

@@ -1,9 +0,0 @@
-#!/bin/bash
-cd $(dirname $0)/..
-
-if /usr/local/bin/docker-compose ps | grep -q -i 'monitor'; then
-  # https://github.com/docker/compose/issues/3352
-  COMPOSE_INTERACTIVE_NO_CLI=1 /usr/local/bin/docker-compose exec -T monitor /bin/bash -c 'yarn check-all'
-else
-  echo "Monitor is not running, skipping checks."
-fi

monitor/scripts/getBridgeStats.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+cd $(dirname $0)/..
+
+if /usr/local/bin/docker-compose ps | grep -q -i 'monitor'; then
+  for file in configs/*.env
+  do
+    docker run --rm --env-file $file -v $(pwd)/responses:/mono/monitor/responses monitor_monitor /bin/bash -c 'yarn check-all'
+  done
+else
+  echo "Monitor is not running, skipping checks."
+fi

monitor/utils/file.js

@@ -0,0 +1,36 @@
+const fs = require('fs')
+const path = require('path')
+
+async function readFile(filePath) {
+  try {
+    const content = await fs.readFileSync(filePath)
+    const json = JSON.parse(content)
+    const timeDiff = Math.floor(Date.now() / 1000) - json.lastChecked
+    return Object.assign({}, json, { timeDiff })
+  } catch (e) {
+    console.error(e)
+    return {
+      error: 'the bridge statistics are not available'
+    }
+  }
+}
+
+function writeFile(filePath, object) {
+  fs.writeFileSync(path.join(process.cwd(), filePath), JSON.stringify(object, null, 4))
+}
+
+function createDir(dirPath) {
+  try {
+    fs.mkdirSync(path.join(process.cwd(), dirPath), { recursive: true })
+  } catch (e) {
+    if (!e.message.includes('exists')) {
+      throw e
+    }
+  }
+}
+
+module.exports = {
+  readFile,
+  writeFile,
+  createDir
+}