datablogs: monitoring

In this experimental scenario , gathering the table growth statistics using query exporter and loading the data into time series Prometheus database . Grafana dashboards shows the growth in minutes basis

Table growth statistics gathered from localhost mysql instance for testing purpose

Frequently receiving questions like , how to we monitor the data growth of each critical tables. We can have some statistics about data growth and add convenient space for future

Launched three separate docker containers ( Query Exporter , Prometheus and PMM Server ) for each components . Added additional Dashboards in PMM Server for testing purpose

Prometheus and Query Exporter Configuration files mapped with localhost volume like below ,

docker run -p 9560:9560/tcp -v "$PWD/config.yaml:/config.yaml" --rm -it adonato/query-exporter:latest -- /config.yaml

docker run -p 9090:9090 -v /tmp/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

config.yaml ( Query Exporter configuration file ) :

databases:
  datablogs:
    dsn: mysql://queryexporter:test123@172.31.90.32:3306/test
    keep-connected: true

metrics:
  metric1:
    type: gauge
   # type: counter
    description: A sample gauge

queries:
  query1:
    interval: 3s
    databases: [datablogs]
    metrics: [metric1]
    sql: SELECT TABLE_ROWS as metric1 FROM information_schema.tables WHERE table_schema = 'test' and table_name='actor';

prometheus ( Prometheus configuration file ) :

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'queryexporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['172.17.0.3:9560']

Both Query Exporter and Prometheus docker container started and we have to allow appropriate ports to access the web console through browser