Monitoring Data Growth With Query Exporter + Prometheus + Grafana ~ datablogs

Monday, May 4, 2020

Monitoring Data Growth With Query Exporter + Prometheus + Grafana

In this experimental scenario , gathering the table growth statistics using query exporter and loading the data into time series Prometheus database . Grafana dashboards shows the growth in minutes basis

Table growth statistics gathered from localhost mysql instance for testing purpose

Frequently receiving questions like , how to we monitor the data growth of each critical tables. We can have some statistics about data growth and add convenient space for future

Launched three separate docker containers ( Query Exporter , Prometheus and PMM Server ) for each components . Added additional Dashboards in PMM Server for testing purpose

Docker Containers

Prometheus and Query Exporter Configuration files mapped with localhost volume like below ,

docker run -p 9560:9560/tcp -v "$PWD/config.yaml:/config.yaml" --rm -it adonato/query-exporter:latest -- /config.yaml

docker run -p 9090:9090 -v /tmp/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

config.yaml ( Query Exporter configuration file ) :

    dsn: mysql://queryexporter:test123@
    keep-connected: true

    type: gauge
   # type: counter
    description: A sample gauge

    interval: 3s
    databases: [datablogs]
    metrics: [metric1]
    sql: SELECT TABLE_ROWS as metric1 FROM information_schema.tables WHERE table_schema = 'test' and table_name='actor';

prometheus ( Prometheus configuration file ) :

  scrape_interval: 15s

  - job_name: 'queryexporter'
    scrape_interval: 5s
      - targets: ['']

Both Query Exporter and Prometheus docker container started and we have to allow appropriate ports to access the web console through browser

AWS Console Inbound Rules

Lets verify the Query Exporter and Prometheus running status

Query exporter metrics
Prometheus targets status

Query Exporter is gathering the statistics and loading the data into Prometheus database . Then

Add the Prometheus data source into Grafana

Using the collected metrics display the dashboards on every minutes

Monitoring Dashboards

References :

Query Exporter :

PMM Server :

Prometheus :



Post a Comment