This guide explains how to monitor the usage, temperature, and metrics of two NVIDIA V100 GPUs on an Ubuntu 24.04 workstation using a professional-grade monitoring stack.

The industry standard for high-end GPU monitoring is the Prometheus + Grafana + NVIDIA DCGM Exporter stack:

* NVIDIA DCGM Exporter: Scrapes GPU metrics (Temperature, Power, Memory, Clock Speeds) and converts them into a format databases can read. * Prometheus: A time-series database that collects and stores the metrics every few seconds. * Grafana: The visualization layer that turns raw numbers into interactive, professional-grade graphs.

This is the cleanest method for Ubuntu 24.04. It keeps all monitoring tools isolated in containers.

To allow Docker to “see” your V100 GPUs, you must install the toolkit: sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list # Update and install sudo apt-get update sudo apt-get install -y nvidia-container-toolkit # Configure Docker to use the NVIDIA driver sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker

1. Create a Directory:

  {{
  mkdir ~/gpu-monitor
  cd ~/gpu-monitor
  }}

2. Create Prometheus Configuration:

  Create `prometheus.yml`:
  {{
  global:
    scrape_interval: 5s

  scrape_configs:
    - job_name: 'nvidia-gpu'
      static_configs:
        - targets: ['nvidia-dcgm-exporter:9400']
  }}

3. Create Docker Compose File:

  Create `docker-compose.yml`:
  {{
  version: '3.8'

  services:
    nvidia-dcgm-exporter:
      image: nvidia/dcgm-exporter:latest
      container_name: nvidia-dcgm-exporter
      deploy:
        resources:
          reservations:
            devices:
              - driver: nvidia
                count: all
                capabilities: [gpu]
      ports:
        - "9400:9400"
      restart: always

    prometheus:
      image: prom/prometheus:latest
      container_name: prometheus
      volumes:
        - ./prometheus.yml:/etc/prometheus/prometheus.yml
        - prometheus_data:/prometheus
      ports:
        - "9090:9090"
      restart: always

    grafana:
      image: grafana/grafana:latest
      container_name: grafana
      ports:
        - "3000:3000"
      volumes:
        - grafana_data:/var/lib/grafana
      restart: always
      environment:
        - GF_SECURITY_ADMIN_PASSWORD=admin

  volumes:
    prometheus_data:
    grafana_data:
  }}

4. Launch the Stack:

  {{
  docker compose up -d
  }}

* Checking the Exporter:

  Check if the container is actually seeing the hardware by running:
  {{
  docker logs nvidia-dcgm-exporter
  }}
  *Look for: "DCGM successfully initialized!"*

* Verifying the Data Feed:

  Visit `http://<your-ip>:9400` in your browser. You should see a wall of raw text (metrics).

* Connecting Grafana to Prometheus:

  1. Open Grafana (`http://<your-ip>:3000`).
  2. Go to **Connections** -> **Data Sources** -> **Add Prometheus**.
  3. Set the URL to: `http://prometheus:9090` (Use the Docker service name, not localhost).
  4. Click **Save & Test**.

To get the actual visual graphs: 1. Visit the [Grafana Dashboard Marketplace](https://grafana.com/grafana/dashboards/). 2. Search for “NVIDIA DCGM”. 3. Find a dashboard you like (e.g., ID 17499). 4. In your Grafana instance, click Import and enter the ID, or download the JSON file from the website and upload it via the Import menu.