Metrics Endpoint

A flexBlock volume is a service with a cache, a queue and two storage tiers behind it. Its metrics tell you when write-back is falling behind, when the bucket or the NVMe device is unhealthy, and when the license is about to change how the volume behaves.

Metrics are off by default. You turn them on per daemon with one start flag.

Enabling the endpoint

flexblock start --metrics-port <PORT> serves a Prometheus text endpoint on 0.0.0.0:<PORT>. The examples in this guide use port 10056.

Flag	Default	Meaning
--metrics-port <PORT>	Off	Prometheus text endpoint on 0.0.0.0:<PORT>/metrics, unauthenticated.

1. Under systemd, add the flag to the instance’s environment file, for example /etc/flexblock/nvme1n1p1.env:

   FLEXBLOCK_START_ARGS=--ublk-dev-id 0 --metrics-port 10056

2. Restart the service so the daemon picks up the new arguments:

   sudo systemctl restart flexblock@nvme1n1p1.service

   A restart is a full stop

   Restarting the service unmounts the filesystem (and removes any NVMe-oF export) through the Requires= dependencies, runs the full shutdown flush, then remounts. Stop applications using the volume first and plan for the flush time. If anything else holds the device, the stop is refused and the restart hangs; see Running Under systemd.

3. Confirm the endpoint answers:

   curl -s http://localhost:10056/metrics | grep -E '^flexblock_(cache_dirty_slots|cache_hits_total|cache_misses_total|cache_oldest_dirty_age_seconds|license_state|obj_errors_total) '

See Running Under systemd for the full service template.

Endpoint behavior

• Answers GET /metrics and GET / in the Prometheus text format (text/plain; version=0.0.4; charset=utf-8).
• Returns 405 for any other method on those paths and 404 for any other path.
• Is unauthenticated.
• Closes each connection after one response, with a 5 second timeout.
• Renders on a background thread pool, so a scrape never runs on the I/O threads.
• Prefixes every series with flexblock_.

The endpoint binds all interfaces

It exposes no data and accepts no commands, but it does reveal volume size, license identity and workload shape. Restrict the port with a host firewall so that only your Prometheus servers can reach it.

Scraping with Prometheus

A minimal static scrape job:

scrape_configs:
  - job_name: flexblock
    static_configs:
      - targets: ['storage-01:10056']

Each daemon serves one volume. With more than one volume on a host, give each daemon its own --metrics-port and list each port as a separate target.

Histograms

Every histogram (the *_duration_seconds, *_wait_seconds and *_hold_seconds series) shares one bucket set, emitted in seconds with the usual _bucket, _sum and _count series. There are 19 upper bounds from 5 microseconds to 5 seconds, plus +Inf:

Range	Upper bounds
Microseconds	5, 10, 25, 50, 100, 250, 500 µs
Milliseconds	1, 2.5, 5, 10, 25, 50, 100, 250, 500 ms
Seconds	1, 2.5, 5 s

Because the bounds are identical across series, you can compare latency histograms directly, for example guest write latency (flexblock_ublk_write_duration_seconds) against cache device write latency (flexblock_nvme_write_duration_seconds).

Next steps

• Alerting: the series that need a person, with example rules.
• Metrics Reference: every series the daemon exposes.
• Inspecting State: flexblock show and quick metric checks.