Write-Back Behavior

Writes are acknowledged once they are durable on the cache device. A background flush loop then uploads dirty chunks to object storage. The flags on this page decide how long data may stay dirty, how much of the cache it may fill, and how hard flexBlock pushes the bucket to keep up.

Why it matters

Until a dirty chunk is uploaded, it exists only on the cache device. That gap is the write-back window: the acknowledged writes a cache device failure, or the loss of an ephemeral instance, would cost. flexblock_cache_oldest_dirty_age_seconds reports it at any moment.

Not a point-in-time copy

The bucket is not a point-in-time copy of the volume. Chunks upload one at a time as they age, and a filesystem flush only reaches the cache device, so after an abrupt loss of the cache the bucket holds each chunk as it was last uploaded: recent writes to different chunks may be only partly present. Expect the filesystem to replay its journal or need a repair (xfs_repair, fsck), and verify databases before trusting them. See Where Data Can Be Lost.

Flush timing

Flag	Default	Meaning
--flush-interval <D>	5s	How often the flush loop wakes; shorter as the dirty ratio climbs.
--flush-quiet <D>	5s, or half of --flush-threshold in whole seconds if shorter	Skip the age-based flush of a chunk written this recently, but not past twice --flush-threshold. 0s disables. Must be shorter than --flush-threshold.
--flush-threshold <D>	4m	Age at which a dirty chunk is flushed.
--max-dirty-ratio <R>	0.8	Above this share of the cache dirty, flush hard and pace writes to new chunks. In (0, 1].
--target-dirty-ratio <R>	0.6	Stop flushing hard and pacing below this. In (0, 1), below --max-dirty-ratio.

--flush-threshold is how long a chunk may sit dirty before age alone flushes it. Lowering it shrinks the acknowledged data a cache device failure would cost, at the price of more uploads. With the defaults, the write-back window is about four minutes while uploads keep up with writes, and about eight for a chunk under continuous writes.

--flush-quiet keeps the age-based pass off chunks written more recently than this, so hot data is not uploaded and then dirtied again at once. The deferral ends at twice the threshold, and flushes driven by the dirty ratio or by write-ahead log pressure ignore it.

--max-dirty-ratio and --target-dirty-ratio bound how much of the cache may be dirty. When the dirty share reaches the maximum, the loop flushes aggressively regardless of age, and paces writes that dirty new chunks, until the share falls below the target.

Write pacing under pressure

From the moment the dirty ratio reaches --max-dirty-ratio until it falls below --target-dirty-ratio, a write that would dirty a chunk that is clean or not cached is paced to the measured upload rate, slowing further as the cache fills. This keeps the dirty set shrinking instead of pinning the cache full.

• Writes to chunks that are already dirty are never delayed, so a database’s hot pages, journal and log keep their latency.
• If uploads stop altogether, pacing still admits one new dirty chunk a second, so writes slow to a crawl rather than fail, until a flush batch exhausts its retry budget (flexblock_flush_exhausted), after which a write that finds the cache full fails.
• During pacing, flexblock_writeback_throttle_chunks_per_second is above 0 and flexblock_writeback_throttle_wait_seconds_total climbs.

This is expected during a write burst larger than the cache. If it persists, see Common Issues.

Object storage concurrency

Flag	Default	Meaning
--concurrent-compressions <N>	Physical cores, 2 to 32	Chunks compressed at once during flush.
--concurrent-deletes <N>	4 × vCPUs, 8 to 256	Concurrent DELETE batches.
--concurrent-downloads <N>	4 × vCPUs, 8 to 128	Concurrent GETs for misses and prefetch.
--concurrent-flush-reads <N>	24	Flush reads and fetches at once; raised along with uploads under pressure.
--concurrent-uploads <N>	4 × vCPUs, 8 to 64	Concurrent PUTs; up to 4× under pressure.
--obj-rate-limit <RPS>	Unlimited	Cap on object storage GET and PUT requests per second, for stores with quotas; WAL-head flushes are exempt.
--retry-cutoff <D>	24h	How long a failing object storage operation is retried before it counts in flexblock_obj_errors_total.

When write-back falls behind (the dirty ratio is high, a write is waiting on the write-ahead log, or the oldest dirty chunk is older than twice --flush-threshold plus --flush-interval), the daemon raises upload concurrency by a quarter of --concurrent-uploads each second, up to four times that, or less where free memory would not hold the extra buffers. It backs off when object storage starts retrying requests or uploads slow down, and steps back to the base value once the pressure is gone. flexblock_obj_upload_capacity shows the current value, and the start log records the writeback boost ceiling.

Tip

If writes stall with no object storage errors or retries, and flexblock_obj_upload_capacity sits at the boost ceiling, raise --concurrent-uploads or use a larger cache. Higher concurrency also raises memory use; see Memory.

Shutdown

Flag	Default	Meaning
--shutdown-flush-timeout <D>	1h	Time budget for flushing and discard deletes on SIGINT, SIGTERM or SIGHUP.

Whatever is still dirty at the timeout stays on the cache device for the next start to flush. Under systemd, keep TimeoutStopSec above this value; see Running Under systemd.

Cache device write durability

Flag	Default	Meaning
--no-fua	Off	Sync write-ahead log, superblock and chunk data writes with fdatasync instead of RWF_DSYNC; do not advertise FUA to the guest.
--volatile-write-cache	Off	Advertise a volatile write cache and sync on FLUSH, for a cache device with one.

CPU idle states

Every request crosses several thread wakeups (application, kernel, flexBlock queue thread, NVMe completion). A CPU core parked in a deep idle state (C-state) adds its exit latency to each one: 170 to 290 µs for C6 on current Xeons. Under light or synchronous load that multiplies. In one test, a single-threaded workload that ran fsync after each file was six times slower with the kernel default than with the cap.

So by default, while it serves, the daemon caps CPU idle states to C1.

Flag	Default	Meaning
--cpu-idle-latency <US|off>	1 (allows only C1)	Deepest CPU idle state allowed while serving, as an exit latency in microseconds. off or 0 leaves the platform policy alone. At most 1000000.

• The cap applies host-wide while the daemon runs, and also speeds up the raw NVMe device.
• A host that cannot open /dev/cpu_dma_latency (no root, or a container without the device) logs a warning and serves without the cap.
• Set --cpu-idle-latency off if you prefer the platform’s power policy over latency.

Related

• Alerting
• Cache Replacement Policy
• flexblock start