Where Data Can Be Lost

Everything in flexBlock is designed to refuse rather than lose data. This page lists the exceptions: the few commands that discard data on purpose, and the one hardware event that costs acknowledged writes. Read it before you run any command with an override flag.

Why dirty data is special

A write is acknowledged once it is durable on the cache device. The chunk is then dirty: it exists only on the cache device until the flush loop uploads it to the bucket. Anything that throws away a cache holding dirty chunks throws away those writes.

Watch flexblock_cache_dirty_slots and flexblock_discard_deletes_pending. When both read 0, the bucket holds everything, and the cache can be discarded safely.

Three flags that discard dirty data

Three commands discard a cache’s dirty data. All three require a flag as confirmation instead of prompting, so that a script cannot answer a prompt by accident.

Data loss

detach --allow-dirty-loss. Abandons dirty slots, pending discards (whose deleted data may come back) and an unclean stop. Prefer start followed by a clean stop, which flushes everything, and then a plain detach.

Data loss

attach --overwrite. On a device that holds a cache for a different volume, or for this volume after another host has owned it, discards that cache. A cache still in step with this volume is preserved.

Data loss

init --attach <DEVICE> --overwrite. Zeroes whatever cache the device holds, dirty slots included.

Each of these commands accepts --dry-run, which prints what would happen without doing it.

Deleting the volume

Data loss

wipe is not reversible. wipe deletes every object under the prefix, the volume superblock, the ownership marker and the local cache. Without bucket versioning or a backup outside flexBlock, the data is gone. Preview with --dry-run, and check that the grant names the volume you mean. See Destroying a Volume.

Taking ownership

Data loss

attach --take-ownership. Taking ownership abandons whatever the previous owner had not flushed. That data stays on the old host’s cache device, invisible to the new owner, and the old cache can no longer be flushed. Take ownership only when the old host is gone; otherwise stop the daemon and detach there. See Ownership.

The write-back window

A failed cache device is the one hardware event that costs acknowledged writes: at most what flexblock_cache_oldest_dirty_age_seconds reported at the time.

This matters most on ephemeral instance storage, whose contents vanish when the instance stops or is replaced. The cache needs no durability of its own, since everything it holds is either already in the bucket or on its way there. What is at risk is only the write-back window:

• With the defaults, --flush-threshold holds it to about four minutes while uploads keep up with writes, and about eight for a chunk under continuous writes (see --flush-quiet).
• A write burst faster than the bucket absorbs, or a backlog recovered after a crash, stretches it until the backlog drains.

Not a point-in-time copy

The bucket is not a point-in-time copy of the volume. Chunks upload one at a time as they age, and a filesystem flush only reaches the cache device, so after an abrupt loss of the cache the bucket holds each chunk as it was last uploaded: recent writes to different chunks may be only partly present. Expect the filesystem to replay its journal or need a repair (xfs_repair, fsck), and verify databases before trusting them.

To shrink the window, lower --flush-threshold (at the price of more uploads), and alert when flexblock_cache_oldest_dirty_age_seconds stays high. See Write-Back Behavior and Alerting.

A replacement instance attaches the same volume, with --take-ownership if the old one could not detach, and serves it from a cold cache.

Things that look risky but are not

• A daemon crash. The kernel holds the device and its in-flight requests. Run start again; it replays the write-ahead log and resumes. See Recovering From a Crash.
• A shutdown that runs out of time. Dirty data not uploaded within --shutdown-flush-timeout, or left behind by a SIGKILL, stays on the cache device and the next start flushes it.
• A license expiry. Guest I/O is throttled, but flushing to object storage is not, and data can still be read out. See Licensing.

Do not reboot a host with a hung mount

If the daemon has died and a mounted filesystem is blocked, run start before anything else. A reboot discards the in-flight requests the kernel is holding. Likewise, --replace-ublk forfeits the reissue of in-flight requests; use it only when recovery is refused.

Related

• Common Issues
• Moving a Volume
• Stopping Cleanly