NVMe-oF/TCP Export

A flexBlock device can be published as an NVMe namespace through the Linux kernel’s nvmet target, so another host attaches it with the standard nvme CLI. The initiator sees an ordinary NVMe namespace with the volume’s logical block size, and can format and mount it as usual.

This example exports /dev/ublkb0, served by flexblock@nvme1n1p1.service from the systemd page.

Caution

Do not also mount the device locally while it is exported.

The export unit (target host)

The export unit depends on the flexBlock service the same way a mount does. That guarantees the namespace is removed before the daemon is signaled, which matters because the daemon cannot see an NVMe-oF export and would otherwise stop under it.

Install as /etc/systemd/system/nvmet-analytics.service:

[Unit]
Description=NVMe-oF/TCP export of the flexBlock analytics volume
Requires=flexblock@nvme1n1p1.service
After=flexblock@nvme1n1p1.service network-online.target
Wants=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
Environment=NVMET=/sys/kernel/config/nvmet
Environment=NQN=nqn.2026-09.com.example:flexblock-analytics
Environment=SERIAL=FLEXBLOCK-ANALYTICS1
Environment=NSUUID=6f1c2a4e-3b7d-4c5e-9a8f-1d2e3f4a5b6c
ExecStartPre=/sbin/modprobe nvmet-tcp
ExecStart=/bin/sh -ec '\
    mkdir -p $NVMET/subsystems/$NQN/namespaces/1; \
    echo $SERIAL > $NVMET/subsystems/$NQN/attr_serial; \
    echo $NSUUID > $NVMET/subsystems/$NQN/namespaces/1/device_uuid; \
    echo 1 > $NVMET/subsystems/$NQN/attr_allow_any_host; \
    echo /dev/ublkb0 > $NVMET/subsystems/$NQN/namespaces/1/device_path; \
    echo 1 > $NVMET/subsystems/$NQN/namespaces/1/enable; \
    mkdir -p $NVMET/ports/1; \
    echo tcp > $NVMET/ports/1/addr_trtype; \
    echo ipv4 > $NVMET/ports/1/addr_adrfam; \
    echo 0.0.0.0 > $NVMET/ports/1/addr_traddr; \
    echo 4420 > $NVMET/ports/1/addr_trsvcid; \
    ln -sfn $NVMET/subsystems/$NQN $NVMET/ports/1/subsystems/$NQN'
ExecStop=/bin/sh -c '\
    rm -f $NVMET/ports/1/subsystems/$NQN; \
    rmdir $NVMET/ports/1; \
    echo 0 > $NVMET/subsystems/$NQN/namespaces/1/enable; \
    rmdir $NVMET/subsystems/$NQN/namespaces/1; \
    rmdir $NVMET/subsystems/$NQN'

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now nvmet-analytics.service

If you prefer nvmetcli, save the same configuration with nvmetcli save and replace the two shell blocks with nvmetcli restore and nvmetcli clear, keeping the [Unit] dependencies.

Pinned serial and UUID

The serial number and namespace UUID are pinned on purpose. Left unset, the kernel generates new random ones every time the unit recreates the subsystem. An initiator that reconnects after the export restarts would then find a namespace whose identifiers changed, drop it, and fail every outstanding I/O.

With fixed identifiers, the same namespace comes back, and the initiator can address it by the stable path /dev/disk/by-id/nvme-uuid.<NSUUID> instead of an nvmeXnY name that changes with each reconnect.

Choose your own values: a UUID from uuidgen, and a serial of at most 20 characters, unique per exported volume.

Access control

Restrict who can connect

attr_allow_any_host set to 1 lets any initiator connect. On anything but an isolated storage network, leave it at 0 and register each initiator’s host NQN instead: create $NVMET/hosts/<hostnqn> and symlink it into the subsystem’s allowed_hosts directory. For the same reason, set addr_traddr to the storage-network address rather than 0.0.0.0.

Connecting (initiator host)

sudo modprobe nvme-tcp
sudo nvme discover -t tcp -a 192.0.2.10 -s 4420
sudo nvme connect -t tcp -a 192.0.2.10 -s 4420 -n nqn.2026-09.com.example:flexblock-analytics --ctrl-loss-tmo -1 --reconnect-delay 5
sudo nvme list
ls -l /dev/disk/by-id/nvme-uuid.6f1c2a4e-3b7d-4c5e-9a8f-1d2e3f4a5b6c

Why —ctrl-loss-tmo -1

--ctrl-loss-tmo -1 makes the initiator retry the connection indefinitely, every --reconnect-delay seconds. By default it gives up after 600 seconds and deletes the namespace, which fails every I/O on it.

Retrying forever matches how the flexBlock device itself behaves when its daemon is away: I/O waits, then completes. An export restart, a daemon crash on the target host or a network blip therefore shows up on the initiator as a stall, not as errors. Use a finite value only where an application is better served by errors than by waiting.

Reconnecting at boot

1. Put the same parameters in /etc/nvme/discovery.conf, which the nvmf-autoconnect.service that ships with nvme-cli reads:

   --transport=tcp --traddr=192.0.2.10 --trsvcid=4420 --ctrl-loss-tmo=-1 --reconnect-delay=5

2. Enable the service.

   sudo systemctl enable --now nvmf-autoconnect.service

3. Mount the namespace by its /dev/disk/by-id/nvme-uuid.* path with _netdev and nofail, so that the mount is ordered after the connection.

What the initiator sees

Event on the target	What the initiator sees
Stopping the flexBlock service	The export is removed first, which looks like a connection loss; the initiator keeps retrying. The daemon then flushes and stops. Starting the export again reconnects it.
Daemon crash	The export stays in place, and the initiator’s I/O waits for the restarted daemon.

Related

• Running Under systemd
• Recovering From a Crash
• Stopping Cleanly