HBA Queue Depth and SAN Multipath Tuning

Storage performance complaints that arrive as "the database is slow" are frequently a host-side queueing problem. A QLogic HBA defaulting to a shallow queue depth, or a multipath configuration that stops queueing as soon as one path drops, will throttle a perfectly healthy array. This guide covers the two host-side knobs that matter most for ONTAP-backed SAN volumes — HBA queue depth and multipath failover behaviour — and how to make the changes survive a reboot.

Why queue depth is the first thing to check

Queue depth defines how many I/O requests a host adapter can have outstanding to the storage. Too shallow and the array never sees enough concurrency to reach its potential IOPS; the latency curve stays flat and throughput caps out early. Too deep and excess commands sit in device queues adding latency without adding throughput. The right value depends on the array, the number of LUNs and how the workload is shared across paths.

# Find the queue depth currently in use
cat /sys/module/qla2xxx/parameters/ql2xmaxqdepth
cat /sys/class/scsi_host/host*/host/../device/queue_depth 2>/dev/null
lsscsi -l | head

# Set it for the current boot (example: 64 per path)
modprobe -r qla2xxx
modprobe qla2xxx ql2xmaxqdepth=64

NetApp's documented process for ONTAP SAN hosts is explicit that only the ql2xmaxqdepth parameter should be modified, and that after changing it you must regenerate the initramfs and reboot the host for the setting to persist.

Making it persistent

# RHEL / Oracle Linux / Rocky
echo "options qla2xxx ql2xmaxqdepth=64" >/etc/modprobe.d/qla2xxx.conf
dracut -f -v                        # ALTLinux/Debian: update-initramfs -u -k all
reboot

# Verify after reboot
cat /sys/module/qla2xxx/parameters/ql2xmaxqdepth
systool -m qla2xxx -v | grep -i ql2xmaxqdepth

Raising queue depth multiplies outstanding I/O per path, so validate with real measurements rather than by feel:

# Per-device queue and latency visibility
iostat -x 5
cat /sys/block/sdX/queue/nr_requests
cat /proc/interrupts | grep -i qla          # interrupt distribution across CPUs

# Application-level proof
fio --name=qd --rw=randread --bs=8k --iodepth=32 --runtime=60 --filename=/dev/mapper/mpathX

Multipath: keep queueing when a path dies

The second half of the tuning problem is what the host does when a path disappears. If multipath stops queueing IO as soon as the last path fails, applications see immediate EIO errors — which is fine for a test lab and catastrophic for a production database failover that takes 45 seconds.

# /etc/multipath.conf
defaults {
    user_friendly_names yes
    path_grouping_policy  group_by_prio
    path_selector         "service-time 0"
    failback              immediate
    no_path_retry         30            # queue for 30 polling cycles before failing IO
    # queue_if_no_path is implied by no_path_retry being set
    rr_weight             priorities
}

# ONTAP-specific tuning block often used on Linux hosts
device {
    vendor                "NETAPP"
    product               "LUN.*"
    path_checker          tur
    path_selector         "service-time 0"
    hardware_handler      "1 alua"
    prio                  alua
    no_path_retry         queue
}

systemctl reload multipathd
multipath -ll                  # shows path groups, priorities and current policy
multipathd show config | grep no_path_retry

A repeatable host-side checklist

1. Record the baseline: queue depth, multipath -ll path groups, IOPS and latency at a known load.
2. Confirm both paths to each LUN are visible and belong to the expected ALUA groups (alua prio, active/optimized vs active/non-optimized).
3. Increase queue depth in measured steps (16 → 32 → 64) and re-measure; stop when latency rises faster than IOPS.
4. Set no_path_retry so a transient path loss becomes a queue instead of an application error.
5. Regenerate initramfs and reboot once — then verify the parameters actually survived, because a tuning change that silently reverts is worse than none.

Nothing here is vendor-specific magic: it is the same loop of measure, change one parameter, measure again, persist. The difference between a SAN that behaves and one that does not is usually these two settings, not the array.

Related Reading on This Site

• 华为交换机恢复出厂设置 3 种方法详解（按钮/CLI/BootROM）
• FC vs iSCSI vs NVMe-oF: Choosing a Storage Network Protocol
• Linux Multipath Configuration: multipath.conf for iSCSI SAN

原文链接：https://docs.netapp.com/us-en/ontap/san-config/set-queue-depth-task.html (NetApp ONTAP SAN configuration documentation)