FC vs iSCSI SAN: Choosing the Right Storage Protocol

When you design a storage network, the first fork in the road is FC or iSCSI. Fibre Channel has been the enterprise SAN standard for two decades with dedicated switches and guaranteed performance; iSCSI runs storage traffic over standard Ethernet, cutting cost at the expense of tuning effort. Both are alive and well in 2026 - all-flash arrays like NetApp AFF, Dell PowerStore and HPE Alletra support both. This article compares them honestly across the metrics that matter - speed, latency, cost, cabling, skills and management - and gives you a decision framework instead of a religious debate.

Protocol Fundamentals

• FC: dedicated Fibre Channel fabric, SCSI commands over FC frames, own addressing (WWNs), own zoning and LUN masking model.
• iSCSI: SCSI commands encapsulated in TCP/IP over Ethernet, uses IP addresses, VLANs and standard switches.

Under the marketing-level summary there is a real architectural difference. Fibre Channel is a purpose-built storage transport with its own five-layer stack (FC-0 through FC-4). Layer FC-0 defines the physical media and signalling, FC-1 the 8b/10b or 64b/66b encoding and frame delimiting, FC-2 the framing and flow control, FC-3 the common services, and FC-4 the upper-layer protocol mapping - which is where SCSI lives. Because the fabric is closed and lossless by construction, it does not need a retransmission mechanism; a frame that arrives corrupted is simply discarded, and buffer-to-buffer credits guarantee the sender never overruns the receiver.

iSCSI is the opposite philosophy. It takes the same SCSI command set and pushes it into a TCP stream over IP. That means it inherits everything TCP provides - sequencing, retransmission, congestion control - and everything TCP costs: extra headers, acknowledgement traffic, per-packet processing, and the possibility of retransmission storms when a congested switch drops a frame. Each protocol data unit (PDU) has to be built, checksummed and reassembled, either on the host CPU or on a dedicated offload engine.

Identity works differently too. FC names endpoints with 64-bit World Wide Names, and you zone by WWN pairs or by a WWN alias. iSCSI uses IQN names of the form iqn.2026-01.com.example:host01 and authenticates with CHAP. The mental model is the same - "which initiator may see which target" - but the objects you manipulate are different, and that is where operational knowledge lives.

Performance and Latency Comparison

On modern all-flash arrays with NVMe, both protocols reach tens of thousands of IOPS; FC Gen 6/7 retains an edge in consistent low latency, while 100G iSCSI can match or exceed older FC generations in raw throughput.

The numbers deserve to be read carefully because the headline bandwidth figure is the least interesting part of the comparison. A 32 GFC link carries roughly 3200 MB/s of usable throughput; a 25 GbE iSCSI link carries roughly 2800 MB/s after TCP and Ethernet overhead. Four paths of each give you comparable aggregate bandwidth. Where the protocols genuinely differ is in the tail of the latency distribution. A well-tuned FC fabric delivers a p99 latency that is very close to its median. An iSCSI path crossing a shared switch with an oversubscribed uplink can have a median of 400 microseconds and a p99 of 12 milliseconds, because a single dropped frame triggers a TCP retransmission and a congestion-window reduction. Database workloads care about the tail far more than the average, which is why "iSCSI is just as fast as FC" is true on average and misleading at the ninety-ninth percentile.

Queue depth and multipathing shape the result more than the transport. One iSCSI session is a single TCP connection and behaves like a single queue. Four sessions across four physically separate paths, combined with a host multipath driver in round-robin or queue-length mode, is what produces enterprise-grade IOPS. The same is true on FC, where a single HBA port still benefits enormously from multiple paths to the target array. In both cases, single-path storage is a performance and availability mistake.

Cost Comparison

• FC: FC switches (Brocade/Cisco) cost more per port, plus FC HBAs and optics. Small deployments rarely justify a dedicated fabric.
• iSCSI: reuses existing Ethernet switches (with care), standard NICs and copper/optical DACs; software initiators are free.
• Storage array license differences: many arrays price FC and iSCSI ports identically; the real cost is in the fabric.

A more honest cost model separates four buckets. First, host adapters: an FC HBA costs several times a dual-port 25 GbE NIC, and you need two per host for redundancy. Second, switching: FC switches are sold in fixed port counts at a high cost per port, while Ethernet ports are cheap and already present - although using them for storage may force a switch upgrade you would otherwise not have needed. Third, optics and cabling: FC optics have historically been expensive and vendor-coded; Ethernet offers a wide range of DAC, AOC and transceiver options at every price point. Fourth, and most often underestimated, people: FC skills are scarce, and a fabric that nobody on the team can troubleshoot becomes a dependency on external consultants.

A worked example with round numbers makes the trade-off concrete. A small deployment with four hosts and two arrays needs roughly eight HBAs, two FC switches with at least sixteen ports each, and sixteen optics - call it a substantial five-figure sum before any storage arrives. The iSCSI alternative reuses two existing ToR switches with free 10/25 GbE ports, eight standard NICs (often already installed), and four DAC cables, which can be an order of magnitude cheaper in hardware. The counter-argument is that the iSCSI design consumes switch capacity and ToR uplink bandwidth that the network team may want for something else, and that poorly isolated storage traffic can degrade both storage and user traffic. The cost is not zero, it is just relocated.

Management and Skills

• FC management: zoning (soft/hard), WWN-based LUN masking, fabric monitoring (Brocade Network Advisor / Cisco DCNM).
• iSCSI management: VLANs, IP SAN subnets, CHAP authentication, MPIO configuration on the host.
• Skills: FC expertise is scarcer and more expensive; iSCSI skills overlap with general networking.

Zoning and LUN masking are the two FC concepts a newcomer must internalise. Zoning controls which ports may exchange traffic at the fabric level; LUN masking controls which initiator may see which LUN at the array level. Zone by initiator-target pairs (single-initiator, single-target) rather than by a broad alias, because a wide zone means every host can see every target and a misbehaving HBA can disrupt the whole fabric. On the iSCSI side the equivalent discipline is putting storage traffic in its own VLAN, using a dedicated subnet, enabling CHAP (and even better, IPsec for confidentiality if the traffic crosses untrusted links), and configuring the host's multipath layer deliberately. A typical Linux multipath configuration tunes path-checking, failback and queuing policy - see Linux multipath configuration: multipath.conf for iSCSI SAN for the parameter-by-parameter explanation.

Decision Guide

• Choose FC when: mission-critical databases, lowest consistent latency, dedicated storage team, existing FC fabric, regulatory environments that expect FC.
• Choose iSCSI when: budget-sensitive, small/medium environments, converged networks, staff already strong in Ethernet, ROBO/edge sites.
• Many enterprises run both: FC for primary databases, iSCSI for backups, file services and DR replication traffic.

The mixed model is the most common real-world answer, and it is not a compromise so much as a rational split by workload class. Tier-1 databases with hard latency SLOs stay on FC. Backup staging, Veeam repositories, dev/test clusters and DR replication - all bandwidth-sensitive but latency-tolerant - run on iSCSI, where the cost per gigabyte of capacity is lowest. This also means a single team must be fluent in both, which happens naturally once the network engineers own the iSCSI side and the storage engineers own the FC side, with a clear boundary at the array's port layer.

Practical Considerations

• iSCSI over shared Ethernet: isolate storage VLANs, enable PFC/ETS if the switch supports DCB, avoid oversubscribed ToR switches for latency-critical workloads.
• Always configure MPIO (multipath) for iSCSI - single-path iSCSI is a SPOF and performs poorly.
• FC Gen 7 (64G) and NVMe-oF (FC-NVMe) keep FC relevant for all-flash and NVMe arrays.

A specific warning about iSCSI on converged Ethernet: enabling PFC is not a one-line change. Lossless behaviour requires a coordinated configuration across the whole path - host NIC, server switch, uplink, core, and array port - plus a consistent priority mapping. A PFC configuration applied on one switch only makes things worse, not better, because the traffic is now paused at one hop while the next hop keeps sending. If your Ethernet infrastructure cannot be made end-to-end lossless with a change-control process behind it, do not chase lossless iSCSI: keep storage traffic on dedicated switches or accept a slightly higher latency tail. The same reasoning applies to RoCEv2, which solves a similar problem over the same fabric and is covered in depth in RoCEv2 lossless tuning: PFC, ECN and DCQCN.

Where NVMe-oF Fits

NVMe over Fabrics changes the conversation because it replaces the SCSI command set rather than just the transport. SCSI was designed for spinning disks and a queue depth of a few dozen commands per device; NVMe supports thousands of queues with thousands of commands each. An all-flash array fronted by SCSI - whether over FC or iSCSI - leaves protocol capability on the table. NVMe-oF addresses that, and it comes in two flavours relevant here: FC-NVMe, which runs NVMe over the existing Fibre Channel fabric with no changes to the switches, and NVMe/TCP, which runs NVMe over ordinary TCP and therefore competes directly with iSCSI on standard Ethernet.

The practical consequence is that the FC-versus-iSCSI decision is now also a decision about which NVMe-oF path you can reach later. An FC fabric can be upgraded to FC-NVMe with the same switches and cabling, which is a genuine argument for keeping FC in latency-critical environments. An iSCSI deployment can be migrated to NVMe/TCP, which reuses the same IP infrastructure and often delivers a noticeable latency improvement with no hardware change. Setup detail for the latter is in NVMe over TCP: target and initiator setup guide.

Security Compared

FC is inherently harder to attack from a random compromised workstation because there is no routing path to it. iSCSI, running on IP, inherits IP's exposure, so the isolation discipline matters. Mutual CHAP plus a dedicated storage VLAN plus no default gateway on the storage subnet is a well-understood baseline. Note that IPsec for iSCSI adds latency and is rarely used inside a controlled data centre; it is mostly relevant for replication over WAN or metro links.

How to Decide Without Guessing

If the two designs look close on paper, benchmark before committing. A defensible process is:

1. Define the workload: block size, read/write ratio, queue depth, and the latency SLO (median and p99, not just average).
2. Build a small test: one host, two paths of each protocol, the same array and the same LUN characteristics.
3. Measure with a tool that reports latency percentiles - fio with --latency_percentiles, or a vendor I/O generator.
4. Run the test twice: once on an idle fabric and once with realistic background traffic, because tail latency only shows up under contention.
5. Compare cost per IOPS and cost per gigabyte, including the switch ports and optics, not just the HBAs.
6. Write down the operational cost: who will support it at 3 a.m., and do they have the skills today?

fio --name=randread --ioengine=libaio --direct=1 --rw=randread \
    --bs=4k --iodepth=32 --numjobs=4 --runtime=300 --time_based \
    --filename=/dev/mapper/mpatha --latency_percentiles=99

Common Deployment Mistakes

• Single-path iSCSI. One NIC, one switch, one cable. Every failure is an outage and the throughput is a fraction of what the array can do.
• Storage on the management VLAN. Convenient during installation, disastrous afterwards: a broadcast storm from a user device can stall storage I/O.
• Wide-open FC zones. One big zone containing every host and every target removes the isolation that makes FC safe.
• Ignoring switch oversubscription. A ToR switch with a 3:1 uplink ratio is fine for user traffic and terrible for a latency-sensitive storage flow that happens to leave the rack.
• Mixing LUN masking and zoning responsibilities. If the array lets an initiator see a LUN, no zone will save you - both layers need to be correct.
• No monitoring of the fabric. Port-level CRC errors, credit-starving ports and SFP degradation are visible long before they become outages, but only if someone collects the counters.

Conclusion

FC and iSCSI are not competitors so much as two points on a cost-versus-predictability curve. Choose FC when consistent low-latency matters more than anything else and a dedicated fabric is affordable; choose iSCSI when budget, existing Ethernet skills and operational simplicity dominate; and do not feel obliged to pick one when the workload mix clearly wants both. Whichever you choose, the design discipline is the same: multiple paths, isolated traffic, correct masking, and monitoring that runs before the first incident rather than after it. For a three-way view that includes NVMe-oF, see FC vs iSCSI vs NVMe-oF: choosing a storage network protocol. Array-side detail is in Dell PowerStore best practices and NetApp ONTAP network configuration best practices.

原文链接：https://www.netapp.com/blog/