posted Nov 19, 2013, 4:58 PM by Rick McGee
[
updated May 7, 2014, 10:44 AM
]
What is Fibre Channel? - From a high level, it replaces SCSI disk cable with a network
- Protocol stack primarily used to send SCSI commands over the SAN
- Technically you could run IP over FC, but main application is SAN
- RFC 2625 - IP and ARP over FC
- Standard "T11" per International Committee for Information Technology Standards
- FCOE is T11's FC-BB-5 standard
- Fibre Channel vs. OSI Stack
Fibre Channel Protocol (FCP) is analogous to TCP
Fibre Channel Topologies Fibre Channel Port Types Fibre Channel Addressing FC World Wide Names FC Identifiers Fibre Channel Routing FC doesn't use flooding to build topologies like Ethernet Fabric Shortest Path First (FSPF) is used to route the traffic between switches Same Dijkstra SPF as OSPF and IS-IS Node ID in the SPT is the FCID's Domain ID (Similar OSFP Router ID) Traffic is routed via lowest cost path between Domain ID's ECMP is supported for equal SPT branches Unequal cost load distribution is not supported
FSFP runs automatically as a Fabric Service
Fibre Channel Logins Ethernet network are connectionless Fibre Channel networks are connection oriented Fabric Registration has three parts Fabric Login (FLOGI) Node Port (N_Port) tells the switch's Fabric Port (F_Port) it wants to register Switch learns the WWNN and WWPN of Node Switch assings FCID to Node
Port Login (PLOGI) End-to-End login between Node Ports Initiator tells Target it wants to talk Reads/Writes Used for application such as end-to-end flow control
Process Login (PLRI)
Fibre Channel Name Server Fibre Channel Name Server (FCNS) is similar to ARP Cache Used to resolve the WWN (physical address) to the FCID (logical address) Like Principle Switch and FSPF, FCNS is a distributed fabric service that requires NO configuration
Zoning By default all initiators learn about all targets during the login process Servers mounting the wrong volumes can corrupt the data Zoning prevents this by limiting which resource an initiator can use Zoning is similar to ACL's in Ethernet and IP world Associate WWN's, FCID's, aliases, etc. to control who can talk to who
Like FCNS, Zoning is a distributed fabric service Controls which initiators can talk to which targets Zoning is REQUIRED, and is not optional
Virtual SAN's - Traditionally multiple SAN's were designed as physical separate networks
- i.e. SAN Islands
- Physical separation is costly in term of equipment, power, space, cooling, management, etc.
- VSAN's solve the isolation problem similar to how VLAN's segment broadcast domains
- Isolate the management and failure domain of the network
- Separate FLOG,FCNS, Zoning, Aliases, etc. per VSAN
- With VSAN's E Ports now become TE Ports
- Similar to 802.1Q trunks in Ethernet
SAN Port Channeling - Like Ethernet Port-Channeling, SAN PC's can be sued to aggregate the bandwidth of physical links
- Supports Port Channeling Protocol (PCP) for negotiation of links
- Similar to 802.3ad LACP in Ethernet
Soft vs. Hard Zoning Zone vs. Zoneset Zone is used to create a mapping between WWPN's, FCID, Aliases, Interface, Domain-ID, etc....
Zones are grouped together in a Zoneset Zoneset is applied to the VSAN and then activated
Full vs Active Zoneset Only one Zoneset per VSAN can be "Active" in the fabric at a time "Full" Zoneset is the one in the configuration "Active" Zoneset is the one being enforced in the Fabric By default only the Active Zoneset is advertised, no the full Zoneset
Zoning Configuration and Verification "show zone status vsan 1" "show zone" "show zone active" "show Zoneset" "show Zoneset active" "clear zone database vsan 1" "Zoneset distribute full vsan 1" "Zoneset distribute vsan 1"
FC Aliases Zoning based on WWPN is error-prone FC Aliases give user-friendly names to WWN's, FCIDs, etc.. Configured with "fcalias name" Can be advertised through Zoneset distribution
Basic vs Enhanced Zoning By default the Full Zoneset is local and the Active Zoneset is Fabric-Wide Order of operations errors can corrupt the Active Zoneset Enhanced Zoning" prevents this by "locking" the Fabric
Using Enhanced Zoning
FC Device Aliases Using Device Aliases
SAN Port Channels - Used to aggregate the bandwidth of physical links
- Ethernet PC's and SAN PC's use the SAME number space
- Created with the link level "channel-group 1"
- New Members added with link level "channel-group 1 force"
- Port Channeling Protocol (PCP) enabled with PC link level "channel mode active"
- "interface Port-Channel" in MDS
- "interface SAN-Port-Channel" in Nexus
- Verified as "show {SAN-}port-channel summary"
Fibre Channel Switching Review Fibre Channel Domain ID's Domain ID is first byte of the FCID Used to identify the Switch in the Fabric's SPT Implies that a hard limit of switches per Fabric would be 256 Scaling the Fabric requires fixing the Domain ID limitation
Node Port Virtualization (NVP) NPV fixes the Domain ID problem by removing the need for a switch to participate in Fabric Services Switches running NPV appear to the rest of the fabric as an end host Upstream facing links on the NPV switch is call the NP_Port
Node Port ID Virtualization (NPIV) Switch upstream of the NPV switch is the NPV core switch NPV core switch runs Node Port ID Virtualization (NPIV) Allows multiple FLOGI's and FCID assignments on it's F port facing downstream NPIV is also applicable in virtualization environments
NPV/NPIV Configuration |
|