Tons of terms that mean the same thing Unifed Fabric (Cisco) Unified Wire Converged Ethernet Converged Enhanced Ethernet Data Center Ethernet Data Center Bridging They are refer to the same infrastructure, Ethernet and Fibre Channel on the same wire  FCoE Terminology FCoE Initialization Protocol (FIP) Control Plane for FCoE Initiator and FCoE switch FCoE Forwarder (FCF) (Nexus 5K, 7K, or MDS with FCoE blade) Running the Ethernet and FC switching stack FIP Snooping Bridge (NPV don't run full FC stack) ENode FCoE node port (Ethernet attached) Virtual Fibre Channel (VFC) interface Logical interface presented to a Ethernet interface (like iSCSi interface) Where FIP run's between the ENode and VFC Virtual Port Types VN_Port Same as N port in FC VF_Port Same as F port in FC VE_Portn (Multi Hop FCoE) Same as E port in FC How FCoE Works FCoE replaces Layer 1 and Layer 2 transport for FC All upper layer FC services remain Domain ID's, FSPF, FCNS, FLOGI, Zoning, etc.... New FCoE Initialization Protocol (FIP) to negotiate between the Fabric and Node Fabric is the FCF Node is the ENode FIP vs FCoE FIP is the control plane of FCoE New EtherType 0x8914 Used to disvoer FCF's and perform FLOGI FCoE is the actual data plane New EtherType 0x8906 Max length of 2240 bytes FC-BB-5 T11 Org. Norman FC Payload is 2112 FC Framing----> FCoE Framing-----> Ethernet Implies Jumbo Frames are required FCoE Addressing Fibre Channel uses 3 byte FCID's Domain, Area, WWPN Ethernet uses a 6 byte MAC Addresses How to correlate the two? FCoE ENode gets a Fabric Provided MAC (FPMA) for FCoE Still uses it's regular MAC for LAN traffic Fabric Provided MAC Addresses (FPMA) During FIP ENode is assigned a 3 byte FCID (Domain, Area, and WWPN) FLOGI is part of FIP FCF is configured with a 3 byte FCoE MAC address Prefix (FC-MAP) ENodes appends FC-MAP to FCID 3 byte FC-MAP + 3 byte FCID = 6 byte FPMA Configure FCoE Create VSAN VSAN to a VLAN Configure a VFC Associate physical Ethernet FCoE Topology  Single Hop Topology N5K1 and N5K2 (assuming there is a basic config) feature fcoe feature lldp (this has a subset of DCB extensions ETS and PFC) conf t slot t port 25 - 32 type fc copy run start reload Want to have the FC up first before the FCoE configuration N5K1 conf t int fc1/27 - 28 no shut vsan database vsan 1010 zone mode enhanced vsan 1010 (make sure the trunking is working show int fc1/27 - 28 trunk vsan) vlan 1010 fcoe vsan 1010 int vfc117 bind interface e1/17 (make sure int e1/17 is shutdown) vsan database vsan 1010 interface vfc117 int e1/17 switchport mode trunk spanning-tree port type edge trunk switchport trunk native vlan 10 switchport trunk allowed vlan 10,1010 no shut show flogi database  This also shows that FCoE FIP (control plane worked) show int vfc 117  device-alias database device-alias name SERVER1_FCOE_SAN_A pwwn 10:00:00:00:c9:bb:19:9f device-alias commit show zoneset active vsan 1010  N5K2 conf t int fc1/29 no shut vsan database vsan 1020 zone mode enhanced vsan 1020 (make sure the trunking is working show int fc1/29 trunk vsan) vlan 20 interface e1/17 shutdown switchport mode trunk switchport trunk native vlan 20 switchport trunk allowed vlan 20,1020 spanning-tree port type edge trunk interface vfc117 bind interface ethernet1/17 switchport trunk allowed vsan 1020 no shut interface e1/17 no shut MDS1 zone mode enhanced vsan 1010 device-alias database device-alias name JBOD0_DISK13_SAN_A_PWWN 21:00:00:04:cf:f3:73:5F device-alias name JBOD0_DISK14_SAN_A_PWWN 21:00:00:04:cf:84:b7:9b device-alias name JBOD0_DISK15_SAN_A_PWWN 21:00:00:04:cf:84:48:77 device-alias commit vsan database vsan1010 vsan 1010 interface fc1/17 int fc1/1 -2 switchport rate-mode dedicated switchport mode e no shut zoneset name VSAN1010 vsan 1010 zone SERVER1_TO_RAID0 member device-alias JBOD0_DISK13_SAN_A member device-alias JBOD0_DISK14_SAN_A member device-alias JBOD0_DISK15_SAN_A member device-alias SERVER1_FCOE-SAN_A zoneset activate name VSAN1010 vsan 1010 zone commit vsan 1010 (you should now be able to see the active zoneset on N5K1) MDS2 zone mode enhanced vsan 1020 device-alias database device-alias name JBOD0_DISK13_SAN_B_PWWN 22:00:00:04:cf:f3:73:5F device-alias name JBOD0_DISK14_SAN_B_PWWN 22:00:00:04:cf:84:b7:9b device-alias name JBOD0_DISK15_SAN_B_PWWN 22:00:00:04:cf:84:48:77 device-alias commit vsan database vsan1020 vsan 1020 interface fc1/13 int fc1/5 switchport rate-mode dedicated switchport mode e no shut overall configuration feature fcoe feature lldp ! vsan databases vsan 1010 ! vlan 1010 fcoe vsan 1010 vsan 1010 interface vfc117 ! interface vfc117 bind interface e1/17 switchport trunk allowed vsan 1010 no shut ! ! interface e1/17 shutdown switchport mode trunk switchport trunk native vlan 10 switchport trunk allowed 10,1010 spanning-tree port type edge trunk FCoE status commands show fcoe  FC-MAP is the first 3 bytes 0e:fc:00 that will make up the FPMA QoS for FCOE  show policy-map type qos  On the ingress if it's FCOE (data plane) traffic it will be set to the qos-group to 1 show class-map type qos  show policy-map type queuing  Queuing FCOE is for 50 percent of the link. This is only enforced when congestion is present show class-map type queuing  show policy-map type network-qos  for DCBX the ETS is used for the % of bandwidth that can be used per queue and the PFC allows the Pause frame to be set. show interface ethernet 1/17 priority-flow-control This output will show you your Pause fames sent and received. If you are sending large amounts of pause frame it could mean that your Server hardware CPUnot fast enough and or an application issue. Hints FIP Gen1 CNA's are not support on Nexus 2232 FEX's show system internal dcvx info interface e1/17  Cisco Intel Nuovo (CIN) GEN 1 CNA |
Blog >