Blog‎ > ‎

Nexus vPC Overview

posted Nov 12, 2013, 10:23 AM by Rick McGee   [ updated Nov 12, 2013, 12:46 PM ]
Port Channel overview
    Port channels , EtherChannel, NIC teaming are all bonding terms that can be used interchangeably
Regardless of vendor 802.3ad (LACP) refers to port channeling
Port Channeling load balances between member links
    Single flow cannot exceed BW of any physical member
    Similar to a lanes on a highway where one car cannot drive 65 Mph in multiple lanes.
    Yon can have up to 4 flows at 1Gbps each going up 4 different member links in the port channel
    Flows are load balanced based on L2,L3, and L4 header information
    Source/Destination, VLAN, MAC IP, and TCP/UDP port
NX-OS default is Source/Destination for IPv4/v6 for Layer 3 packets and Source/Destination for MAC frames
Single vs Multi Chassis EtherChannel
    1 Down Stream Device and 2 upstream devices
    There are three types
                    1.) Catalyst 3xxx Cross Stackwise Port Channels (Has a single control plane)    
                    2.) Catalyst 65xx VSS (virtual switch system Can have L2/L3 over peer link) (Has a single control Plane)
                    3.) Nexus vPC (virtual port channel)
vPC use two separate control planes
    Configurations managed independently
    Separate control plane protocol instances
        STP, FHRP, IGP's, BGP, etc.
    Synchronizton via a Peer Link 
        Similar to the VSS's VSL
    vPC's are made up to two (2) physical switches
        These are the vPC peers
    vPC Peers each have
        vPC peer link
            - This sync's the control plane
        vPC Peer Keep alive
            - link Current Live status "ping" L3 connected peer
        vPC Member ports
            - To south bound facing links "servers" to switch destination
vPC Peer Link
    Layer 2 (this is important) link used to sync control plane between vPC peers
        CAM Table, ARP cache, IGMP Snooping DB, etc...
        Uses Cisco Fabric Service over Ethernet (CFSoE) protocol
        Used to elect vPC primary and vPC secondary roles
    Normally not used for DATA PLANE, it can become a bottleneck
vPC Peer Keepalive Link
    Layer 3 link used as a heartbeat in the control plane
        used to prevent active/active or "Split Brain" vPC roles
        Not used in the vPC DATA PLANE!
        Uses unicast UDP port 3200
Peer Keepalive link can be....
    Mgmt0 port
        Potential problem with Dual supervisors in the Nexus 7K
    Layer 3 routed link or port-channel
        back to back or over a routed infrastructure
        ideally in an isolated VRF
vPC Member Ports
    Data Plane port channel towards downstream neighbor
Each vPC peer has at least one member port per vPC
    Can be more, up to hardware platform limits
From the perspective of the downstream neighbor , the upstream vPC peers look as one switch
    Physical result is a triangle
    Logical result is a point-to-point Port-Channel with no STP blocking ports
vPC configuration
    Enable "feature vPC"
    Create vPC domain
    Configure the vPC peer Keepalive link
    Create the vPC peer link
    Move member port to a vPC
        Configurations must be consistent to avoid Type 1 and Type 2 Errors
N5 K vPC Configuration
    int port-channel 51
    switchport mode trunk
    Int eth 1/6,7,8,9
    switchport mode trunk
    spanning-tree port type network
    channel-group 51 mode active
N7K1 vPC configuration 
    feature vpc
    feature LACP
    vPC domain 1
        This must match between vPC peers
    vPC Keepalive destination
    #Peerllink this is a port-channel with 1/1-1/2
    int eth 1/1-1/2
    switchport mode trunk
    channel-group 1 mode active (active=LACP)
    no shut
    Int port-channel 1
    switchprot mode trunk
    spanning-tree port type network 
    vPC peer-link    
Some other useful commands
    No system default switch port shutdown
        This is a confusing command, it actually doesn't shut this feature off, but allows you to see if the interface is up/down when you show             the statistics for the interface
In order to for a vPC adjacency they have to be configured on the same type of module per switch
Show vPC
    ID        PORT        Status          Consistency             Reason            Active Van
    51        PO51         UP                Success                  Success                1
show vPC consistency parameter vPC10
switch# show vpc consistency-parameters interface po 10 

vPC consistency checks

    CFSoE run on the vPC peer link to sync up the control plane, this includes advertisement of "consistency parameters" that much match
    to form a vPC adjacencies
3 Types
    Type 1 Global and Interface consistency check
        A mismatch result in the vPC failing to form if new vPC
        A mismatch  results in VLAN"s being suspended if change in an active vPC
    Type 2
        Mismatch results in log message but not vPC failure
        can result in failures in the data plane
"graceful consistency-check" causes only the secondary peer to disable vPC in case of Type 1 mismatch
vPC loop prevention
    vPC check if packet comes in a peer link, and will not send out to member ports
    Exception is if a link fails and creates an "Orphan Port" in which the other peerlink has failed
 Nexus 5K's have a unique command to allow synching of configuration between two switches in a peerlink
    "config-sync" (this is not in the Nexus 7K's)
vPC's and FHRP's
    Nexus 7K's are typically the L2 and L3 network boundary
        N7K is vPC peer, but also end host's FHRP Default gateway
    FHRP behavior changes to accommodate active/active forwarding over vPC
        Traffic received in a vPC member port of FHRP standby to FHRP, virtual MAC is not forwarded over Peer Link to active FHRP                     member
        The HSRP standby acts as a HSRP active
    FHRP vPC can break in certain non-standard vendor applications
        Frames sent to FHRP standby with physical DST MAC of FHRP active are sent out the Peer Link.
        "peer-gateway" hack allows FHRP Standby to forward frames on behalf of the DST MAC of the FHRP active without going over             the Peer Link
    If I'm the standby switch I still listen to traffic
        Can form Active/Active forwarding
    3 MAC address forwarding
    1.) Active MAC
    2.) Standby MAC
    3.) Real MAC address
    Peer-Gateway allows the standby HSRP device to send traffic northbound without sending over the peer link
vPC failure Scenarios
    The worst case scenarios in vPC failure is "Split Brain"
        vPC control plane is broken and both vPC peers assume vPC primary role
    Peer Keepalive and peer link have built in protection against "Split Brain"
        Upon failure vPC secondary suspends it's local vPC member Port and SVI's
        Normally the desired behavior in Split Brain
        Can isolate Orphan Port that use vPC secondary's SVI as their DFGW
            Only surefire work around is not to have Orphan Ports
            vPC Peers ideally only have vPC member port, e.g. all downstream devices are dual attached
vPC Peer Switch
    Makes vPC peers appear as the same root bridge
    Same BPDU's generated by both the Primary and Secondary vPC Peers
    Useful in failure scenarios to reduce RSTP reconvergence time when vPC Primary fails and then recovers
            With Peer Switch, secondary vPC peer doesn't need to run RSTP Sync when primary comes back
vPC and Multicast
    When source is reachable via a vPC member port, both vPC peers act a PIM DR
        Called "Dual DR" or "Proxy DR"
    Allows either vPC Primary or Secondary to receive traffic from source and forward it north without having to cross the vPC peer link
        Respects vPC check rule
    When the receiver is reachable via a vPC member port
        IGMP reports synchronized over Peer Link
        Normally Assert winner forwards traffic
            In vPC, Primary peer forwards towards Peer Link and vPC member ports
            In the vPC secondary traffic via Peer Link is forwarded to Orphan Ports but not vPC members
        Peer Link is used in the data plane for Multicast
            Bandwidth budget of Peer Link must be adjusted accordingly
IGMP Snooping is sync'd over vPC Peer Link with CFSoE
Peer Link is used for Data Plane in multicast if member ports require multicast traffic
Look at and search for