Blog‎ > ‎

UCS Architecture

posted Dec 16, 2013, 3:19 PM by Rick McGee   [ updated Jun 11, 2014, 6:48 PM ]
UCS Hardware Options
  • Fabric Interconnects
  • UCS B-Series Blade Chassis
    • IOM FEX
    • Blade Servers
      • CPU/RAM/HDD
      • Mezzanine Cards
  • UCS C-Series Server
    • Mezzanine Card
Physical Architecture of the UCS

  • Fabric Interconnects (FI's)
    • Core of the UCS platform
    • Everything connects to the FI's
    • FI's run the actual UCS manager software
  • Chassis
    • Chassis contains blades, but no intelligence
    • Blades contain CPU/RAM/CNA's
  • IOM
    • IO muxes data from the FI's to the blades
    • CMS (Chassis Management Switch)
      • Carries Management traffic to/from the CIMC on FI
    • CMC (Chassis Management Controller)
      • Monitors all sensors, voltage, controls fan speeds
LAN/SAN Connectivity - Fabric Interconnects
  • 6120,6140
    • Older Models
    • 3.2us latency
    • 1/10G Ethernet only
    • GEM (Generic Expansion Module) to support FC
  • 6248/6296 Unified Ports
    • Current Models
    • 2.0us latency (1200ns faster)
    • 1/10GE or 1/2/4/8 FC
  • 2104XP IOM/FEX
    • Older model
    • .8us latency
    • 4 10GE ports Northbound to FI's
    • 8 10GE port Southbound to Blades
    • Since two IOM's per chassis, means
      • 80Gbps Chassis BW Northbound to FI's
      • 2 10GE port per half-width blade (1 per IOM)
  • 2204XP IOM/FEX
    • Current model
    • .5us latency
    • 4 10GE port Northbound to FI's
    • 16 10GE port Southbound to blades
    • Support Port Channeling traces up to IOM/FEX
    • Since 2 IOM's per chassis, means
      • 80Gbps Chassis BW Northbound to FI's
      • 4 10GE port per half-width blade (2 per IOM)
  • 2208XP IOM/FEX
    • Current model
    • .5us latency
    • 8 10GE port Northbound to FI's
    • 32 10GE port Southbound to blades
    • Support Port Channeling traces up to IOM/FEX
    • Since 2 IOM's per chassis, means
      • 160Gbps Chassis BW Northbound to FI's
      • 8 10GE port per half-width blade (2 per IOM)
  • Blade Servers and Mezzanine Adapters
    • Various models and benefits
    • Half and Full-Width blades
    • Mezzanine Adapters
      • Emulex/QLogic
      • Palo/VIC (support Port Channels to the IOM)

  • UCS C-Series Server
    • Can be managed standalone
      • Can use Adapter-FEX with N5K in the mode
      • P81 or VIC1225
      • Creates vEth and vFC interfaces on the N5k
    • Can be managed by UCSM
      • LOM and SFP on C-Server plug into N232PP* as their IOM/FEX
        • v2.0 uses N2232PP , v1.4 used N2248TP
      • N2232PP Uplinks plug into FI's that must be in EHM and ports defined as "Server"
      • Must be redundant to begin with, otherwise won't show up
  • Extended Memory Architecture
    • Cisco create a custom ASIC to address electrical limitation of Intel CPU's in addressing more than 12 DIMM's of memory per socket
      • 12 DIMM's at full performance or 18 at reduced performance (~80% of normal speed)
    • Allows 4 DIMM's to appear as on logical DIMM to CPU. Allowing up to 4x memory architecture of traditional servers 

    • UCS Configuration Options
      • UCS Manager XML API
        • This is the actual UCS manger system/subystem
        • Gives UCS it's complete flexibility to be automated and remotely controlled
        • UCS GUI (HTTPS) and UCS CLI (SSH) are both fronts to this interface, both modify XML API, doesn't make any changes directly to DB
        • UCS CLI changes make to UCSM on either FI-A or FI-B both make changes via the API, which are putin the transactor gueu, processed in a FIFI manner, and written to the management information tree, which writed to the DB, this way it doesn't matter which FI you make the changes to
        • Not like a Supervisor on a C6K or N7K where you can only make changes on the primary, and not make any changes on the Secondary supervisor

  • GUI
    • Simplest, most common management tool
    • What most people think of when they think of UCS manager
  • CLI
  • SNMP (RO)
  • CIM-XML (RO)
    • System Even Log, Onboard Failure Log, View Alarms, Power Control
    • Port 5988
    • CIM= Comman Information Model
    • Standard defined by DMTF ( Distributed Management Task Force)
    • Alternative to Intelligent Platform Management Interface (IPMI)
    • SMASH= Systems Management Architecture for Server Hardware
    • CLP= Command Line Protocol
    • Alternative to IPMI with richer command set
    • Industry Standard maintained by the DMTF
  • Multiple Shells to the UCS Platform CLI
    • UCS Manager Shell
      • Not IOS-like
      • Navigate through equipment hierarchy with Scope, Where, Up, and Top
      • Create, Modify, Delete
        • Service Profiles and Template, LAN and SAN Policies, Backup and Import jobs
        • Local and remote user authentication
        • Show Fault and Logging data
      • Local-mgmt Shell
        • IOS-Like
        • Reboot FI's (your only method of doing so)
        • Ping and Traceroute
        • Manage flash  
  • Interface Types/Names In UCS 
    • Ethernet Uplink: goes from the FI north to neighbor switch
    • FC Uplink: From FI north to FC NPIV mode switch
    • Server: FI southbound to UCS IOM/FEX in chassis
    • Appliance: FI northbound direct to NAS/NFC Filer- if attached in small environments
    • Storage; FI northbound direct to a SAN arrary - if attached in a small environments 
    • Fabric: IOM/FEX northbound to FI
    • Backplane: IOM/FEX southbound to blade adapter
    • DCE: from adpter to IOM/FEX
    • Adapter/vNIC: presented from Adapter south to OS
  • Power Modes in Cisco UCS
When describing power supply configurations, “N” refers to the minimum number of supplies required to deliver
power without any redundancy

  • Non-Redundant (N)
     In Non-redundant mode, the system may go down with the loss of any supply or power grid associated with any particular
     chassis. To operate in non-redundant mode, each chassis should have at least two power supplies installed.
  • Redundant (N+1)
         In N+1 mode, the chassis tolerates the failure of any single supply without any interruption of service. In this mode, the chassis
            should have at least three supplies installed. In other words, N+1 describes a power supply configuration that has one extra
            power supply installed and active
  • Grid (N+N)
         The purpose of the grid redundant mode is to enable a configuration that can tolerate the loss of either a power supply or a input
          power circuit. In grid-redundant mode the system can withstand the loss of any two power supplies. In this mode, the chassis
          should have at least four supplies installed
  •  Power Cap Groups
      Allow higher-value servers/blades to continue to run while others are forced to power down,should there be a decrease in the overall power necessary to deliver adequate power to         all blades, and can help with data center cooling efforts
      Example 1-10, 1 being the highest priority

  • UCS Full backup and FI interaction
    • Two main type of Backups
      • Full-State-Backup: Which is binary and stored as a tarball (.tar)
      • Config Backup: Which is stored as a XML file
        • System- Backs up only system info
        • Logical - Backs up only logical info (service profiles, LAN, SAN, etc.)
      • All - Backs up both
      • Pay attention to "PResere Identities" with either a Logical or a All Config backup. "All config with Preserve Identities" is the perfect backup before performing a firmware upgrade (Full-State is not, simply because it includes firmware information, something you are upgrading, and don't wish to preserve)
    • All Backups aremanually created and exported to a remote server
      • Scheduling comes in v2.1
    • Restore
      • Full-State can be applied without any prior configuration
      • Config is applied after initial configuration is completed