Blog‎ > ‎

CCIE DC N1Kv Installation

posted Jul 20, 2014, 11:43 AM by Rick McGee   [ updated Aug 1, 2014, 4:20 PM ]

Start by installing the VSM on your hosts

You can do this one of two ways
    OVA and or OVF File
    Or via an Installer App (Java Based)
The Installer App gives you a few choices, choose Cisco Nexus 1000V Complete Installation; with the custom option

A list of Prerequisites that need to be meet.

Enter the IP Address and user name and password for the vCenter server credentials, and at the lower left hand section you'll see a status indication.

Click on the browse radio button and it will open another windows where you can pick the host you want.

Same action for the Datastore, click the Browse button and choose the correct DataStore.

Choose the correct vSwtich in the same manner as the host and datastore.
For the second host you can also Key in the values if you know them without browsing. You'll also name the VDS and enter the credentials for the N1Kv, and finally choose the OVA file for the VSM.

The next section you have to choose Layer 2 (older way) and Layer 3 Connectivity. Layer 2 as you would think all VSM's and VEM's have to be on the same subnet. Layer 3 you could have VSM's in a main DC and the VEM's in another data center in different subnets. The Layer 3 option also gives you more trouble shooting advantages; such as ping and trace route.

Control Port Group used for 
    VEM to VSM Heartbeats
    VSM to VSM AIPC Traffic
    IGMP state information 
    This could be the same as Mgmt. VLAN or could be in a separate VLAN

    SNMP, Telnet, and SSH 

Click next, and it will validate the information we provided thus far regarding the hosts.
It will now give you the option to review the configuration before moving on. Click next...

You can now see that it's deploying the secondary VSM after already deploying successfully the Primary VSM.

You can see now that the Nexus 1000V plug-in is installed.

You can see that the N1000 V has added the VSM-MGMT switch to the standard switch layout.
The installation will go through the check list.

Final choices to add additional modules. Click Next... You can also add a VIB (vSphere Installation Bundle) file to add the VEM sat this time is you would like.

In this section you can migrate hosts over to the VSM, but it's better to do in manually. So you would just close the program.

You should be able to know log into the N1KV and issue the command "show module" and it will give you all the information that was just configured.

Here is the "show run" You can see it's only running the Essential's version (no DAI or DHCP Snooping SGT, SGACL's)
Here you see the SVS connection status the remote IP address the the Vcenter IP address and the vmware-vim protocol

Next you would need to deploy the VEM's (virtual ethernet modules)

On the CLI on the N1Kv

conf t
vlan 4093
name MGMT
vlan 110
name VM-Guests
port-profile type ethernet VM-Sys-Uplink
vmmware port-group
no shut
switchport mode trunk
switchport trunk allowed vlan 1,4093
system vlan 1,4093
state enable d

vlan 115
name vMotion 
port-pr type ethernet Vmotion-Uplink 
vmware port-group
no shut 
switchport mode trunk
switchport trunk allowed vlan 115
system vlan 115
state enabled

port-profile type ethernet VM-Guests
vmware port-group
no shut
switchport mode trunk
switchport trunk allowed vlan 110
state enabled

copy run start (this will save to both VSM's)

As you can see from the above example, the changes we made from the N1Kv is now showing up in Vcenter.

Know you can add a host by right clicking on the N1K and click Add Host

In this example we are moving every other vmnic from the standard vswtich to the N1Kv VDS and choosing the uplink port from the port groups we created earlier.

It should look the the following, when you click next it will migrate these selected vmnics to the N1Kv VDS.

HINT: Make sure you don't overlap VLAN's for the ethernet port profiles, VMWare doesn't care for those.

This is a warning that while migrating the host ports vmnics might look connectivity.

Here is a graphical layout of the N1Kv VDS and the ports.

This example is showing the VUM (VMWare Update Mgr.) installing the VEM's onto the hosts.

You can now see the hosts under the DVS (distributed virtual switch), one has an error relating to no adapter redundancy on host 2.

You have to add the vmnics to the N1Kv DVS.
Click on Manage Physical Adapters (These are the Eth interfaces in the N1Kv) 

Click to add NIC's for the VM-Sys-Uplink

Add the vmnic1 to migrate to the N1Kv DVS. And choose the vmnic 03 for the Vmotion, and vmnic 05 for vm-guest
It will ask you if you really want to move it from the Vswitch to the NK1V-01 DVS. Hit Yes!....

It will look similar to this output, the VM-Guest is grayed out because there are no Guests that are registered wit the VM-Guest vmnic/vlan 110 created yet.
You would now do the same for the second host.

You know have to create the veth port-profiles.

conf t
port-profile type vethernet VMKernel
vwmare port-group
no shut
switchport mode access
swtichport access vlan 1
system vlan 1
state enabled
You'll now see that VMKernel show up under DVS in VCENTER

Back on the N1KV-01 Switch
Issue the command

config t

capability l3control

You'll see this warning, to make sure the system vlan is the same for the port-profle.
This will look for the VEM heartbeats and encapsulated in UDP 4785 and send it to the mgmt. IP address

You can now create the other vEth port-profiles

conf t
port-profile type vethernet VSM-MGMT
vwmare port-group
no shut
switchport mode access
swtichport access vlan 4093
system vlan 4093
state enabled

port-profile type vethernet vMotion
vwmare port-group
no shut
switchport mode access
swtichport access vlan 115
system vlan 115
state enabled

port-profile type vethernet VM-Guest-110
vwmare port-group
no shut
switchport mode access
swtichport access vlan 110
system vlan 110
state enabled

You can see all the veth port show up in the VCENTER N1Kv-01 DVS

Now we can move the vEth to the Eth through the N1Kv DVS

Click on Add

Click on Migrate existing virtual adapters.

You can do one at a time or both at the same time, you just pick the Port Group that we created earlier.

You'll see that the VMKernel vEth is associated with VM-Sys-Uplink Eth port. Click Finish

You'll see Mod 3 show up in the N1Kv switch from the show module command. You can also see the ESXi version and the IP address for the server, as well as the Server UUID. 
The N1Kv uses the UUID to distinguish the VEM, so even if the VEM was to shut down it would and then brought back up it would still use the same VEM 3 designation.