Lab: NSX Manager deployment (Single site)

From Iwan
Jump to: navigation, search

Lab: NSX Manager deployment (Single site)

In this lab I am working with the following software and versions:

Software Version Filename
VMware vCenter Server Appliance 7.00U3G VMware-VCSA-all-7.0.3-20150588.iso
VMware ESXi Server 7.00U3F VMware-VMvisor-Installer-7.0U3f-20036589.x86_64.iso
VMware NSX 4.0.0.1 nsx-unified-appliance-4.0.0.1.0.20159694.ova
  1. Deploy three NSX Manager Nodes and make sure they operate in a cluster.
    1. Do not use any external load balancers.
  1. Add a valid license to NSX.

Untitled.png

The Steps


STEP 1» Deploy the first NSX Manager

The NSX Manager can be deployed using the vCenter Server or an ESXi host. I will describe both.

For the deployment the following configuration parameters are required:

Parameter Name Input
vSphere Cluster Management
VM Name Pod-120-NSXT-LM-1
Storage Management_vSAN
Network (Port Group) Management
System Root User Password VMware1!VMware1!
CLI "admin" User Password VMware1!VMware1!
CLI "audit" User Password VMware1!VMware1!
Hostname Pod-120-NSXT-LM-1
Rolename NSX Manager
Management Network IPv4 Address 10.203.120.12
Management Network Netmask 255.255.255.0
Default IPv4 Gateway 10.203.120.1
DNS Server list 10.203.0.5
Domain Search List sddc.lab
NTP Server List 10.203.0.5
Enable SSH yes
Allow root SSH logins yes

Deploying the first NSX Manager using vCenter Server

Let’s start with the vCenter Server.

Right-click the vSphere cluster where I want to deploy the NSX Manager to and select “Deploy OVF Template”.

Untitled%201.png

Select “Local File” and select the NSX OVA file that I downloaded from the VMware website.

Untitled%202.png

Specify a Virtual Machine Name.

Untitled%203.png

Specify the compute resource.

Untitled%204.png

Review the details.

Untitled%205.png

Specify a deployment size.

Untitled%206.png

Specify the storage.

Untitled%207.png

Specify the network.

Untitled%208.png

Specify the parameters as per this table.

Untitled%209.png

⚠️ In the customise template section there are some parameters that are not used for this deployment, I can just leave those blank and only use the parameters I specified above as these are required for a successful deployment.

Review the details and click on “Finish” to start the deployment.

Untitled%2010.png

In the “Recent Tasks” section in the vCenter Server I can track the deployment status.

Untitled%2011.png

When the deployment is finished I need to turn on the NSX Manager VM by right-clicking on the VM and select Power → Power On.

Untitled%2012.png

The NSX Manager VM is now powered on.

Untitled%2013.png

Deploying the first NSX Manager using ESXi Server

Another way to deploy the fist NSX Manager Node is to do the deployment on an ESXi host. Click on “Create/Register VM”

Untitled%2014.png

Select “Deploy a virtual machine from an OVF file”.

Untitled%2015.png

Specify a Virtual Machine Name and select the NSX OVA file that I downloaded from the VMware website.

Untitled%2016.png

Specify the storage.

Untitled%2017.png

Specify the network and specify a deployment size. Make sure the VM is powered on automatically.

Untitled%2018.png

Specify the parameters as per this table:

Untitled%2019.png

Review the details and click on “Finish” to start the deployment.

Untitled%2020.png

In the “Recent Tasks” section in the ESXi Server I can track the deployment status.

Untitled%2021.png

Verify if the VM is powered on.

Untitled%2022.png

Log in for the first time

Now that the first NSX Manager Node is deployed I can log in to the GUI.

Untitled%2023.png

Accept the End User Licence Agreement.

Untitled%2024.png

Specify if I want to join the “Customer Experience Improvement Program”.

Untitled%2025.png

NSX 4.0.0.1 has a “Welcome wizard” to point out where I can find what. Click on “Next”.

Untitled%2026.png

The wizard point out “Networking”.

Untitled%2027.png

The wizard point out “Security”.

Untitled%2028.png

The wizard point out “Plan & Troubleshoot”.

Untitled%2029.png

The wizard point out “System”. Click the box “Don’t show this again” and “Got it”.

Untitled%2030.png

Now that the first NSX Manager Node is deployed and I am logged in I will see a blue bar at the top to inform that a license needs to be added.

Untitled%2031.png

STEP 2» Add the NSX License

Product License key
VMware NSX 3M2U7-XXXXX-XXXXX-XXXXX-XXXXX

I can either click on the “Manage Licences” button in the blue top bar.

Untitled%2032.png

Or I can go to System → Settings → Licences and click on “Add License”.

Untitled%2033.png

Add the license.

Untitled%2034.png

Verify if the license is added.

Untitled%2035.png

STEP 3» Add a Compute Manager

To prepare ESXi hosts to be an NSX Host Transport Node I need to add a Compute Manager. This is also needed if I want to deploy NSX Edge Transport Nodes and additional NSX Manager Nodes (to form an NSX Manager Cluster” using the NSX GUI.

I add a compute manager by going to System → Configuration → Fabric → Compute Managers → and click on “Add Compute Manager”.

Untitled%2036.png

Use the following parameters to add the Compute Manager and click on “Add” when I am finished.

Parameter Name Input
Name pod-120-vcenter.sddc.lab
FQDN or IP Address pod-120-vcenter.sddc.lab
Username administrator@vsphere.local
Password VMware1!
Enable Trust Yes
Access Level Full Access

Untitled%2037.png

When I click “Save” I get a message to add the missing Fingerprint of the vCenter server. Click on “Add”.

Untitled%2038.png

After I clicked on add I will see that the add is “In Progress”.

Untitled%2039.png

Look for the “Registered” and “Up” status when the Compute Manager is added successfully.

Untitled%2040.png

Untitled%2041.png

STEP 4» Deploy the second and third NSX Manager to form an NSX Manager Cluster

To form an NSX Manager Cluster I need two additional NSX Manager Nodes. These Manager Nodes can be deployed manually and added to the cluster using some CLI commands, or I can deploy the NSX Manager Nodes from the first NSX Manager Node’s GUI (Compute Manager configuration is required for this).

Deploy the second and third NSX Manager to form an NSX Manager Cluster 〈using the CLI〉 | Using the NSX Manager standalone deployments

Let’s start with the manual deployment of the second and third NSX Manager Nodes. Just for demonstration purposes I will also deploy these new NSX Manager Nodes in a different IP range (network) then the first one.

Untitled%2042.png

The table below will contain all the configuration parameters that I will need to deploy the second and third NSX Manager Node manually.

Parameter Name Input Second Node Input Third Node
vSphere Cluster N/A N/A
VM Name Pod-120-NSXT-LM-2 Pod-120-NSXT-LM-3
Storage ESX-02-DS-01 ESX-02-DS-01
Network (Port Group) Lab-Routers Lab-Routers
System Root User Password VMware1!VMware1! VMware1!VMware1!
CLI "admin" User Password VMware1!VMware1! VMware1!VMware1!
CLI "audit" User Password VMware1!VMware1! VMware1!VMware1!
Hostname Pod-120-NSXT-LM-2 Pod-120-NSXT-LM-3
Rolename NSX Manager NSX Manager
Management Network IPv4 Address 10.203.0.11 10.203.0.12
Management Network Netmask 255.255.255.0 255.255.255.0
Default IPv4 Gateway 10.203.0.1 10.203.0.1
DNS Server list 10.203.0.5 10.203.0.5
Domain Search List sddc.lab sddc.lab
NTP Server List 10.203.0.5 10.203.0.5
Enable SSH yes yes
Allow root SSH logins yes yes

When I look at the NSX Home screen I will see the warning “3 node cluster recommended”.

When I click on “Deploy Nodes” it will take me to the Appliances screen.

Untitled%2043.png

I can also manually browse to the Appliances screen by browsing to System → Configuration → Appliances.

Untitled%2044.png

Notice that only one Node is listed.

Get information from the first NSX Manager Node

Make note of the IP address of the first NSX Manager Node. 

Pod-120-NSXT-LM-1> get interface eth0
Wed Aug 24 2022 UTC 13:28:29.070
Interface: eth0
  IPv4 Address:
    Address: 10.203.120.12/24
    MAC address: 00:0c:29:75:b4:14
    MTU: 1500
    Default gateway: 10.203.120.1
    Broadcast address: 10.203.120.255
  Link status: up
  Admin status: up
  RX packets: 104708
  RX bytes: 13665374
  RX errors: 0
  RX dropped: 0
  TX packets: 16258
  TX bytes: 26292049
  TX errors: 0
  TX dropped: 0
  TX collisions: 0

Pod-120-NSXT-LM-1>

Make note of the certificate thumbprint of the first NSX Manager Node. 

Pod-120-NSXT-LM-1> get certificate api thumbprint
Wed Aug 24 2022 UTC 13:31:23.839
bb0e0933b21052d8a29fac65d6a250877ed2cefb913033ccb962bd79bcae0a46

Pod-120-NSXT-LM-1>

Make note of the Cluster ID. 

Pod-120-NSXT-LM-1> get cluster config
Wed Aug 24 2022 UTC 13:31:38.069
Cluster Id: 7262902c-d10a-4c40-a1d4-d24f737ccfaa
Cluster Configuration Version: 0
Number of nodes in the cluster: 1

Node UUID: fd0e4d56-4c33-0e61-8bff-9d8a1375b414
Node Status: JOINED
    ENTITY                               UUID                                       IP ADDRESS        IPV6 ADDRESS
              PORT     FQDN
    HTTPS                                9834c769-e08c-48de-beb9-d2d2509e1d0d       10.203.120.12     -
              443      Pod-120-NSXT-LM-1.SDDC.Lab
    ASYNC_REPLICATOR                     f8da85a9-66dd-4566-ac32-bd9f99644b9c       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    CONTROLLER                           2b97a266-fa0d-4c38-b857-e88552aa5c13       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    CLUSTER_BOOT_MANAGER                 fbca4c2f-28c3-4ce3-8452-9dfe04a91272       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    CM-INVENTORY                         a18a55d1-8889-4352-912d-c340eb00a03d       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    DATASTORE                            7de9e729-c7ea-482c-8014-7a613fe13e8b       10.203.120.12     -
              9000     Pod-120-NSXT-LM-1.SDDC.Lab
    CORFU_NONCONFIG                      26c912bd-0fb4-4067-961d-737e34521744       10.203.120.12     -
              9040     Pod-120-NSXT-LM-1.SDDC.Lab
    IDPS_REPORTING                       a1a10066-1508-4923-a197-fff5fc3b759b       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    MESSAGING-MANAGER                    e6be3d88-5ebf-435e-8609-700929a427b6       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    MONITORING                           f9dd4b01-2581-4d12-b4b2-4b09938657c3       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    MANAGER                              4360cb53-c2b4-467a-8e24-e96b44a846ea       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    SITE_MANAGER                         f9373036-1629-452b-b9e9-6470f06243ce       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab
    UPGRADE_COORDINATOR                  c75bb26e-7151-46a9-a3e0-9235fccd05ac       10.203.120.12     -
              -        Pod-120-NSXT-LM-1.SDDC.Lab

Pod-120-NSXT-LM-1>

All the items I just made a note of are listed in the configuration parameters in the table below, and are required to join the new Nodes to the first Node.

Parameter Name Input
IP address 10.203.120.12
Cluster ID 7262902c-d10a-4c40-a1d4-d24f737ccfaa
User name admin
Password VMware1!VMware1!
Certificate thumbprint bb0e0933b21052d8a29fac65d6a250877ed2cefb913033ccb962bd79bcae0a46

Add the second NSX Manager Node

The deployment of the second / third NSX Manager Node can be done using the vCenter Server or ESXi Server deployments described earlier.

Join the second NSX Manager Node to the first Node (cluster). 

Pod-120-NSXT-LM-2> join 10.203.120.12 cluster-id 7262902c-d10a-4c40-a1d4-d24f737ccfaa username admin password VMware1!VMware1! thumbprint bb0e0933b21052d8a29fac65d6a250877ed2cefb913033ccb962bd79bcae0a46
Data on this node will be lost. Are you sure? (yes/no): yes
Join operation successful. Services are being restarted. Cluster may take some time to stabilize.

Pod-120-NSXT-LM-2>

After I have issued the join command I can look at the Appliances screen to track the progress. During the join I might see some strange outputs like the one, but patience is key.

Untitled%2045.png

Joining and cluster stabilising will take around 15 minutes to finish.

Untitled%2046.png

When the join is completed I will now have an NSX Manager Cluster with two NSX Manager Nodes.

I can issue the following CLI commands to verify is the status is “UP” for all the services.

Pod-120-NSXT-LM-1> get cluster status
Pod-120-NSXT-LM-2> get cluster status

Add the third NSX Manager Node

When the join is completed I will now have an NSX Manager Cluster with two NSX Manager Nodes.

Untitled%2046.png

Join the third NSX Manager Node to the first and second Nodes (cluster). 

Pod-120-NSXT-LM-3> join 10.203.120.12 cluster-id 7262902c-d10a-4c40-a1d4-d24f737ccfaa username admin password VMware1!VMware1! thumbprint bb0e0933b21052d8a29fac65d6a250877ed2cefb913033ccb962bd79bcae0a46
Data on this node will be lost. Are you sure? (yes/no): yes
Join operation successful. Services are being restarted. Cluster may take some time to stabilize.

Pod-120-NSXT-LM-3>

After I have issued the join command I can look at the Appliances screen to track the progress. During the join I might see some strange outputs like the one, but patience is key.

Untitled%2047.png

Joining and cluster stabilising will take around 15 minutes to finish.

Untitled%2048.png

When the join is completed I will now have an NSX Manager Cluster with three NSX Manager Nodes.

I can issue the following CLI commands to verify is the status is “UP” for all the services.

Pod-120-NSXT-LM-1> get cluster config
Pod-120-NSXT-LM-1> get cluster status
Pod-120-NSXT-LM-2> get cluster config
Pod-120-NSXT-LM-2> get cluster status
Pod-120-NSXT-LM-3> get cluster config
Pod-120-NSXT-LM-3> get cluster status

Deploy the second and third NSX Manager to form an NSX Manager Cluster - Using the NSX Manager GUI

When I do not want to deploy the NSX Manager Nodes manually I can deploy the NSX Manager Nodes from the first NSX Manager Node using the GUI (Compute Manager configuration is required for this).

When I look at the NSX Home screen I will see the warning “3 node cluster recommended”.

When I click on “Deploy Nodes” it will take me to the Appliances screen.

Untitled%2043.png

I can also manually browse to the Appliances screen by browsing to System → Configuration → Appliances.

Untitled%2049.png

Notice that only one Node is listed.

Again for demonstration purposes I will now deploy these new NSX Manager Nodes in the same IP range (network) then the first one. I will need this later when I want to configure a Virtual IP address on the NSX Manager Cluster.

Untitled%2050.png

The table below will contain all the configuration parameters that I will need to deploy the second and third NSX Manager Node using the GUI.

Parameter Name Input Second Node Input Third Node
vSphere Cluster N/A N/A
VM Name Pod-120-NSXT-LM-2 Pod-120-NSXT-LM-3
Storage ESX-03-DS-01 ESX-03-DS-01
Network (Port Group) Pod-120-Mgmt Pod-120-Mgmt
System Root User Password VMware1!VMware1! VMware1!VMware1!
CLI "admin" User Password VMware1!VMware1! VMware1!VMware1!
CLI "audit" User Password VMware1!VMware1! VMware1!VMware1!
Hostname Pod-120-NSXT-LM-2 Pod-120-NSXT-LM-3
Rolename NSX Manager NSX Manager
Management Network IPv4 Address 10.203.120.13 10.203.120.14
Management Network Netmask 255.255.255.0 255.255.255.0
Default IPv4 Gateway 10.203.120.1 10.203.120.1
DNS Server list 10.203.0.5 10.203.0.5
Domain Search List sddc.lab sddc.lab
NTP Server List 10.203.0.5 10.203.0.5
Enable SSH yes yes
Allow root SSH logins yes yes

Add the second NSX Manager Node

After I clicked “Add NSX Appliance” I can start with the deployment of the second NSX Manager Node. Use the Configuration Parameters from the table above to specify the Appliance Information.

Untitled%2051.png

Use the Configuration Parameters from the table above to specify the Configuration.

Untitled%2052.png

Use the Configuration Parameters from the table above to specify the Access & Credentials.

Untitled%2053.png

After I have clicked “Install Appliance” I can look at the Appliances screen to track the progress.

Untitled%2054.png

After I have clicked “Install Appliance” I can look at the Appliances screen to track the progress.

Untitled%2055.png

When the deployment is completed I will now have an NSX Manager Cluster with two NSX Manager Nodes.

Untitled%2056.png

Add the third NSX Manager Node

Click on “Add NSX Appliance” again to start with the deployment of the third NSX Manager Node. Use the Configuration Parameters from the table above to specify the Appliance Information.

Untitled%2057.png

Use the Configuration Parameters from the table above to specify the Configuration.

Untitled%2058.png

Use the Configuration Parameters from the table above to specify the Access & Credentials.

Untitled%2059.png

After I have clicked “Install Appliance” I can look at the Appliances screen to track the progress.

Untitled%2060.png

After I have clicked “Install Appliance” I can look at the Appliances screen to track the progress.

Untitled%2061.png

When the deployment is completed I will now have an NSX Manager Cluster with three NSX Manager Nodes.

Untitled%2062.png

One thing that I want to point out is that if I have manually deployed an NSX Manager Node I will not be able to delete it from the GUI. Look as the button that is greyed out.

Untitled%2063.png

When I have deployed an NSX Manager Node using the GUI I will be able to delete it from the GUI. Look as the button that can now be used.

Untitled%2064.png

Also pay attention to the NSX Home screen that the W warning “3 node cluster recommended” is now gone.

Untitled%2065.png

STEP 5» Configure the NSX Virtual IP Address - VIP

Now that I have an NSX Manager Cluster consisting out of three Nodes I can configure the NSX Manager Cluster Virtual IP Address.

Untitled%2066.png

Parameter Name Input
Virtual IP 10.203.120.11

NSX Manager Nodes not in the same network

One of the constraints of configuring an NSX Manager Cluster Virtual IP Address is that all the Nodes needs to be part of the same network.

For demonstration purpose I have three NSX Manager Nodes that are NOT in the same network. Click on “Set Virtual IP”.

Untitled%2067.png

Specify the IP address.

Untitled%2068.png

Make note of the error, as I am trying to do something that is not allowed.

Untitled%2069.png

NSX Manager Nodes in the same network

Let’s now try it with three NSX Manager Nodes that ARE in the same network. Click on “Set Virtual IP”.

Untitled%2070.png

Specify the IP address.

Untitled%2071.png

Look at the status.

Untitled%2072.png

Verify if the Virtual IP is correctly configured below the “Virtual IP” label.

Untitled%2073.png

I can now browse to the FQDN that translates to the Virtual IP address to connect to the NSX Manager Cluster using this new Virtual IP address.

Untitled%2074.png

When I have the Virtual IP address (VIP) configured one of the NSX Manager Nodes will always be the leader, and this VIP is bound to the leader. To find out which NSX Manager Node is the leader issue the following command:

Pod-120-NSXT-LM-1> get cluster status verbose

The output shows that the leader is a7d94d56-cbfd-4ca9-8b9b-b2a9daa2d45a and this is the NSX Manager Node with the IP address of 10.203.120.12/24.

Untitled%2075.png

Continue with >> Lab: NSX Manager backup configuration

Retrieved from "http://www.iwanhoogendoorn.nl/index.php?title=Lab:_NSX_Manager_deployment_(Single_site)&oldid=5304"