VMware Cloud Foundation 4.0.1 – actual challenges

In a proof of concept I am building a VMware Cloud Foundation deployment using it’s latest release – 4.0.1. Before the bring up, there were some questions how to deploy the management domain that uses 4 pnics per server, sorted into two distributed switches. 4.0 major release solved that, Cloud Builder do that straight from the preparation workbook, no need to use the API.

vRealize Suite Lifecycle Manager telepítése SDDC Manager-ből

VCF is no longer requires the deployment of Application Virtual Network – AVN. I admint it can help with network services for components like vRealize products and othere machines deployed here by a customer, but this happens by using NSX. The clisnt wants to be NSX less in the management domain to have totally independent from overlay networking.

Using no AVN does not mean that NSX will not be deployed in the management domain at all, since 3 NSX Managers will be deployed either way, but no NSX Edges, no segments will be created.

SDDC Manager can deploy the vRealize Suite Lifecycle Manager, but wants to do it to target an AVN segment or a VLAN backed segment, but that is a VLAN backed segment in NSX. So that way it would not be independent from NSX, since that VLAN segment is connected to a T0/T1 – effecively terminated on an NSX Edge.

There is a knowledge base article about this: https://kb.vmware.com/s/article/78608

In that article there is a PDF attached, which talks about how to deploy vRSLCM to an NSX VLAN backed PG, but in the 7th step it talks about a migration. A migration where it discusses how to move the VM from a vDS PG to an NSX VLAN segment. Well I want to know how to get to the starting point.

There is absolutely zero information available how to deploy it from SDDC Manager to a regular vDS PG.

Workload domain deployment using 4 pnics and 2 vDS

As said in version 4.0+ it is possible to do the bring-up from Cloud Builder, still based on the preparation XLS if you have 4 pnics and you want 2 distributed switches.Earlier on this was possible only if API was used.

But all this is true only when talking about the management domain. Once the SDDC Manager is deployed and stand up is complete, the SDDC Manager can be used to to deploy workload domains. And SDDC manager still cannot deploy a workload domain that useds more than two network interfaces.

Once again. If you want to have workload domains with more than 2 pnics, you still need to use the API.

Let’s go and use the API guide to create a JSON. https://code.vmware.com/apis/1002/vmware-cloud-foundation Point 2.9.

{
  "domainName" : "wld01",
  "vcenterSpec" : {
    "name" : "wld01-vc",
    "networkDetailsSpec" : {
      "ipAddress" : "192.168.10.10",
      "dnsName" : "wld01-vc.newman.lab",
      "gateway" : "192.168.10.254",
      "subnetMask" : "255.255.255.0"
    },
    "rootPassword" : "VMware1!",
    "datacenterName" : "wld01-dc"
  },
  "computeSpec" : {
    "clusterSpecs" : [ {
      "name" : "wld01-clu01",
      "hostSpecs" : [ {
        "id" : "681f2085-a888-498c-b68a-07f5817d8c31",
        "username" : "root",
        "hostNetworkSpec" : {
          "vmNics" : [ {
            "id" : "vmnic0",
            "vdsName" : "wld01-vds01"
          }, {
            "id" : "vmnic2",
            "vdsName" : "wld01-vds01"
          }, {
            "id" : "vmnic1",
            "vdsName" : "wld01-vds02"
          }, {
            "id" : "vmnic3",
            "vdsName" : "wld01-vds02"
          } ]
        }
      }, {

        "id" : "8a129883-7ab3-4a9f-a0d1-513435e6cbd9",
        "username" : "root",
        "hostNetworkSpec" : {
          "vmNics" : [ {
            "id" : "vmnic0",
            "vdsName" : "wld01-vds01"
          }, {
            "id" : "vmnic2",
            "vdsName" : "wld01-vds01"
          }, {
            "id" : "vmnic1",
            "vdsName" : "wld01-vds02"
          }, {
            "id" : "vmnic3",
            "vdsName" : "wld01-vds02"
          } ]
        }
      }, {
        "id" : "755e516e-564e-453a-bc73-8d9f320122fc",
        "username" : "root",
        "hostNetworkSpec" : {
          "vmNics" : [ {
            "id" : "vmnic0",
            "vdsName" : "wld01-vds01"
          }, {
            "id" : "vmnic2",
            "vdsName" : "wld01-vds01"
          }, {
            "id" : "vmnic1",
            "vdsName" : "wld01-vds02"
          }, {
            "id" : "vmnic3",
            "vdsName" : "wld01-vds02"
          } ]
        }
      } ],
      "datastoreSpec" : {
	  "vmfsDatastoreSpec" : {
		"fcSpec" : [ {
		"datastoreName" : "FC_Datastore_01"
        } ]
      }
      },
      "networkSpec" : {
        "vdsSpecs" : [ {
          "name" : "wld01-vds01",
          "portGroupSpecs" : [ {
            "name" : "wld01-pg-mgmt",
            "transportType" : "MANAGEMENT"
          },  {
            "name" : "wld01-pg-vmotion",
            "transportType" : "VMOTION"
          } ]
        }, {
          "name" : "wld01-vds02",
          "isUsedByNsxt" : true
        } ],
        "nsxClusterSpec" : {
          "nsxTClusterSpec" : {
            "geneveVlanId" : 400
          }
        }
      }
    } ]
  },
  "nsxTSpec" : {
    "nsxManagerSpecs" : [ {
      "name" : "wld01-nsx01",
      "networkDetailsSpec" : {
        "ipAddress" : "192.168.10.21",
        "dnsName" : "wld01-nsx01.newman.lab",
        "gateway" : "192.168.10.254",
        "subnetMask" : "255.255.255.0"
      }
    }, {
      "name" : "wld01-nsx02",
      "networkDetailsSpec" : {
        "ipAddress" : "192.168.10.22",
        "dnsName" : "wld01-nsx02.newman.lab",
        "gateway" : "192.168.10.254",
        "subnetMask" : "255.255.255.0"
      }
    }, {
      "name" : "wld01-nsx03",
      "networkDetailsSpec" : {
        "ipAddress" : "192.168.10.23",
        "dnsName" : "wld01-nsx03.newman.lab",
        "gateway" : "192.168.10.254",
        "subnetMask" : "255.255.255.0"
      }
    } ],
    "vip" : "192.168.10.20",
    "vipFqdn" : "wld01-nsx.newman.lab",
    "licenseKey" : "will not post it here",
    "nsxManagerAdminPassword" : "VMware1!VMware1!",
	"formFactor" : "medium"
  }
}

This JSON can be validated by SDDC Manager.

If this is “Completed” and “Succeeded” it is all good. API guide confirms that.

After kicking off the deployment of that workload domain, it is being created but fails. Tried five times, all failed. The vCenter is not being deployed for the WLD so it stops before that by displaying this in the log on the SDDC Manager.

I suspect I will need to renumber the vmnics in the server as document says that it must use “traditional numbering”. Wonder why the validation succseeds than. Furthermore the managment domain uses the same vmnic numbering as it has 4 1Gbit/s NICs that are unused, and the bringup had no issues with that.

I have no solution for these issues so far, but will update this post once I manage to resolve them.