Setting up Multi-node vSphere Cluster with VSan Support on Equinix Metal
This guide describes how to deploy a a multi-node vSphere cluster with vSan support enabled on Equinix Metal. vSphere is VMware's cloud computing virtualization platform. vSan is enterprise-class storage virtulization software. It allows you to manage compute and storage with a single platform.
An Equinix Metal account, with the ability to provision servers, and the ability to SSH into them.
Terraform, it is a single binary file. Visit their download page choose the specific OS. Once downloaded make the binary executable and move into your particular environment path.
Git Clone VMware repo here.
curl -LO https://releases.hashicorp.com/terraform/0.12.18/terraform_0.12.18_darwin_amd64.zip unzip terraform_0.12.18_darwin_amd64.zip chmod +x terraform sudo mv terraform /usr/local/bin/
git clone https://github.com/c0dyhi11/vmware-on-packet.git cd vmware-on-packet
Terraform uses modules to deploy infrastructure. In order to initialize the modules your simply run:
terraform init. This should download five modules into a hidden directory
You need to use an S3 compatible object store in order to download closed source packages such as vCenter and the vSan SDK. MinIO works great for this.
You will need to layout the S3 structure to look like this:
https://s3.example.com: | |__ vmware | |__ VMware-VCSA-all-6.7.0-14367737.iso | |__ vsanapiutils.py | |__ vsanmgmtObjects.py
These files can be downloaded from My VMware. Once logged in to "My VMware" the download links are as follows:
VMware vCenter Server 6.7U3 - VVMware vCenter Server Appliance ISO
VMware vSAN Management SDK 6.7U3 - Virtual SAN Management SDK for Python
You will need to find the two individual Python files in the vSAN SDK zip file and place them in the S3 bucket as shown above.
There are many variables which can be set to customize your install within
00-vars.tf. The default variables to bring up a 3 node vSphere cluster and Linux router. Change each default variable at your own risk.
There are some variables you must set with a terraform.tfvars files. You need to set
organization_id and the
project_name which will be created in Equinix Metal. Make use of your compatable S3 solution, in our case MinIO, to download "Closed Source" packages such as vCenter. You'll provide
s3_secret_key as well as the vCenter ISO file name as
Here is a quick command plus sample values to start file for you (make sure you adjust the variables to match your environment, pay special attention that the
vcenter_iso_name matches whats in your bucket):
terraform.tfvars auth_token = "cefa5c94-e8ee-4577-bff8-1d1edca93ed8" organization_id = "42259e34-d300-48b3-b3e1-d5165cd14169" project_name = "vmware-packet-project-1" s3_url = "https://s3.example.com" s3_bucket_name = "vmware" s3_access_key = "4fa85962-975f-4650-b603-17f1cb9dee10" s3_secret_key = "becf3868-3f07-4dbb-a6d5-eacfd7512b09" vcenter_iso_name = "VMware-VCSA-all-6.7.0-XXXXXXX.iso" EOF
All there is left to do now is to deploy the cluster:
terraform apply --auto-approve
This should end with output similar to this:
Apply complete! Resources: 50 added, 0 changed, 0 destroyed. Outputs: VPN_Endpoint = 126.96.36.199 VPN_PSK = @U69neoBD2vlGdHbe@o1 VPN_Pasword = 0!kfeooo?FaAvyZ2 VPN_User = vm_admin vCenter_Appliance_Root_Password = n4$REf6p*oMo2eYr vCenter_FQDN = vcva.packet.local vCenter_Password = bzN4UE7m3g$DOf@P vCenter_Username = Administrator@vsphere.local
There is an L2TP IPsec VPN setup. There is an L2TP IPsec VPN client for every platform. You'll need to reference your operating system's documentation on how to connect to an L2TP IPsec VPN.
Make sure to enable all traffic to use the VPN (aka do not enable split tunneling) on your L2TP client.
Some corporate networks block outbound L2TP traffic. If you are experiencing issues connecting, you may try a guest network or personal hotspot.
To clean up a created environment (or a failed one), run
terraform destroy --auto-approve.
If this does not work for some reason, you can manually delete each of the resources created in Equinix Metal (including the project) and then delete your terraform state file,
rm -f terraform.tfstate.