How to use VMware vSphere with Juju

This guide takes you through how to register a VMware ESXi cluster with vSphere as a “vSphere cloud”.

Contents:

Prerequisites

In order to add a vSphere cloud you will need to have an existing vSphere installation which supports, or has access to, the following:

  • VMware Hardware Version 8 (or greater)
  • ESXi 5.0 (or greater)
  • Internet access
  • DNS and DHCP

Juju supports both high-availability vSAN deployments as well and standard deployments.

Add a vSphere cloud

There are two methods to define a cloud for Juju.

  • Use an interactive prompt
  • Define a YAML file and provide that to Juju as a command-line argument

Both methods make use of the juju add-cloud command. During the process, you will be asked to provide the following information:

  • A name that Juju will refer to the cluster as
  • IP address of your cluster’s vCenter
  • The name(s) of any Datacenter(s) that you want to enable Juju to be able to deploy to

To access detailed help about all of its options, use this command:

juju help add-cloud

Finding Datacenter names in vSphere Your Datacenters are available through the vSphere web client by selecting vCenter Inventory Lists > Resources > Datacenters from the hierarchical menu at the top left. The values you need are listed in the ‘Name’ column, such as the ‘dc0’ and ‘dc1’ Datacenters shown here: vSphere web client showing Datacenters

Use an interactive prompt

Use the add-cloud command to interactively add your vSphere cloud to Juju’s list of clouds:

juju add-cloud --local

What does --local do? Using the --local option instructs Juju to store the cloud definition on the machine that you’re executing the command from. Omitting it will store the cloud definition on the controller machine. This enables controllers to control models on multiple clouds, but isn’t recommended while you are creating your first model.

An session with multiple datacenters:

Cloud Types
  lxd
  maas
  manual
  openstack
  vsphere

Select cloud type: vsphere

Enter a name for your vsphere cloud: vsp-cloud

Enter the vCenter address or URL: 192.0.2.1

Enter datacenter name: dc0

Enter another datacenter? (y/N): y

Enter datacenter name: dc1

Enter another datacenter? (y/N): n

Cloud "vsp-cloud" successfully added

You will need to add credentials for this cloud (`juju add-credential vsp-cloud`)
before creating a controller (`juju bootstrap vsp-cloud`).

Define a YAML file

The manual method makes use of configuration files defined in YAML. To define a configuration file that mimics the parameters provided by the interactive example, you can follow:

clouds:
 vsp-cloud:
  type: vsphere
  auth-types: [userpass]
  endpoint: 192.0.2.1
  regions:
   dc0: {}  # these empty maps
   dc1: {}  # are necessary

Adding a cloud manually can either be done locally or the cloud can be stored on a current controller). Here, we’ll show how to do it locally (client cache).

To add cloud ‘vsp-cloud’, assuming the configuration file is vsp-cloud.yaml in the current directory, we would run:

juju add-cloud --local vsp-cloud vsp-cloud.yaml

Confirm that you’ve added the cloud correctly

Ask Juju to report the clouds that it has registered:

juju clouds --local

Add a credential

Use the add-credential command to interactively add your credentials to the new cloud:

juju add-credential vsp-cloud

Example user session:

Enter credential name: vsp-cloud-creds

Using auth-type "userpass".

Enter user: jlaurin@juju.example.com

Enter password: ********

Enter vmfolder (optional): juju-root

Credential "vsp-cloud-creds" added locally for cloud "vsp-cloud".

We’ve called the new security credential ‘vsp-cloud-creds’. You will need to provide your VMware account username–this looks like an email address–and the associated password.

Locked out?: Credentials for the vSphere cloud have been reported to occasionally stop working over time. If this occurs, then you can “remind” vSphere of your credentials. See Dealing with inert credentials for guidance.

To update the credential, you can use juju update-credential, as usual. However, it has been reported that sometimes that does not work. In that case, use juju add-credential to add a completely new credential, and then set the default for the cloud with juju default-credential.

For more information about credentials, read through the Credentials page.

Confirm that you’ve added the credential correctly

To view the credentials that Juju knows about, use the credentials command and inspect both remote and locally stored credentials:

juju credentials
juju credentials --local

Create a controller

You are now ready to create a Juju controller for cloud ‘vsp-cloud’:

juju bootstrap vsp-cloud vsp-controller

Above, the name given to the new controller is ‘vsp-controller’. vSphere will provision an instance to run the controller on.

For a detailed explanation and examples of the bootstrap command see the Creating a controller and Configuring Controllers pages.

VMware-specific bootstrapping options

There are three VMware-specific options available for specifying the network and datastore to use:

  • primary-network The primary network that VMs will be connected to. If this is not specified, Juju will look for a network named VM Network.
  • external-network An external network that VMs will be connected to. The resulting IP address for a VM will be used as its public address. An external network provides the interface to the internet for virtual machines connected to external organization vDC networks.
  • datastore The datastore in which to create VMs. If this is not specified, the process will abort unless there is only one datastore available.
  • force-vm-hardware-version This change adds a new model level flag, that allows operators to set a newer compatibility version for the instances that get spawned by juju.
  • disk-provisioning-type This change allows operators to set a new model-level config option which dictates how template VM disks should be cloned when creating a new machine.

For example:

juju bootstrap vsp-cloud vsp-controller \
    --config primary-network=$PRIMARY_NET \
    --config external-network=$EXTERNAL_NET \
    --config datastore=$DATA_STORE

The above --config options will only apply to the ‘controller’ and ‘default’ models. Use option --model-default instead if you want any newly-created models to be affected. You can also use the model-defaults command once the controller is created to do the same thing.

Initial bootstrap duration: When creating a controller with vSphere, a cloud image is downloaded to the client and then uploaded to the ESX host. This depends on your network connection and can take a while. Using templates can speed up bootstrap and machine deployment.

vSphere-specific features

Using templates

To speed up bootstrap and deploy, you can use VM templates, already created in your vSphere. Templates can be created by hand on your vSphere, or created from an existing VM.

Examples assume that the templates are in directory $DATA_STORE/templates.

Via simplestreams:

mkdir -p $HOME/simplestreams
juju-metadata generate-image -d $HOME/simplestreams/ -i "templates/juju-focal-template" -s focal -r $DATA_STORE -u $CLOUD_ENDPOINT
juju-metadata generate-image -d $HOME/simplestreams/ -i "templates/juju-bionic-template" -s bionic -r $DATA_STORE -u $CLOUD_ENDPOINT
juju bootstrap --metadata-source $HOME/image-streams vsphere

Bootstrap juju with the controller on a VM running focal:

juju bootstrap vsphere --bootstrap-image="templates/focal-test-template"  --bootstrap-series=focal --bootstrap-constraints "arch=amd64"

Using add-image:

juju metadata add-image templates/bionic-test-template --series bionic

Specifying a VM Hardware version

The VM Hardware version to use launching an instance can be specified at bootstrap and/or at the model level with the model config force-vm-hardware-version.

To bootstrap juju using VM Hardware version 17:

juju bootstrap vsphere --config force-vm-hardware-version=17

To use VM Hardware version 15 for the next machines deployed in a model:

juju model-config force-vm-hardware-version=15

Disk provisioning

Dictate how template VM disks should be cloned when creating a new machine by setting a the model-level config option disk-provisioning-type. Supported values are:

  • thin - Sparse provisioning, only written blocks will take up disk space on the datastore
  • thick - The entire size of the virtual disk will be deducted from the datastore, but unwritten blocks will not be zeroed out. This adds 2 potential pitfalls. See comments in provider/vsphere/internal/vsphereclient/client.go regarding DiskProvisioningType.
  • thickEagerZero (default) - The entire size of the virtual disk is deducted from the datastore, and unwritten blocks are zeroed out. Improves latency when committing to disk, as no extra step needs to be taken before writing data.

To use thin disk provisioning with machine in the model:

juju model-config disk-provisioning-type=thin

Levels of placement

When creating a controller, there are three levels of placement that Juju understands: cloud, region, and availability zone. In vSphere, these are mapped in two different ways depending on your topology:

  • cloud (vSphere endpoint), region (Datacenter), availability zone (host)
  • cloud (vSphere endpoint), region (Datacenter), availability zone (cluster)

If your topology has a cluster without a host, Juju will see this as an Availability Zone and may fail silently. To solve this, either make sure the host is within the cluster, or use a placement directive:

juju bootstrap vsphere/<datacenter> <controllername> --to zone=<cluster|host>

To create a controller using Datacenter ‘dc1’ you would enter the following:

juju bootstrap vsp-cloud/dc1 vsp-controller

Specify a datastore when deploying an application

There is a constraint called ‘root-disk-source’ that can stipulate the name of a vSphere datastore to house the root disk:

juju deploy myapp --constraints root-disk-source=mydatastore

Deploy applications to a specific host or cluster

Resource pools within a host or cluster can be specified with the ‘zones’ constraint:

juju deploy myapp --constraints zones=myhost
juju deploy myapp --constraints zones=myfolder/myhost
juju deploy myapp --constraints zones=mycluster/mypool
juju deploy myapp --constraints zones=mycluster/myparent/mypool

Last updated 1 year, 28 days ago.

Help improve this document in the forum.