How to back up the controller
A backup of a controller enables one to save (and later re-establish) the configuration and state of a controller. It does not influence workload instances on the backing cloud. That is, if such an instance is terminated directly in the cloud then a controller restore cannot re-create it.
- The Juju controller
- High availability considerations
You can restore a previously backed up system. See the Restore from a backup page for guidance.
Data backups can also be made of the Juju client. See the Juju client page for guidance.
Juju provides commands for recovering a controller in case of breakage or in the case where the controller no longer exists.
The current state is held within the ‘controller’ model. Therefore, all backup commands need to operate within that model explicitly or by ensuring the current model is the ‘controller’ model. In the examples provided below, both the controller name and the model name are expressed explicitly (e.g.
-m aws:controller). Due to the delicate nature of data backups, this method is highly recommended.
create-backup command is used to create a backup. It does so by generating an archive and downloading it to the Juju client system as a ‘tar.gz’ file (a local backup). If the
--keep-copy option is used then a copy of the archive will also remain on the controller (a remote backup). With the aid of the
--no-download option a local backup can be prevented, but since the archive must be kept somewhere, this option implies
The name of the backup is composed of the creation time (in UTC) and a unique identifier.
The below examples assume the existence of the following controllers (output to
Controller Model User Access Cloud/Region Models Nodes HA Version aws default admin superuser aws/us-east-1 2 1 none 126.96.36.199 lxd* default admin superuser localhost/localhost 2 1 none 188.8.131.52
To create a backup of the ‘aws’ controller (and keep a copy on the controller):
juju create-backup -m aws:controller --keep-copy
backup ID: 20210604-002522.eb41da95-9a75-4768-8359-d24d0ecef911 backup format version: 1 juju version: 184.108.40.206 series: focal controller UUID: 820c616c-3310-4ea3-8acc-85b62df1f0a3 model UUID: eb41da95-9a75-4768-8359-d24d0ecef911 machine ID: 0 created on host: juju-cef911-0 checksum: 4zqgfbnlQeuvPNsqAz1RyOFys14= checksum format: SHA-1, base64 encoded size (B): 77842749 stored: 2021-06-04 00:25:43 +0000 UTC started: 2021-06-04 00:25:22.902342579 +0000 UTC finished: 2021-06-04 00:25:38.599732549 +0000 UTC notes: Remote backup stored on the controller as 20210604-002522.eb41da95-9a75-4768-8359-d24d0ecef911. Downloaded to juju-backup-20210604-002522.tar.gz.
From the name of the archive we see that the backup was made on June 4, 2021 at 00:25:22 UTC. Note that the ‘backup ID’ field will be blank if
--keep-copy was not specified.
Archive filenames do not include the associated controller name. Care should therefore be taken when archiving from multiple controllers. To specify a custom name use the
--filename option. This option does not affect the remote archive name.
To create a backup of the ‘lxd’ controller while both using a custom filename and adding an optional note:
juju create-backup -m lxd:controller --keep-copy --filename juju-backup-lxd-20180515-193724.tar.gz "fresh lxd controller"
The optional note is exposed via the
show-backup command detailed below.
A backup of a fresh (empty) environment, regardless of cloud type, is approximately 75 MiB in size.
The following commands are available for managing backups (apart from restoring):
backups command displays the names of all remote backups for a given controller. For instance, to see all remotely stored backups for the ‘lxd’ controller:
juju backups -m lxd:controller
show-backup command provides a metadata record for a specific remote backup (identified via the
backups command). For example, to query a backup stored on the ‘lxd’ controller:
juju show-backup -m lxd:controller 20210604-002522.eb41da95-9a75-4768-8359-d24d0ecef911
backup ID: 20210604-002522.eb41da95-9a75-4768-8359-d24d0ecef911 backup format version: 1 juju version: 220.127.116.11 series: focal controller UUID: model UUID: eb41da95-9a75-4768-8359-d24d0ecef911 machine ID: 0 created on host: juju-cef911-0 checksum: 4zqgfbnlQeuvPNsqAz1RyOFys14= checksum format: SHA-1, base64 encoded size (B): 77842749 stored: 2021-06-04 00:25:43 +0000 UTC started: 2021-06-04 00:25:22 +0000 UTC finished: 2021-06-04 00:25:38 +0000 UTC notes:
started time is the most pertinent of the various timestamps. It refers to the time the backup was created.
download-backup command downloads a specific remote backup (again, identified via the
backups command). Here, we download a backup that is stored on the ‘aws’ controller:
juju download-backup -m aws:controller 20180515-191942.7e45250b-637a-4dc9-8389-c6aa70100cd6
upload-backup command uploads a specific local backup to a controller. For example:
juju upload-backup -m lxd:controller juju-backup-20180515-193724.tar.gz
It is not possible to upload a file that is equivalent to a backup stored remotely. The process will be cancelled and an error message will be printed.
remove-backup command removes a specific remote backup from a controller. For instance:
juju remove-backup -m aws:controller 20180515-191942.7e45250b-637a-4dc9-8389-c6aa70100cd6
To clean up any possible remote backups the
--keep-latest option can be used. This option instructs Juju to remove all remote backups with the exception of the most recently created one:
juju remove-backup -m aws:controller --keep-latest
To revert the state of an environment to a previous time, use the Stand-alone
juju restore-backup command was used, but as of Juju 2.9 that command is deprecated, and
juju-restore should be used instead.
Although Controller high availability makes for a more robust (and load balanced) Juju infrastructure, it should not replace the need for data backups. It does, however, make the prospect of restoring from backup less likely, since as long as one controller cluster member remains operational, the others can be replaced via the
enable-ha command. A restore in an HA scenario therefore only becomes necessary when all controllers have failed. However, if a restore is applied to a cluster with active members all reachable controllers will naturally have their data overwritten.
In the advent that all controllers are unresponsive the following steps should be taken:
- Remove the controller
- Create a pristine controller
- Perform a data restore
- Enable HA
To demonstrate this, consider an initial AWS-based controller named ‘aws-ha3-1’ with three cluster members. The new controller will be called ‘aws-ha3-2’:
juju kill-controller aws-ha3-1 juju bootstrap aws aws-ha3-2 # use juju-restore to perform the data restore juju enable-ha -m aws-ha3-2:controller -n 3
Section Recovering from controller failure details how to deal with a partially degraded cluster.
Last updated 5 months ago.