How to test a charm

The Charmed Operator Framework provides a testing harness, so you can check your charm does the right thing in different scenarios without having to create a full deployment. When you run charmcraft init, the template charm it creates includes some sample tests, along with a means to run them (see the run_tests shell script) and provide a short report of unit test coverage.

Contents:

Testing basics
Testing log output
Simulating container networking

Testing basics

Here’s a minimal example, taken from the charmcraft init template with some additional comments:

import unittest
# Import the Charmed Operator Framework testing harness
from ops.testing import Harness
# Import your charm class
from charm import TestCharmCharm

class TestCharm(unittest.TestCase):
    def test_config_changed(self):
        # Instantiate the Charmed Operator Framework test harness
        harness = Harness(TestCharmCharm)
        # Set a name for the testing model created by Harness (optional).
        # Cannot be called after harness.begin()
        harness.set_model_name("testing")
        # Instruct unittest to use Harness' cleanup method on tear down
        self.addCleanup(harness.cleanup)
        # Instantiate an instance of the charm (harness.charm)
        harness.begin()
        # Test initialisation of shared state in the charm
        self.assertEqual(list(harness.charm._stored.things), [])
        # Simulates the update of config, triggers a config-changed event
        harness.update_config({"thing": "foo"})
        # Test the config-changed method stored the update in state
        self.assertEqual(list(harness.charm._stored.things), ["foo"])

We use Python’s standard unit testing framework, augmenting it with Harness, the Charmed Operator Framework’s testing harness. Harness provides some convenient mechanisms for mocking charm events and processes.

A common pattern is to specify some minimal metadata.yaml content for testing like this:

harness = Harness(TestCharmCharm, meta='''
    name: test-app
    peers:
        cluster:
            interface: cluster
    requires:
      db:
        interface: sql
    ''')
harness.begin()
...

When using Harness.begin() you are responsible for manually triggering events yourself via other harness calls:

...
# Add a relation and trigger relation-created.
harness.add_relation('db', 'postgresql') # <relation-name>, <remote-app-name>
# Add a peer relation and trigger relation-created 
harness.add_relation('cluster', 'test-app') # <peer-relation-name>, <this-app-name>

Notably, specifying relations in metadata.yaml does not automatically make them created by the harness. If you have e.g. code that accesses relation data, you must manually add those relations (including peer relations) for the harness to provide access to that relation data to your charm.

In some cases it may be useful to start the test harness and fire the same hooks that Juju would fire on deployment. This can be achieved using the begin_with_initial_hooks() method , to be used in place of the begin() method. This method will trigger the events: install -> relation-created -> config-changed -> start -> relation-joined depending on whether any relations have been created prior calling begin_with_initial_hooks(). An example of this is shown in the testing relations section.

Using the harness variable, we can simulate various events in the charm’s lifecycle:

# Update the harness to set the active unit as a "leader" (the default value is False).
# This will trigger a leader-elected event
harness.set_leader(True)
# Update config.
harness.update_config({"foo": "bar", "baz": "qux"})
# Disable hooks if we're about to do something that would otherwise cause a hook
# to fire such as changing configuration or setting a leader, but you don't want
# those hooks to fire.
harness.disable_hooks()
# Update config
harness.update_config({"foo": "quux"})
# Re-enable hooks
harness.enable_hooks()
# Set the status of the active unit. We'd need "from ops.model import BlockedStatus".
harness.charm.unit.status = BlockedStatus("Testing")

Any of your charm’s properties and methods (including event callbacks) can be accessed using harness.charm. You can check out the harness API docs for more ways to use the harness to trigger other events and to test your charm (e.g. triggering leadership-related events, testing pebble events and sidecar container interactions, etc.).

Testing log output

Charm authors can also test for desired log output. Should a charm author create log messages in the standard form:

# ...
logger = logging.getLogger(__name__)


class SomeCharm(CharmBase):
# ...
    def _some_method(self):
        logger.info("some message")
# ...

The above logging output could be tested like so:

with self.assertLogs(level="INFO") as logger:
    harness.charm._some_method()
self.assertEqual(sorted(logger.output), ["some message"])

Simulating Container Networking

version:

Initially in 1.4, changed in version 2.0

In ops 1.4, functionality was added to the Harness to more accurately track connections to workload containers. As of ops 2.0, this behaviour is enabled and simulated by default (prior to 2.0, you had to enable it by setting ops.testing.SIMULATE_CAN_CONNECT to True before creating Harness instances).

Containers normally start in a disconnected state, and any interaction with the remote container (push, pull, add_layer, and so on) will raise an ops.pebble.ConnectionError.

To mark a container as connected, you can either call harness.set_can_connect(container, True), or you can call harness.container_pebble_ready(container) if you want to mark the container as connected and trigger its pebble-ready event.

However, if you’re using harness.begin_with_initial_hooks() in your tests, that will automatically call container_pebble_ready() for all containers in the charm’s metadata, so you don’t have to do it manually.

If you have a hook that pushes a file to the container, like this:

def _config_changed(event):
    c = self.unit.get_container('foo')
    c.push(...)
    self.config_value = ...

Your old testing code won’t work:

harness = Harness(CharmBase, meta='''
    name: test-app
    containers:
      foo:
        resource: foo-image
    ''')
self.addCleanup(harness.cleanup)
harness.begin()
c = harness.model.unit.get_container('foo')

# THIS NOW FAILS WITH A ConnectionError:
harness.update_config(key_values={'the-answer': 42})

Which suggests that your _config_changed hook should probably use Container.can_connect():

def _config_changed(event):
    c = self.unit.get_container('foo')
    if not c.can_connect():
        # wait until we can connect
        event.defer()
        return
    c.push(...)
    self.config_value = ...

Now you can test both connection states:

harness.update_config(key_values={'the-answer': 42}) # can_connect is False
harness.container_pebble_ready('foo') # set can_connect to True
self.assertEqual(42, harness.charm.config_value)

How to test a charm

Testing basics

Testing log output

Simulating Container Networking

Further Reading