How to test a charm

The Charmed Operator Framework provides a testing harness, so you can check your charm does the right thing in different scenarios without having to create a full deployment. When you run charmcraft init, the template charm it creates includes some sample tests, along with a means to run them (see the run_tests shell script) and provide a short report of unit test coverage.

Contents:

Testing basics

Here’s a minimal example, taken from the charmcraft init template with some additional comments:

import unittest
# Import the Charmed Operator Framework testing harness
from ops.testing import Harness
# Import your charm class
from charm import TestCharmCharm

class TestCharm(unittest.TestCase):
    def test_config_changed(self):
        # Instantiate the Charmed Operator Framework test harness
        harness = Harness(TestCharmCharm)
        # Set a name for the testing model created by Harness (optional).
        # Cannot be called after harness.begin()
        harness.set_model_name("testing")
        # Instruct unittest to use Harness' cleanup method on tear down
        self.addCleanup(harness.cleanup)
        # Instantiate an instance of the charm (harness.charm)
        harness.begin()
        # Test initialisation of shared state in the charm
        self.assertEqual(list(harness.charm._stored.things), [])
        # Simulates the update of config, triggers a config-changed event
        harness.update_config({"thing": "foo"})
        # Test the config-changed method stored the update in state
        self.assertEqual(list(harness.charm._stored.things), ["foo"])

We use Python’s standard unit testing framework, augmenting it with Harness, the Charmed Operator Framework’s testing harness. Harness provides some convenient mechanisms for mocking charm events and processes.

A common pattern is to specify some minimal metadata.yaml content for testing like this:

harness = Harness(TestCharmCharm, meta='''
    name: test-app
    peers:
        cluster:
            interface: cluster
    requires:
      db:
        interface: sql
    ''')
harness.begin()
...

When using Harness.begin() you are responsible for manually triggering events yourself via other harness calls:

...
# Add a relation and trigger relation-created.
harness.add_relation('db', 'postgresql') # <relation-name>, <remote-app-name>
# Add a peer relation and trigger relation-created 
harness.add_relation('cluster', 'test-app') # <peer-relation-name>, <this-app-name>

Notably, specifying relations in metadata.yaml does not automatically make them created by the harness. If you have e.g. code that accesses relation data, you must manually add those relations (including peer relations) for the harness to provide access to that relation data to your charm.

In some cases it may be useful to start the test harness and fire the same hooks that Juju would fire on deployment. This can be achieved using the begin_with_initial_hooks() method , to be used in place of the begin() method. This method will trigger the events: install -> relation-created -> config-changed -> start -> relation-joined depending on whether any relations have been created prior calling begin_with_initial_hooks(). An example of this is shown in the testing relations section.

Using the harness variable, we can simulate various events in the charm’s lifecycle:

# Update the harness to set the active unit as a "leader" (the default value is False).
# This will trigger a leader-elected event
harness.set_leader(True)
# Update config.
harness.update_config({"foo": "bar", "baz": "qux"})
# Disable hooks if we're about to do something that would otherwise cause a hook
# to fire such as changing configuration or setting a leader, but you don't want
# those hooks to fire.
harness.disable_hooks()
# Update config
harness.update_config({"foo": "quux"})
# Re-enable hooks
harness.enable_hooks()
# Set the status of the active unit. We'd need "from ops.model import BlockedStatus".
harness.charm.unit.status = BlockedStatus("Testing")

Any of your charm’s properties and methods (including event callbacks) can be accessed using harness.charm. You can check out the harness API docs for more ways to use the harness to trigger other events and to test your charm (e.g. triggering leadership-related events, testing pebble events and sidecar container interactions, etc.).

Testing log output

Charm authors can also test for desired log output. Should a charm author create log messages in the standard form:

# ...
logger = logging.getLogger(__name__)


class SomeCharm(CharmBase):
# ...
    def _some_method(self):
        logger.info("some message")
# ...

The above logging output could be tested like so:

with self.assertLogs(level="INFO") as logger:
    harness.charm._some_method()
self.assertEqual(sorted(logger.output), ["some message"])

Simulating Container Networking

version: 1.4

New functionality has been added to the Harness to more accurately track connections with workload containers. While this behavior is disabled by default, we strongly encourage enabling it by setting ops.testing.SIMULATE_CAN_CONNECT. It should be set before you create Harness instances and not changed after. A good convention is to place this in your test directory’s __init__.py file:

import ops.testing
ops.testing.SIMULATE_CAN_CONNECT = True

Previously with the test harness, containers always behaved as if they were connected - push/pull, add_layer, etc. would succeed. SIMULATE_CAN_CONNECT, however, causes containers to be initially in a disconnected state and the model.Container API methods to track can_connect state for containers accurately. This means that calls that require communication with the remote container (e.g. push/pull, add_layer, etc.) will only succeed if Container.can_connect() returns True and will raise exceptions otherwise. If you have a hook like this:

def _config_changed(event):
    c = self.unit.get_container('foo')
    c.push(...)
    self.config_value = ...

Your old testing code won’t work:

harness = Harness(CharmBase, meta='''
    name: test-app
    containers:
      foo:
        resource: foo-image
    ''')
self.addCleanup(harness.cleanup)
harness.begin()
c = harness.model.unit.get_container('foo')

# THIS NOW FAILS WITH A ConnectionError:
harness.update_config(key_values={'the-answer': 42})

Which suggests that your _config_changed hook should perhaps check Container.can_connect:

def _config_changed(event):
    c = self.unit.get_container('foo')
    if not c.can_connect():
        # wait until we can connect
        event.defer()
        return
    c.push(...)
    self.config_value = ...

Now you can test both connection states:

harness.update_config(key_values={'the-answer': 42}) # can_connect is False
harness.container_pebble_ready('foo') # can_connect is True
self.assertEqual(42, harness.charm.config_value)

The can_connect state evolves automatically to track events associated with container state, (e.g. calling container_pebble_ready makes container.can_connect become True). The can_connect state for containers can also be manually controlled using Harness.set_can_connect(container_name, True/False).

Further Reading

Writing Integration Tests for Charmed Operators


Last updated 2 months ago.