Agent introspection: juju_heap_profile

See also: Agent introspection

The heap profile provides memory allocation samples. Helpful to monitor current memory usage and find memory leaks. This is primarily useful to developers to help debug problems that may be occurring in deployed systems.


Can be run on any juju machine. Suggest putting the output in a file, so it can be compared at different points in time.

juju_heap_profile > heap_profile.01

Example output

heap profile: 31: 694464 [33638: 106713992] @ heap/1048576
1: 196608 [1: 196608] @ 0x2d3fa53 0x2d3f9f6 0x2d40545 0x2d4030f 0x2d05ddb 0x2d0479f 0x2d05bf1 0x2d26c32 0x8a3cef 0x2d0381d 0x2d04146 0x2d008ba 0x2d01fbe 0x8a779b 0x8a2797 0x468fc1
#       0x2d3fa52                        /home/heather/work-test/src/
#       0x2d3f9f5*apiServerLoggers).getLogger+0x135              /home/heather/work-test/src/
#       0x2d40544*agentLoggingStrategy).init+0x1c4               /home/heather/work-test/src/
#       0x2d4030e            /home/heather/work-test/src/
#       0x2d05dda*logSinkHandler).ServeHTTP.func1+0x19a  /home/heather/work-test/src/
#       0x2d0479e                             /home/heather/work-test/src/
#       0x2d05bf0*logSinkHandler).ServeHTTP+0xb0         /home/heather/work-test/src/
#       0x2d26c31*Server).trackRequests.func1+0x111              /home/heather/work-test/src/
#       0x8a3cee        net/http.HandlerFunc.ServeHTTP+0x2e                                             /snap/go/9605/src/net/http/server.go:2084
#       0x2d0381c*BasicAuthHandler).ServeHTTP+0x3fc  /home/heather/work-test/src/
#       0x2d04145*QueryModelHandler).ServeHTTP+0x325 /home/heather/work-test/src/
#       0x2d008b9*PatternServeMux).ServeHTTP+0x199                     /home/heather/work-test/pkg/mod/
#       0x2d01fbd*Mux).ServeHTTP+0x9d              /home/heather/work-test/src/
#       0x8a779a        net/http.serverHandler.ServeHTTP+0x43a                                          /snap/go/9605/src/net/http/server.go:2916
#       0x8a2796        net/http.(*conn).serve+0x5d6                                                    /snap/go/9605/src/net/http/server.go:1966

1: 196608 [1: 196608] @ 0x2ffcec5 0x2ffce0e 0x2ffd07c 0x778495 0x468fc1
#       0x2ffcec4                                        /home/heather/work-test/src/
#       0x2ffce0d                             /home/heather/work-test/src/
#       0x2ffd07b*modelWorkerManager).starter.func1+0x41b        /home/heather/work-test/src/
#       0x778494*Runner).runWorker+0x2d4                                             /home/heather/work-test/pkg/mod/

# and many more

Interesting Output

The output of the heap profile can be difficult to read on its own. Using the pprof go tool can help.

To find a memory leak, compare 2 heap profiles:

go tool pprof -http localhost:8100 -base juju_heap_profile-2022-06-11.00 jujud-2.9.29/jujud juju_heap_profile-2022-06-12.16 

Find jujud binaries in the streams

Last updated a month ago.