[receiver/journald] add support for running journalctl in a chroot

[jefferbrecht]

Component(s)

receiver/journald

Is your feature request related to a problem? Please describe.

Reading journald logs from a containerized collector presents a few challenges:

• It's one of the rare cases where the collector calls out to an external binary
• That binary is not bundled with official collector images
• That binary can only stably read journals that match its own version, which frustrates attempts to re-implement it natively in Go
• That binary is not portable, which frustrates attempts to mount it to a running collector container

Describe the solution you'd like

A robust way for a single collector container image to read journald logs from any journald-enabled host using the same journalctl binary that originally wrote them, to ensure maximal compatibility. Specifically, the ability to run journalctl from a chroot that's mounted from the host.

An analogous feature is already implemented for the hostmetrics receiver in the form of a root_path configuration that allows it to scrape host metrics from a mounted host rootfs.

Describe alternatives you've considered

One alternative is to set up systemd-journal-gatewayd on the host and enhance the journaldreceiver to consume logs from its HTTP endpoint instead of calling the journalctl binary directly; however, my particular use case involves hosts that do not provide systemd-journal-gatewayd, so I still need a solution that calls journalctl directly.

Another alternative is to re-implement the journald interface natively in Go; however, this would not be as robust against binary journal version differences that are presented when running the container across a variety of arbitrary hosts.

Another alternative is to rebuild a collector container that includes its own copy of the journalctl binary; however, this would only work properly on hosts which match that version of journalctl.

Another alternative is to include a shell in the collector image and stub out the journalctl binary with a shell script that implements a chroot; however, I have a requirement to not include a shell in my collector image if possible.

Additional context

I am working on a PR for this feature already and will be able to publish it shortly.

Due to a golang bug, this feature would need a configurable path to the journalctl binary in addition to a configurable chroot path.

Related issues:

• Contrib image: journaldreceiver receiver is missing journalctl opentelemetry-collector-releases#462
• Add journald receiver #2332
• [journaldreceiver]: implement golang based reader for journaldreceiver #32711

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• receiver/journald: @belimawr @namco1992

See Adding Labels via Comments if you do not have permissions to add labels yourself.