feat(journalctl): allow to configure units and matches together

Dominik Rosiek · Dominik Rosiek · commit b788bced795b · 2023-04-14T09:18:51.000+02:00
Signed-off-by: Dominik Rosiek &lt;drosiek@sumologic.com&gt;
diff --git a/pkg/stanza/docs/operators/journald_input.md b/pkg/stanza/docs/operators/journald_input.md
@@ -14,9 +14,9 @@ The `journald_input` operator will use the `__REALTIME_TIMESTAMP` field of the j
 | `output`          | Next in pipeline | The connected operator(s) that will receive all outbound entries. |
 | `directory`       |                  | A directory containing journal files to read entries from. |
 | `files`           |                  | A list of journal files to read entries from. |
-| `units`           |                  | A list of units to read entries from. This option cannot be used together with `matches`. |
-| `matches`         |                  | A list of matches to read entries from. This option cannot be used together with `units`. See [Matches](#matches) example. |
-| `priority`        | `info`           | Filter output by message priorities or priority ranges. |
+| `units`           |                  | A list of units to read entries from. See [Multiple filtering options](#multiple-filtering-options) examples, if you want to use it together with `matches` and/or `priority`. |
+| `matches`         |                  | A list of matches to read entries from. See [Matches](#matches) and [Multiple filtering options](#multiple-filtering-options) examples. |
+| `priority`        | `info`           | Filter output by message priorities or priority ranges. See [Multiple filtering options](#multiple-filtering-options) examples, if you want to use it together with `units` and/or `matches`. |
 | `start_at`        | `end`            | At startup, where to start reading logs from the file. Options are `beginning` or `end`. |
 | `attributes`      | {}               | A map of `key: value` pairs to add to the entry's attributes. |
 | `resource`        | {}               | A map of `key: value` pairs to add to the entry's resource. |
@@ -48,12 +48,51 @@ The following configuration:
       _UID: "1000"
 ```
 
-will be passed to `journald` as the following arguments: `journald ... _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000`,
+will be passed to `journalctl` as the following arguments: `journalctl ... _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000`,
 which is going to retrieve all entries which match at least one of the following rules:
 
 - `_SYSTEMD_UNIT` is `ssh`
 - `_SYSTEMD_UNIT` is `kubelet` and `_UID` is `1000`
 
+#### Multiple filtering options
+
+In case of using multiple following options, conditions between them are logically `AND`ed and within them are logically `OR`ed:
+
+```text
+( priority )
+AND
+( units[0] OR units[1] OR units[2] OR ... units[U] )
+AND
+( matches[0] OR matches[1] OR matches[2] OR ... matches[M] )
+```
+
+Consider the following example:
+
+```yaml
+- type: journald_input
+  matches:
+    - _SYSTEMD_UNIT: ssh
+    - _SYSTEMD_UNIT: kubelet
+      _UID: "1000"
+  units:
+    - kubelet
+    - systemd
+  priority: info
+```
+
+The above configuration will be passed to `journalctl` as the following arguments
+`journalctl ... --priority=info --unit=kubelet --unit=systemd _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000`,
+which is going to effectively retrieve all entries which matches the following set of rules:
+
+- `_PRIORITY` is `6`, and
+- `_SYSTEMD_UNIT` is `kubelet` or `systemd`, and
+- entry matches at least one of the following rules:
+
+  - `_SYSTEMD_UNIT` is `ssh`
+  - `_SYSTEMD_UNIT` is `kubelet` and `_UID` is `1000`
+
+Note, that if you use some fields which aren't associated with an entry, the entry will always be filtered out.
+
 ### Simple journald input
 
 Configuration:
diff --git a/pkg/stanza/operator/input/journald/journald.go b/pkg/stanza/operator/input/journald/journald.go
@@ -98,11 +98,6 @@ func (c Config) Build(logger *zap.SugaredLogger) (operator.Operator, error) {
 }
 
 func (c Config) buildArgs() ([]string, error) {
-	// validate arguments
-	if len(c.Units) > 0 && len(c.Matches) > 0 {
-		return nil, fmt.Errorf("cannot use both 'matches' and 'units' configurations together")
-	}
-
 	args := make([]string, 0, 10)
 
 	// Export logs in UTC time
diff --git a/pkg/stanza/operator/input/journald/journald_test.go b/pkg/stanza/operator/input/journald/journald_test.go
@@ -171,11 +171,11 @@ func TestBuildConfig(t *testing.T) {
 				cfg.Units = []string{"ssh"}
 				cfg.Matches = []MatchConfig{
 					{
-						"-SYSTEMD_UNIT": "dbus.service",
+						"_SYSTEMD_UNIT": "dbus.service",
 					},
 				}
 			},
-			ExpectedError: "cannot use both 'matches' and 'units' configurations together",
+			Expected: []string{"--utc", "--output=json", "--follow", "--unit", "ssh", "--priority", "info", "_SYSTEMD_UNIT=dbus.service"},
 		},
 	}
 
diff --git a/receiver/journaldreceiver/README.md b/receiver/journaldreceiver/README.md
@@ -16,9 +16,9 @@ Journald receiver is dependent on `journalctl` binary to be present and must be
 | `directory` | `/run/log/journal` or `/run/journal` | A directory containing journal files to read entries from |
 | `files`     |                                      | A list of journal files to read entries from |
 | `start_at`  | `end`                                | At startup, where to start reading logs from the file. Options are beginning or end |
-| `units`     |                                      | A list of units to read entries from. This option cannot be used together with `matches` |
-| `matches`   |                                      | A list of matches to read entries from. This option cannot be used together with `units`. See [Matches](#matches) example |
-| `priority`  | `info`                               | Filter output by message priorities or priority ranges |
+| `units`     |                                      | A list of units to read entries from. See [Multiple filtering options](#multiple-filtering-options) examples, if you want to use it together with `matches` and/or `priority`. |
+| `matches`   |                                      | A list of matches to read entries from. See [Matches](#matches) and [Multiple filtering options](#multiple-filtering-options) examples. |
+| `priority`  | `info`                               | Filter output by message priorities or priority ranges. See [Multiple filtering options](#multiple-filtering-options) examples, if you want to use it together with `units` and/or `matches`. |
 | `storage`   | none                                 | The ID of a storage extension to be used to store cursors. Cursors allow the receiver to pick up where it left off in the case of a collector restart. If no storage extension is used, the receiver will manage cursors in memory only. |
 
 ### Example Configurations
@@ -47,11 +47,47 @@ The following configuration:
       _UID: "1000"
 ```
 
-will be passed to `journald` as the following arguments: `journald ... _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000`,
+will be passed to `journalctl` as the following arguments: `journalctl ... _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000`,
 which is going to retrieve all entries which match at least one of the following rules:
 
 - `_SYSTEMD_UNIT` is `ssh`
 - `_SYSTEMD_UNIT` is `kubelet` and `_UID` is `1000`
 
-[alpha]: https://github.com/open-telemetry/opentelemetry-collector#alpha
-[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
+#### Multiple filtering options
+
+In case of using multiple following options, conditions between them are logically `AND`ed and within them are logically `OR`ed:
+
+```text
+( priority )
+AND
+( units[0] OR units[1] OR units[2] OR ... units[U] )
+AND
+( matches[0] OR matches[1] OR matches[2] OR ... matches[M] )
+```
+
+Consider the following example:
+
+```yaml
+- type: journald_input
+  matches:
+    - _SYSTEMD_UNIT: ssh
+    - _SYSTEMD_UNIT: kubelet
+      _UID: "1000"
+  units:
+    - kubelet
+    - systemd
+  priority: info
+```
+
+The above configuration will be passed to `journalctl` as the following arguments
+`journalctl ... --priority=info --unit=kubelet --unit=systemd _SYSTEMD_UNIT=ssh + _SYSTEMD_UNIT=kubelet _UID=1000`,
+which is going to effectively retrieve all entries which matches the following set of rules:
+
+- `_PRIORITY` is `6`, and
+- `_SYSTEMD_UNIT` is `kubelet` or `systemd`, and
+- entry matches at least one of the following rules:
+
+  - `_SYSTEMD_UNIT` is `ssh`
+  - `_SYSTEMD_UNIT` is `kubelet` and `_UID` is `1000`
+
+Note, that if you use some fields which aren't associated with an entry, the entry will always be filtered out.

Original file line number	Diff line number	Diff line change
`@@ -171,11 +171,11 @@ func TestBuildConfig(t *testing.T) {`
`171`	`171`	`cfg.Units = []string{"ssh"}`
`172`	`172`	`cfg.Matches = []MatchConfig{`
`173`	`173`	`{`
`174`		`- "-SYSTEMD_UNIT": "dbus.service",`
	`174`	`+ "_SYSTEMD_UNIT": "dbus.service",`
`175`	`175`	`},`
`176`	`176`	`}`
`177`	`177`	`},`
`178`		`- ExpectedError: "cannot use both 'matches' and 'units' configurations together",`
	`178`	`+ Expected: []string{"--utc", "--output=json", "--follow", "--unit", "ssh", "--priority", "info", "_SYSTEMD_UNIT=dbus.service"},`
`179`	`179`	`},`
`180`	`180`	`}`
`181`	`181`