Filebeat spec is incorporating Packetbeat inputs into its config

[andrewkroh]

When an agent policy contains the Packetbeat redis input, Elastic Agent is generating config for Filebeat that includes a redis log input. The policy for Packetbeat uses input type: packet with a data_stream of type: redis. For example:

# agent policy
inputs:
- type: packet
  streams:
  - data_stream:
      dataset: network_traffic.redis
      type: logs
    type: redis
    ports:
    - 6379

The impact is that this causes Filebeat to report UNHEALTHY status if Packetbeat is deployed at the same time. A workaround is to disable redis collection in the Network Packet Capture integration.

Here is a patch to the Elastic Agent testdata that reproduces the bug using unit tests:

Details

diff --git a/internal/pkg/agent/program/testdata/single_config-packetbeat.yml b/internal/pkg/agent/program/testdata/single_config-packetbeat.yml
index f800d0bd2..4ea37b1fb 100644
--- a/internal/pkg/agent/program/testdata/single_config-packetbeat.yml
+++ b/internal/pkg/agent/program/testdata/single_config-packetbeat.yml
@@ -23,6 +23,13 @@ inputs:
     data_stream:
       dataset: packet.icmp
       type: logs
+  - data_stream:
+      dataset: network_traffic.redis
+      type: logs
+    id: packet-network_traffic.redis-387bdc6a-0acb-4ef2-9552-c21e524a2d21
+    ports:
+      - 6379
+    type: redis
 output:
   elasticsearch:
     hosts:
diff --git a/internal/pkg/agent/program/testdata/single_config.yml b/internal/pkg/agent/program/testdata/single_config.yml
index 16a03f9a7..140a61f79 100644
--- a/internal/pkg/agent/program/testdata/single_config.yml
+++ b/internal/pkg/agent/program/testdata/single_config.yml
@@ -104,6 +104,13 @@ inputs:
       data_stream:
         dataset: packet.icmp
         type: logs
+    - data_stream:
+        dataset: network_traffic.redis
+        type: logs
+      id: packet-network_traffic.redis-387bdc6a-0acb-4ef2-9552-c21e524a2d21
+      ports:
+        - 6379
+      type: redis
 - id: endpoint-id
   type: endpoint
   name: endpoint-1

Workarounds

You can disable the redis protocol in the network packet capture integration.

[andrewkroh]

Filebeat's spec currently filters by input type, but it happens too late. If that filter is moved earlier it should fix the issue. Ideally inputs types destined for Filebeat would follow naming convention like the other Beats (Metricbeat uses .*/metric, Auditbeat uses audit/.*, Packetbeat uses type: packet, etc.).

diff --git a/internal/spec/filebeat.yml b/internal/spec/filebeat.yml
index 10f8ee449..626d6e241 100644
--- a/internal/spec/filebeat.yml
+++ b/internal/spec/filebeat.yml
@@ -12,6 +12,42 @@ artifact: beats/filebeat
 restart_on_output_change: true
 rules:
 - fix_stream: {}
+
+- filter_values:
+    selector: inputs
+    key: type
+    values:
+      - aws-cloudwatch
+      - aws-s3
+      - azure-eventhub
+      - cloudfoundry
+      - container
+      - docker
+      - event/file
+      - event/stdin
+      - event/tcp
+      - event/udp
+      - filestream
+      - gcp-pubsub
+      - http_endpoint
+      - httpjson
+      - journald
+      - kafka
+      - log/docker
+      - log/redis_slowlog
+      - log/syslog
+      - logfile
+      - mqtt
+      - netflow
+      - o365audit
+      - redis
+      - stdin
+      - syslog
+      - tcp
+      - udp
+      - unix
+      - winlog
+
 - inject_index:
     type: logs
 
@@ -63,34 +99,6 @@ rules:
     - remove_key:
         key: data_stream.dataset
 
-- filter_values:
-    selector: inputs
-    key: type
-    values:
-    - aws-cloudwatch
-    - aws-s3
-    - azure-eventhub
-    - cloudfoundry
-    - container
-    - docker
-    - gcp-pubsub
-    - http_endpoint
-    - httpjson
-    - journald
-    - kafka
-    - log
-    - mqtt
-    - netflow
-    - o365audit
-    - redis
-    - stdin
-    - syslog
-    - tcp
-    - udp
-    - unix
-    - winlog
-    - filestream
-
 - filter_values:
     selector: inputs
     key: enabled

[jlind23]

@cmacknz it rather seems to be a data plane issue, am I wrong?

[cmacknz]

I don't know that anyone on the data plane team has touched the spec files before, but the plan for V2 is that we would own this so I think we can own the fix for this.

[cmacknz]

Pulling into 8.4 as we are getting multiple reports about this problem now.

[cmacknz]

@andrewkroh do you know which versions this is affecting? All of them? Or just recent 8.x releases? I'm wondering what backports we need.

We should likely also link this problem and the work around in the agent release notes.

[andrewkroh]

My assumption is that 7.17 and 8.x are affected because those are the stack versions supported by network_traffic (https://github.com/elastic/integrations/blob/21cf4a1e6d80e01ee29651f58ba5cb7455cdad0c/packages/network_traffic/manifest.yml#L12).