[Flaky test] filebeat/input/journald TestDoubleStarCanBeUsed

[AndersonQ]

Flaky Test

• Test Name: TestDoubleStarCanBeUsed
• Link: filebeat/input/journald/input_test.go#L471-L513
• Branch: main, 9.0, 9.2
• Artifact Link:
  • 9.2 https://buildkite.com/elastic/filebeat/builds/23987/steps/canvas?jid=0199a11c-8ae2-47fa-89de-073191865eca
  • 9.0 https://buildkite.com/elastic/filebeat/builds/23976/steps/canvas?jid=0199a062-65d0-461e-85ca-c3b0056f27a4
• Notes: it's flaky, running it 100 times, it fails a few times with different numbers of events. Perhaps it just needs a bigger timeout.

However, only the failing tests take a long time. So there might be something else happening there

can be reproduced with:

go test -v -count 100 -run TestDoubleStarCanBeUsed ./filebeat/input/journald

go test -v -count 100 -run TestDoubleStarCanBeUsed ./filebeat/input/journald 
=== RUN   TestDoubleStarCanBeUsed
--- PASS: TestDoubleStarCanBeUsed (0.04s)
=== RUN   TestDoubleStarCanBeUsed
--- PASS: TestDoubleStarCanBeUsed (0.04s)
=== RUN   TestDoubleStarCanBeUsed
--- PASS: TestDoubleStarCanBeUsed (0.03s)
=== RUN   TestDoubleStarCanBeUsed
    input_test.go:512: 
        	Error Trace:	/home/ainsoph/devel/github.com/elastic/beats-main/filebeat/input/journald/environment_test.go:124
        	            				/home/ainsoph/devel/github.com/elastic/beats-main/filebeat/input/journald/input_test.go:512
        	Error:      	Condition never satisfied
        	Test:       	TestDoubleStarCanBeUsed
        	Messages:   	too few events; expected: 30, actual: 21
--- FAIL: TestDoubleStarCanBeUsed (5.02s)
=== RUN   TestDoubleStarCanBeUsed
--- PASS: TestDoubleStarCanBeUsed (0.03s)
=== RUN   TestDoubleStarCanBeUsed
--- PASS: TestDoubleStarCanBeUsed (0.03s)
=== RUN   TestDoubleStarCanBeUsed
--- PASS: TestDoubleStarCanBeUsed (0.04s)
=== RUN   TestDoubleStarCanBeUsed

Stack Trace

2025-10-01 18:54:02 UTC
=== FAIL: filebeat/input/journald TestDoubleStarCanBeUsed (5.03s)
2025-10-01 18:54:02 UTC
    input_test.go:512:
2025-10-01 18:54:02 UTC
        	Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-gcp-1759344649138934621/elastic/filebeat/filebeat/input/journald/environment_test.go:124
2025-10-01 18:54:02 UTC
        	            				/opt/buildkite-agent/builds/bk-agent-prod-gcp-1759344649138934621/elastic/filebeat/filebeat/input/journald/input_test.go:512
2025-10-01 18:54:02 UTC
        	Error:      	Condition never satisfied
2025-10-01 18:54:02 UTC
        	Test:       	TestDoubleStarCanBeUsed
2025-10-01 18:54:02 UTC
        	Messages:   	too few events; expected: 30, actual: 16```

[belimawr]

I've just run this test 6000+ times and it passed every single time, taking 0.02s every iteration:

tiago@millennium-falcon beats/filebeat  v1.25.1 🐍 v3.9.13  main [$✔] 
% go test -count 1000 -run TestDoubleStarCanBeUsed ./input/journald
ok      github.com/elastic/beats/v7/filebeat/input/journald     22.377s
tiago@millennium-falcon beats/filebeat  v1.25.1 🐍 v3.9.13  main [$✔] 
% go test -count 5000 -run TestDoubleStarCanBeUsed ./input/journald
ok      github.com/elastic/beats/v7/filebeat/input/journald     118.364s
tiago@millennium-falcon beats/filebeat  v1.25.1 🐍 v3.9.13  main [$✔] 
% 

The flakiness is likely related to the host, I'll try on a VM CI to see if I can reproduce the failure.

[andrzej-stencel]

I saw this in CI too yesterday:

• https://buildkite.com/elastic/filebeat/builds/23957#01999f84-2c4d-4611-93bc-6911c198204b
• https://buildkite.com/elastic/filebeat/builds/23956#01999f7d-e5cc-4bbc-b8b8-655e4226d6d1
• https://buildkite.com/elastic/filebeat/builds/23956#0199a41f-8587-4710-8544-ebf122ca645e

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[belimawr]

The flakiness happens when we're reading small files and the jouranlctl process exists before we finish reading its stdout, plus the fact we're using 3 hand crafted journal files, which seems to make journalctl to return a somehow inaccurate cursor.

I'll create a PR loosening the conditions, which should make it always pass.

[belimawr]

The PR fixing the flakiness: #46913