Skip to content

Conversation

kpollich
Copy link
Member

Reverts #212422

We don't want to surface this Beats-specific syntax for dynamic topics because it greatly harms our ability to migrate users to OTel.

The recommended way to handle complex multi-field interpolations for topic names is to use an add_field Beats processor to add the desired topic name as a distinct field, and then use that field for the dynamic topic name in the Fleet-managed Kafka output. I'll be filing a docs issue to get this in our public docs as well.

@kpollich kpollich added the Team:Fleet Team label for Observability Data Collection Fleet team label Apr 28, 2025
@kpollich kpollich self-assigned this Apr 28, 2025
@kpollich kpollich requested review from a team as code owners April 28, 2025 14:54
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@kpollich kpollich added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Apr 28, 2025
Copy link
Member

@nchaulet nchaulet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 That feature was never available to user right? user cannot create outputs in serverless and 9.1 is not yet released

@kpollich
Copy link
Member Author

Correct this feature was only ever in 9.1.0.

Copy link
Contributor

@kilfoyle kilfoyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the docs. 👍

@kpollich
Copy link
Member Author

@elasticmachine merge upstream

@kpollich kpollich enabled auto-merge (squash) April 28, 2025 18:53
@kpollich kpollich merged commit d12dfc0 into main Apr 28, 2025
10 checks passed
@kpollich kpollich deleted the revert-212422-fix-kafka-dynamic-topic-output branch April 28, 2025 20:19
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
aiAssistantManagementSelection 67.0KB 67.0KB +11.0B
fleet 1.7MB 1.7MB -546.0B
lists 113.7KB 113.7KB +11.0B
total -524.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
core 438.9KB 438.9KB +11.0B

cc @kpollich

akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request May 29, 2025
…#219415)

Reverts elastic#212422

We don't want to surface this Beats-specific syntax for dynamic topics
because it greatly harms our ability to migrate users to OTel.

The recommended way to handle complex multi-field interpolations for
topic names is to use an `add_field` Beats processor to add the desired
topic name as a distinct field, and then use that field for the dynamic
topic name in the Fleet-managed Kafka output. I'll be filing a docs
issue to get this in our public docs as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants