Skip to content

Conversation

mykola-elastic
Copy link
Contributor

@mykola-elastic mykola-elastic commented Jul 30, 2025

What does this PR do?

Receivers added:

  • mysqlreceiver
  • apachereceiver
  • iisreceiver
  • postgresqlreceiver

Why is it important?

Needed in order to add OTEL content packs to elastic/integrations which use the data from the receivers listed

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

Related issues

Copy link
Contributor

mergify bot commented Jul 30, 2025

This pull request does not have a backport label. Could you fix it @mykola-elastic? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

@mykola-elastic mykola-elastic changed the title [DRAFT] [OTEL] Add receivers [OTEL] Add receivers Jul 30, 2025
@mykola-elastic mykola-elastic changed the title [OTEL] Add receivers [OTEL] Add receivers: apache, iis, mysql, postgresql, sqlquery Jul 30, 2025
@mykola-elastic mykola-elastic changed the title [OTEL] Add receivers: apache, iis, mysql, postgresql, sqlquery [otel] add receivers: apache, iis, mysql, postgresql, sqlquery Jul 30, 2025
@mykola-elastic mykola-elastic marked this pull request as ready for review July 30, 2025 12:06
@mykola-elastic mykola-elastic requested a review from a team as a code owner July 30, 2025 12:06
@mykola-elastic
Copy link
Contributor Author

mykola-elastic commented Jul 30, 2025

It is very weird that unit tests failed on windows (ubuntu and macOS passed):

    run_test.go:49:
        	Error Trace:	C:/buildkite-agent/builds/bk-agent-prod-gcp-1753872573361947882/elastic/elastic-agent/internal/pkg/otel/run_test.go:49
        	            				C:/Users/Buildkite/.go/go-1.24.5/src/runtime/asm_amd64.s:1700
        	Error:      	Received unexpected error:
        	            	failed to build pipelines: connector "spanmetrics" used as exporter in [traces] pipeline but not used in any supported receiver pipeline
        	Test:       	TestStartCollector/all-components.yml
    run_nofips_test.go:46:
        	Error Trace:	C:/buildkite-agent/builds/bk-agent-prod-gcp-1753872573361947882/elastic/elastic-agent/internal/pkg/otel/run_nofips_test.go:46
        	Error:      	Condition never satisfied
        	Test:       	TestStartCollector/all-components.yml

However, there is no problem with spanmetrics. It is used in the [metrics] pipeline as a receiver

Update: seems the reason is that all-components.windows.yml is overriding all-components.yml, specifically when I added the metrics pipeline. Renamed it to metrics/windows-only

@swiatekm
Copy link
Contributor

@mykola-elastic could you build the agent binary on this branch and compare it to main? We don't currently have tests for this, and I'd like to see what the impact of adding these dependencies is. Just running mage build should work.

@mykola-elastic
Copy link
Contributor Author

mykola-elastic commented Jul 30, 2025

@swiatekm
Building on arm Mac:

  • this branch: 422M
  • main: 392M
    (checked with ls -lh ./build/elastic-agent)

@mykola-elastic
Copy link
Contributor Author

I initially added iis receiver to all-components.windows.yml, however, I removed it because it broke unit tests. It requires a different windows environment configuration. Seems like "Windows Performance Counters." are unavailable or not enough permissions to access them as per the error message:

cannot start pipelines: failed to start "iis" receiver: failed to create perf counter with path \Web Service(*)\Total Put Requests: The specified object was not found on the computer.

@ishleenk17
Copy link
Member

I initially added iis receiver to all-components.windows.yml, however, I removed it because it broke unit tests. It requires a different windows environment configuration. Seems like "Windows Performance Counters." are unavailable or not enough permissions to access them as per the error message:

cannot start pipelines: failed to start "iis" receiver: failed to create perf counter with path \Web Service(*)\Total Put Requests: The specified object was not found on the computer.

Mykola does the CI pass without the IIS receiver addition ?

@mykola-elastic
Copy link
Contributor Author

@ishleenk17 it is still in progress but probably will pass, it needs another hour or so

ishleenk17
ishleenk17 previously approved these changes Jul 31, 2025
Copy link
Member

@ishleenk17 ishleenk17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@ishleenk17 ishleenk17 added the backport-9.1 Automated backport to the 9.1 branch label Jul 31, 2025
@strawgate
Copy link
Contributor

Id like to understand why we are adding SQLquery at this point

@ishleenk17
Copy link
Member

Id like to understand why we are adding SQLquery at this point

Having the generic SQLQuery receiver similar to SQL input module will be beneficial if we have to build custom dashboards by giving the custom SQL queries irrespective of the database.

cmacknz
cmacknz previously approved these changes Aug 1, 2025
Copy link
Member

@cmacknz cmacknz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, assuming @strawgate can accept the inclusion of the sqlquery receiver

@mykola-elastic mykola-elastic dismissed stale reviews from cmacknz and ishleenk17 via 020b766 August 2, 2025 06:12
@mykola-elastic mykola-elastic changed the title [otel] add receivers: apache, iis, mysql, postgresql, sqlquery [otel] add receivers: apache, iis, mysql, postgresql Aug 2, 2025
@mykola-elastic
Copy link
Contributor Author

@ishleenk17 @strawgate removed sqlqueryreceiver

Copy link

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @mykola-elastic

@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Aug 5, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@mykola-elastic
Copy link
Contributor Author

@ishleenk17 I've create a different PR with merge conflicts resolved: #9344
It was safer to resolve conflicts by redoing the changes because it relates to go.mod, go.sum

@mykola-elastic
Copy link
Contributor Author

Closing in favor of #9344

@mykola-elastic mykola-elastic changed the title [otel] add receivers: apache, iis, mysql, postgresql [otel] add receivers: apache, iis, mysql, postgresql v0.129.0 Aug 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-9.1 Automated backport to the 9.1 branch Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants