Skip to content

[receiver/postgresql] add basic query sample collection #39311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Apr 14, 2025
Merged

[receiver/postgresql] add basic query sample collection #39311

merged 3 commits into from
Apr 14, 2025

Conversation

cuichenli
Copy link
Contributor

Description

we introduced the query sample collection to postgresql receiver. this will collect the queries from pg_stat_activity and report the related metrics. this will help the end user to identify the queries were executing in database.

example output:

resourceLogs:
  - resource:
      attributes: []
    scopeLogs:
      - logRecords:
          - attributes:
              - key: user.name
                value:
                  stringValue: otelu
              - key: postgresql.state
                value:
                  stringValue: idle
              - key: postgresql.pid
                value:
                  intValue: 1450
              - key: postgresql.application_name
                value:
                  stringValue: receiver
              - key: db.system.name
                value:
                  stringValue: postgresql
              - key: db.namespace
                value:
                  stringValue: postgres
              - key: network.peer.address
                value:
                  stringValue: "11.4.5.14"
              - key: network.peer.port
                value:
                  intValue: 114514
              - key: postgresql.client_hostname
                value:
                  stringValue: otel
              - key: postgresql.query_start
                value:
                  stringValue: 2025-02-12T16:37:54.843+08:00
              - key: postgresql.wait_event
                value:
                  stringValue: ""
              - key: postgresql.wait_event_type
                value:
                  stringValue: ""
              - key: postgresql.query_id
                value:
                  stringValue: "123131231231"
              - key: db.query.text
                value:
                  stringValue: "select * from pg_stat_activity where id = ?"
            body:
              stringValue: sample
        scope:
          name: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver
          version: v0.0.1

Link to tracking issue

Fixes

Testing

Documentation

@cuichenli cuichenli requested a review from a team as a code owner April 11, 2025 01:54
@cuichenli cuichenli requested a review from atoulme April 11, 2025 01:54
@cuichenli cuichenli changed the title add basic query sample collection for postgresqlreceiver [receiver/postgresql] add basic query sample collection Apr 11, 2025
@cuichenli cuichenli force-pushed the postgresql-sample-query branch 2 times, most recently from 168878f to 7781ea0 Compare April 12, 2025 02:42
@cuichenli cuichenli force-pushed the postgresql-sample-query branch from 7781ea0 to 543872d Compare April 12, 2025 03:25
atoulme
atoulme reviewed Apr 14, 2025
View reviewed changes
receiver/postgresqlreceiver/README.md
| Distributions | [contrib] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fpostgresql%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fpostgresql) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fpostgresql%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fpostgresql) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | \| Seeking more code owners! |
| Emeritus | [@djaglowski](https://www.github.com/djaglowski) |

[unmaintained]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#unmaintained
[development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might like to make yourself a codeowner of this component. Separate PR please.

atoulme
atoulme approved these changes Apr 14, 2025
View reviewed changes
Copy link
Contributor

@atoulme atoulme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is brand new functionality, lgtm though I'm no expert. OK to merge under development label.

@atoulme
Copy link
Contributor

atoulme commented Apr 14, 2025

Given that postgresqlreceiver is unmaintained, there are no codeowners to review the changes. I tend to lean towards letting code that is clearly labeled in development in as I prefer rapid iteration. I will merge this change now.

@atoulme atoulme merged commit 685f2b3 into open-telemetry:main Apr 14, 2025
171 checks passed
@github-actions github-actions bot added this to the next release milestone Apr 14, 2025
akshays-19 pushed a commit to akshays-19/opentelemetry-collector-contrib that referenced this pull request Apr 23, 2025
@cuichenli @AkshayS198
[receiver/postgresql] add basic query sample collection (open-telemet…
dc1f82b 
…ry#39311)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
we introduced the query sample collection to postgresql receiver. this
will collect the queries from `pg_stat_activity` and report the related
metrics. this will help the end user to identify the queries were
executing in database.

example output:
```
resourceLogs:
  - resource:
      attributes: []
    scopeLogs:
      - logRecords:
          - attributes:
              - key: user.name
                value:
                  stringValue: otelu
              - key: postgresql.state
                value:
                  stringValue: idle
              - key: postgresql.pid
                value:
                  intValue: 1450
              - key: postgresql.application_name
                value:
                  stringValue: receiver
              - key: db.system.name
                value:
                  stringValue: postgresql
              - key: db.namespace
                value:
                  stringValue: postgres
              - key: network.peer.address
                value:
                  stringValue: "11.4.5.14"
              - key: network.peer.port
                value:
                  intValue: 114514
              - key: postgresql.client_hostname
                value:
                  stringValue: otel
              - key: postgresql.query_start
                value:
                  stringValue: 2025-02-12T16:37:54.843+08:00
              - key: postgresql.wait_event
                value:
                  stringValue: ""
              - key: postgresql.wait_event_type
                value:
                  stringValue: ""
              - key: postgresql.query_id
                value:
                  stringValue: "123131231231"
              - key: db.query.text
                value:
                  stringValue: "select * from pg_stat_activity where id = ?"
            body:
              stringValue: sample
        scope:
          name: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver
          version: v0.0.1
```
<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
Fiery-Fenix pushed a commit to Fiery-Fenix/opentelemetry-collector-contrib that referenced this pull request Apr 24, 2025
@cuichenli @Fiery-Fenix
[receiver/postgresql] add basic query sample collection (open-telemet…
9784e99 
…ry#39311)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
we introduced the query sample collection to postgresql receiver. this
will collect the queries from `pg_stat_activity` and report the related
metrics. this will help the end user to identify the queries were
executing in database.

example output:
```
resourceLogs:
  - resource:
      attributes: []
    scopeLogs:
      - logRecords:
          - attributes:
              - key: user.name
                value:
                  stringValue: otelu
              - key: postgresql.state
                value:
                  stringValue: idle
              - key: postgresql.pid
                value:
                  intValue: 1450
              - key: postgresql.application_name
                value:
                  stringValue: receiver
              - key: db.system.name
                value:
                  stringValue: postgresql
              - key: db.namespace
                value:
                  stringValue: postgres
              - key: network.peer.address
                value:
                  stringValue: "11.4.5.14"
              - key: network.peer.port
                value:
                  intValue: 114514
              - key: postgresql.client_hostname
                value:
                  stringValue: otel
              - key: postgresql.query_start
                value:
                  stringValue: 2025-02-12T16:37:54.843+08:00
              - key: postgresql.wait_event
                value:
                  stringValue: ""
              - key: postgresql.wait_event_type
                value:
                  stringValue: ""
              - key: postgresql.query_id
                value:
                  stringValue: "123131231231"
              - key: db.query.text
                value:
                  stringValue: "select * from pg_stat_activity where id = ?"
            body:
              stringValue: sample
        scope:
          name: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver
          version: v0.0.1
```
<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
@nozomi-gedimitr nozomi-gedimitr mentioned this pull request May 7, 2025
Merged
mx-psi pushed a commit that referenced this pull request May 7, 2025
@nozomi-gedimitr
Finer grained driver loading in SQL receivers (#39918)
2ab1180 
#### Description

Between 0.123.0 and 0.124.0, the footprint of the `postgresql` receiver
increased sharply when participating in custom builds. As an experiment,
builds using the configuration below were done for both 0.123.0 and
0.124.0 releases in MacOS.

```
dist:
  name: otelcol-custom
  description: Local OpenTelemetry Collector binary
  output_path: /tmp/dist

exporters:
  - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.124.0

receivers:
  - gomod:  github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver v0.124.0

processors:
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.124.0
```

The generated binary size increased significantly
* 0.123.0 --> 25 MB
* 0.124.0 --> 63 MB

The problem was found to be the recent inclusion of the internal
`sqlquery` package (PR #39311): this package registers drivers for all
kinds of databases, and thus the `postgresql` receiver got many new
(unneeded) dependencies.

With this PR, the `internal/sqlquery` package no longer registers
drivers for all possible databases: instead this task is expected to be
done by the actual users of the package, who know best what drivers they
are going to need.

Apart from the `postgresql` receiver, the `sqlserver` one got its
dependencies accordingly trimmed.
dragonlord93 pushed a commit to dragonlord93/opentelemetry-collector-contrib that referenced this pull request May 23, 2025
@nozomi-gedimitr @dragonlord93
Finer grained driver loading in SQL receivers (open-telemetry#39918)
bc71a90 
#### Description

Between 0.123.0 and 0.124.0, the footprint of the `postgresql` receiver
increased sharply when participating in custom builds. As an experiment,
builds using the configuration below were done for both 0.123.0 and
0.124.0 releases in MacOS.

```
dist:
  name: otelcol-custom
  description: Local OpenTelemetry Collector binary
  output_path: /tmp/dist

exporters:
  - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.124.0

receivers:
  - gomod:  github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver v0.124.0

processors:
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.124.0
```

The generated binary size increased significantly
* 0.123.0 --> 25 MB
* 0.124.0 --> 63 MB

The problem was found to be the recent inclusion of the internal
`sqlquery` package (PR open-telemetry#39311): this package registers drivers for all
kinds of databases, and thus the `postgresql` receiver got many new
(unneeded) dependencies.

With this PR, the `internal/sqlquery` package no longer registers
drivers for all possible databases: instead this task is expected to be
done by the actual users of the package, who know best what drivers they
are going to need.

Apart from the `postgresql` receiver, the `sqlserver` one got its
dependencies accordingly trimmed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@atoulme atoulme atoulme approved these changes

Assignees

@songy23 songy23

Labels
receiver/postgresql
Projects
None yet
Milestone
next release
Development

Successfully merging this pull request may close these issues.
3 participants
@cuichenli @atoulme @songy23