Skip to content

Commit 2ab1180

Browse files
Finer grained driver loading in SQL receivers (#39918)
#### Description Between 0.123.0 and 0.124.0, the footprint of the `postgresql` receiver increased sharply when participating in custom builds. As an experiment, builds using the configuration below were done for both 0.123.0 and 0.124.0 releases in MacOS. ``` dist: name: otelcol-custom description: Local OpenTelemetry Collector binary output_path: /tmp/dist exporters: - gomod: go.opentelemetry.io/collector/exporter/debugexporter v0.124.0 receivers: - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/postgresqlreceiver v0.124.0 processors: - gomod: go.opentelemetry.io/collector/processor/batchprocessor v0.124.0 ``` The generated binary size increased significantly * 0.123.0 --> 25 MB * 0.124.0 --> 63 MB The problem was found to be the recent inclusion of the internal `sqlquery` package (PR #39311): this package registers drivers for all kinds of databases, and thus the `postgresql` receiver got many new (unneeded) dependencies. With this PR, the `internal/sqlquery` package no longer registers drivers for all possible databases: instead this task is expected to be done by the actual users of the package, who know best what drivers they are going to need. Apart from the `postgresql` receiver, the `sqlserver` one got its dependencies accordingly trimmed.
1 parent c562518 commit 2ab1180

File tree

12 files changed

+59
-748
lines changed

12 files changed

+59
-748
lines changed
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
7+
component: postgresqlreceiver
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Reduce component footprint by removing the loading of unnecessary SQL drivers
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [39918]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext: |
19+
Custom builds that are using the postgresql receiver and no other SQL related receivers
20+
will see a reduction in the output binary size. A similar effect is expected also for
21+
the sqlserverreceiver.
22+
23+
# If your change doesn't affect end users or the exported elements of any package,
24+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
25+
# Optional: The change log or logs in which this entry should be included.
26+
# e.g. '[user]' or '[user, api]'
27+
# Include 'user' if the change is relevant to end users.
28+
# Include 'api' if there is a change to a library API.
29+
# Default: '[user]'
30+
change_logs: [user]

internal/sqlquery/db_client.go

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,7 @@ import (
77
"context"
88
"errors"
99

10-
// register Db drivers
11-
_ "github.com/SAP/go-hdb/driver"
12-
_ "github.com/go-sql-driver/mysql"
13-
_ "github.com/lib/pq"
14-
_ "github.com/microsoft/go-mssqldb"
15-
_ "github.com/microsoft/go-mssqldb/integratedauth/krb5"
16-
_ "github.com/sijms/go-ora/v2"
17-
_ "github.com/snowflakedb/gosnowflake"
18-
_ "github.com/thda/tds"
10+
// Do not register any Db drivers here: users should register the ones that are applicable to them.
1911
"go.uber.org/zap"
2012
)
2113

internal/sqlquery/go.mod

Lines changed: 0 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,7 @@ module github.com/open-telemetry/opentelemetry-collector-contrib/internal/sqlque
33
go 1.23.0
44

55
require (
6-
github.com/SAP/go-hdb v1.13.5
7-
github.com/go-sql-driver/mysql v1.9.2
8-
github.com/lib/pq v1.10.9
9-
github.com/microsoft/go-mssqldb v1.8.0
10-
github.com/sijms/go-ora/v2 v2.8.24
11-
github.com/snowflakedb/gosnowflake v1.13.3
126
github.com/stretchr/testify v1.10.0
13-
github.com/thda/tds v0.1.7
147
go.opentelemetry.io/collector/component v1.31.1-0.20250505152726-56c7da210783
158
go.opentelemetry.io/collector/component/componenttest v0.125.1-0.20250505155216-829157cef7bb
169
go.opentelemetry.io/collector/pdata v1.31.1-0.20250505152726-56c7da210783
@@ -21,63 +14,16 @@ require (
2114
)
2215

2316
require (
24-
filippo.io/edwards25519 v1.1.0 // indirect
25-
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
26-
github.com/99designs/keyring v1.2.2 // indirect
27-
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1 // indirect
28-
github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 // indirect
29-
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0 // indirect
30-
github.com/BurntSushi/toml v1.4.0 // indirect
31-
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c // indirect
32-
github.com/apache/arrow-go/v18 v18.0.0 // indirect
33-
github.com/aws/aws-sdk-go-v2 v1.26.1 // indirect
34-
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.2 // indirect
35-
github.com/aws/aws-sdk-go-v2/credentials v1.17.11 // indirect
36-
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.15 // indirect
37-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.5 // indirect
38-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.5 // indirect
39-
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.5 // indirect
40-
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 // indirect
41-
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.3.7 // indirect
42-
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.7 // indirect
43-
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.17.5 // indirect
44-
github.com/aws/aws-sdk-go-v2/service/s3 v1.53.1 // indirect
45-
github.com/aws/smithy-go v1.20.2 // indirect
46-
github.com/danieljoos/wincred v1.2.2 // indirect
4717
github.com/davecgh/go-spew v1.1.1 // indirect
48-
github.com/dvsekhvalnov/jose2go v1.6.0 // indirect
49-
github.com/gabriel-vasile/mimetype v1.4.7 // indirect
5018
github.com/go-logr/logr v1.4.2 // indirect
5119
github.com/go-logr/stdr v1.2.2 // indirect
52-
github.com/goccy/go-json v0.10.4 // indirect
53-
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
5420
github.com/gogo/protobuf v1.3.2 // indirect
55-
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
56-
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect
57-
github.com/golang-sql/sqlexp v0.1.0 // indirect
58-
github.com/google/flatbuffers v24.12.23+incompatible // indirect
5921
github.com/google/uuid v1.6.0 // indirect
60-
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect
61-
github.com/hashicorp/go-uuid v1.0.3 // indirect
6222
github.com/hashicorp/go-version v1.7.0 // indirect
63-
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
64-
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
65-
github.com/jcmturner/gofork v1.7.6 // indirect
66-
github.com/jcmturner/goidentity/v6 v6.0.1 // indirect
67-
github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect
68-
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
69-
github.com/jmespath/go-jmespath v0.4.0 // indirect
7023
github.com/json-iterator/go v1.1.12 // indirect
71-
github.com/klauspost/compress v1.17.11 // indirect
72-
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
7324
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
7425
github.com/modern-go/reflect2 v1.0.2 // indirect
75-
github.com/mtibben/percent v0.2.1 // indirect
76-
github.com/pierrec/lz4/v4 v4.1.22 // indirect
77-
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
7826
github.com/pmezard/go-difflib v1.0.0 // indirect
79-
github.com/sirupsen/logrus v1.9.3 // indirect
80-
github.com/zeebo/xxh3 v1.0.2 // indirect
8127
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
8228
go.opentelemetry.io/collector/consumer v1.31.1-0.20250505152726-56c7da210783 // indirect
8329
go.opentelemetry.io/collector/featuregate v1.31.1-0.20250505152726-56c7da210783 // indirect
@@ -92,17 +38,9 @@ require (
9238
go.opentelemetry.io/otel/sdk v1.35.0 // indirect
9339
go.opentelemetry.io/otel/sdk/metric v1.35.0 // indirect
9440
go.opentelemetry.io/otel/trace v1.35.0 // indirect
95-
golang.org/x/crypto v0.37.0 // indirect
96-
golang.org/x/exp v0.0.0-20240909161429-701f63a606c0 // indirect
97-
golang.org/x/mod v0.22.0 // indirect
9841
golang.org/x/net v0.39.0 // indirect
99-
golang.org/x/oauth2 v0.26.0 // indirect
100-
golang.org/x/sync v0.13.0 // indirect
10142
golang.org/x/sys v0.32.0 // indirect
102-
golang.org/x/term v0.31.0 // indirect
10343
golang.org/x/text v0.24.0 // indirect
104-
golang.org/x/tools v0.29.0 // indirect
105-
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
10644
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a // indirect
10745
google.golang.org/grpc v1.72.0 // indirect
10846
google.golang.org/protobuf v1.36.6 // indirect

0 commit comments

Comments
 (0)