You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[receiver/sqlserver] Support query-level log collection [Top Query] (open-telemetry#37958)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
This PR introduced query-level data collection for `sqlserver` receiver
in the logs pipeline.
We introduced `Top Query` collection in this initial PR (`Top Queries`
are those queries which used the most CPU time within a time window)
Co-authored-by: @cuichenli (CLA signed)
##### Configuration
We introduced these four configurations for the feature(see receiver's
README for details):
1. `logs.enable_top_query_collection` to enable the collection
2. `max_sample_query_count`: the initial query count to fetch from
database.
3. `top_query_count`: the number to report to the next consumer.
4. `lookback_time`: the query window for each scrape.
##### Workflow
The `sqlserver` receiver will fetch M(=`max_sample_query_count`) queries
from database and sort the queries according to the difference of
`total_elapsed_time`(CPU time used), and then report the first
N(=`top_query_count`) queries.
##### New Log Attributes
- `db.total_elapsed_time`
- `db.total_rows`
- `db.total_worker_time`
- `db.total_logical_reads`
- `db.total_logical_writes`
- `db.total_physical_reads`
- `db.execution_count`
- `db.total_grant_kb`
- `db.query_hash`
- `db.query_plan_hash`
- `db.query_text`
- `db.query_plan`
##### Additional dependency
* `hashicorp/golang-lru/v2`
* License: MPL-2.0
* Link: https://pkg.go.dev/github.com/hashicorp/golang-lru/v2
* Already been used in the repo
* `DataDog/datadog-agent/pkg/obfuscate`
* License: Apache 2.0
* Link:
https://pkg.go.dev/github.com/DataDog/datadog-agent/pkg/obfuscate
* Already been used in the repo
##### Example Output
```
resourceLogs:
- resource:
attributes:
- key: db.system.type
value:
stringValue: microsoft.sql_server
scopeLogs:
- logRecords:
- attributes:
- key: computer_name
value:
stringValue: DESKTOP-GHAEGRD
- key: sql_instance
value:
stringValue: sqlserver
- key: db.query_hash
value:
stringValue: "307833373834394538373431373145334633"
- key: db.query_plan_hash
value:
stringValue: "307844333131323930393432394131423530"
- key: db.total_elapsed_time
value:
intValue: "2"
- key: db.total_rows
value:
intValue: "1"
- key: db.total_logical_reads
value:
intValue: "2"
- key: db.total_logical_writes
value:
intValue: "3"
- key: db.total_physical_reads
value:
intValue: "4"
- key: db.execution_count
value:
intValue: "5"
- key: db.total_worker_time
value:
intValue: "2"
- key: db.total_grant_kb
value:
intValue: "3095"
- key: db.query_text
value:
stringValue: with qstats SELECT TOP ( @topNValue ) REPLACE ( @@ServerName, ? ) ...
- key: db.query_plan
value:
stringValue: <ShowPlanXML xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan"> ...
body: {}
spanId: ""
timeUnixNano: "1739690452135336000"
traceId: ""
scope:
name: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/sqlserverreceiver
version: development
```
<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Part of open-telemetry#36462
<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added
<!--Describe the documentation added.-->
#### Documentation
Updated
<!--Please delete paragraphs that you did not use before submitting.-->
---------
Co-authored-by: Will Li <[email protected]>
Co-authored-by: Antoine Toulme <[email protected]>
The `sqlserver` receiver grabs metrics about a Microsoft SQL Server instance. The receiver works by either using the
17
+
The `sqlserver` receiver grabs metrics/logs about a Microsoft SQL Server instance. The receiver works by either using the
16
18
Windows Performance Counters, or by directly connecting to the instance and querying it. Windows Performance Counters
17
19
are only available when running on Windows.
18
20
@@ -36,6 +38,14 @@ Windows-specific options:
36
38
-`computer_name` (optional): The computer name identifies the SQL Server name or IP address of the computer being monitored.
37
39
If specified, `instance_name` is also required to be defined. This option is ignored in non-Windows environments.
38
40
41
+
Top-Query collection specific options (only useful when top-query collection are enabled):
42
+
-`lookback_time` (optional, example = `60`, default = `2 * collection_interval`): The time window (in second) in which to query for top queries.
43
+
- Queries that were finished execution outside the lookback window are not included in the collection. Increasing the lookback window (in seconds) will be useful for capturing long-running queries.
44
+
-`max_query_sample_count` (optional, example = `5000`, default = `1000`): The maximum number of records to fetch in a single run.
45
+
-`top_query_count`: (optional, example = `100`, default = `200`): The maximum number of active queries to report (to the next consumer) in a single run.
46
+
-`enabled`: (optional, default = `false`): Enable collection of top queries.
47
+
- e.g. `sqlserver` receiver will fetch 1000 (value: `max_query_sample_count`) queries from database and report the top 200 (value: `top_query_count`) which used the most CPU time.
48
+
39
49
Example:
40
50
41
51
```yaml
@@ -50,7 +60,7 @@ Example:
50
60
port: 1433
51
61
```
52
62
53
-
When a named instance is used on Windows, a computer name and a instance name must be specified.
63
+
When a named instance is used on Windows, a computer name and an instance name must be specified.
54
64
Example with named instance:
55
65
56
66
```yaml
@@ -68,10 +78,31 @@ Example with named instance:
68
78
69
79
The full list of settings exposed for this receiver are documented in [config.go](./config.go) with detailed sample configurations in [testdata/config.yaml](./testdata/config.yaml).
70
80
81
+
Top query collection enabled:
82
+
```yaml
83
+
receivers:
84
+
sqlserver:
85
+
collection_interval: 5s
86
+
username: sa
87
+
password: securepassword
88
+
server: 0.0.0.0
89
+
port: 1433
90
+
top_query_collection:
91
+
enabled: true
92
+
lookback_time: 60
93
+
max_query_sample_count: 1000
94
+
top_query_count: 200
95
+
96
+
```
71
97
## Metrics
72
98
73
99
Details about the metrics produced by this receiver can be found in [documentation.md](./documentation.md)
74
100
101
+
102
+
## Logs
103
+
104
+
Details about the logs produced by this receiver can be found in [logs-documentation.md](./logs-documentation.md)
105
+
75
106
## Known issues
76
107
SQL Server docker users may run into an issue that the collector fails to parse certificate from server due to `x509: negative serial number`. That's because we adopted Go `1.23` starting from contrib `v0.121.0`:
77
108
> Before Go 1.23, ParseCertificate accepted certificates with negative serial numbers.
0 commit comments