[clickhouse] Add attributes for event in ClickHouse storage #7512
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
mahadzaryab1
added
the
changelog:exprimental
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7512 +/- ##
==========================================
- Coverage 96.64% 96.62% -0.03%
==========================================
Files 377 378 +1
Lines 23145 23146 +1
==========================================
- Hits 22368 22364 -4
- Misses 592 596 +4
- Partials 185 186 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Metrics Comparison SummaryTotal changes across all snapshots: 53 Detailed changes per snapshotsummary_metrics_snapshot_cassandra📊 Metrics Diff SummaryTotal Changes: 53
❌ Removed Metrics
View diff sample-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
...View diff sample-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.005",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.01",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.025",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.05",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.075",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.1",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
...View diff sample-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.62.0",server_address="localhost",server_port="13133",url_scheme="http"}
... |
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
yurishkuro
reviewed
Sep 15, 2025
| } | ||
|
|
||
| func populateComplexAttributes(span ptrace.Span, complexAttributes []Attribute[string]) { | ||
| func populateComplexAttributes(span ptrace.Span, attributes pcommon.Map, complexAttributes []Attribute[string]) { |
Member
There was a problem hiding this comment.
why pass attributes explicitly? Aren't they part of span argument?
Member
There was a problem hiding this comment.
ah, I see why, span is for warnings. It should be in the last position as an argument, and maybe called spanForWarnings
Member
when will we be ready to integrate CH into db e2e tests? We can disable most of the tests for now, but it would be good to have CI going, rather than manual testing. |
Signed-off-by: Mahad Zaryab <[email protected]>
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
Comment on lines
247
to
251
| BoolAttributes: zipAttributes(boolAttributeKeys[i], boolAttributeValues[i]), | ||
| DoubleAttributes: zipAttributes(doubleAttributeKeys[i], doubleAttributeValues[i]), | ||
| IntAttributes: zipAttributes(intAttributeKeys[i], intAttributeValues[i]), | ||
| StrAttributes: zipAttributes(strAttributeKeys[i], strAttributeValues[i]), | ||
| ComplexAttributes: zipAttributes(complexAttributeKeys[i], complexAttributeValues[i]), |
Contributor
There was a problem hiding this comment.
Array bounds error: The code accesses event attribute arrays using index i without checking if the arrays have sufficient length. If any of the event attribute arrays (boolAttributeKeys, boolAttributeValues, etc.) have fewer elements than the number of events, this will cause an index out of bounds panic at runtime. Add bounds checking before accessing these arrays, or ensure all arrays have the same length as the events arrays.
Suggested change
| BoolAttributes: zipAttributes(boolAttributeKeys[i], boolAttributeValues[i]), | |
| DoubleAttributes: zipAttributes(doubleAttributeKeys[i], doubleAttributeValues[i]), | |
| IntAttributes: zipAttributes(intAttributeKeys[i], intAttributeValues[i]), | |
| StrAttributes: zipAttributes(strAttributeKeys[i], strAttributeValues[i]), | |
| ComplexAttributes: zipAttributes(complexAttributeKeys[i], complexAttributeValues[i]), | |
| BoolAttributes: safeZipAttributes(boolAttributeKeys, boolAttributeValues, i), | |
| DoubleAttributes: safeZipAttributes(doubleAttributeKeys, doubleAttributeValues, i), | |
| IntAttributes: safeZipAttributes(intAttributeKeys, intAttributeValues, i), | |
| StrAttributes: safeZipAttributes(strAttributeKeys, strAttributeValues, i), | |
| ComplexAttributes: safeZipAttributes(complexAttributeKeys, complexAttributeValues, i), |
Spotted by Diamond
Is this helpful? React 👍 or 👎 to let us know.
Member
There was a problem hiding this comment.
This is not a bad suggestion. If there is some hiccup in the storage / driver and we get malformed data back the server will panic, which is not good. I would add an extra function to check that all array sizes are correct.
Collaborator
Author
There was a problem hiding this comment.
I'm going to address this in a follow-up PR
Collaborator
Author
@yurishkuro I think we should be able to do so once we complete the Writer implementation |
yurishkuro
reviewed
Sep 16, 2025
| EventStrAttributeKeys [][]string | ||
| EventStrAttributeValues [][]string | ||
| EventComplexAttributeKeys [][]string | ||
| EventComplexAttributeValues [][]string |
Member
There was a problem hiding this comment.
This SpanRow struct repeats the same shape as an assortment of standalone variables in reader.go. Why not move the struct there, use it for loading, and reuse it here?
yurishkuro
reviewed
Sep 16, 2025
| ComplexAttributeValues []string | ||
| EventNames []string | ||
| EventTimestamps []time.Time | ||
| EventBoolAttributeKeys [][]string |
Member
There was a problem hiding this comment.
This group of fields should be its own struct because you will have to repeat them for Links.
yurishkuro
approved these changes
Sep 16, 2025
Member
Why don't we do that instead of going after more exotic data elements? I would start with having CI-tested read/write cycle for core fields, then incrementally add more fields. |
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Comment on lines
+162
to
+166
| BoolAttributes: zipAttributes(boolAttributeKeys[i], boolAttributeValues[i]), | ||
| DoubleAttributes: zipAttributes(doubleAttributeKeys[i], doubleAttributeValues[i]), | ||
| IntAttributes: zipAttributes(intAttributeKeys[i], intAttributeValues[i]), | ||
| StrAttributes: zipAttributes(strAttributeKeys[i], strAttributeValues[i]), | ||
| ComplexAttributes: zipAttributes(complexAttributeKeys[i], complexAttributeValues[i]), |
Contributor
There was a problem hiding this comment.
Array bounds vulnerability: The buildEvents function accesses event attribute arrays by index [i] without bounds checking. If the attribute arrays have fewer elements than the names/timestamps arrays, this will cause a panic with 'index out of range'. The function should validate that all attribute arrays have at least i+1 elements before accessing index [i], or use safe indexing with length checks.
Suggested change
| BoolAttributes: zipAttributes(boolAttributeKeys[i], boolAttributeValues[i]), | |
| DoubleAttributes: zipAttributes(doubleAttributeKeys[i], doubleAttributeValues[i]), | |
| IntAttributes: zipAttributes(intAttributeKeys[i], intAttributeValues[i]), | |
| StrAttributes: zipAttributes(strAttributeKeys[i], strAttributeValues[i]), | |
| ComplexAttributes: zipAttributes(complexAttributeKeys[i], complexAttributeValues[i]), | |
| BoolAttributes: safeZipAttributes(boolAttributeKeys, boolAttributeValues, i), | |
| DoubleAttributes: safeZipAttributes(doubleAttributeKeys, doubleAttributeValues, i), | |
| IntAttributes: safeZipAttributes(intAttributeKeys, intAttributeValues, i), | |
| StrAttributes: safeZipAttributes(strAttributeKeys, strAttributeValues, i), | |
| ComplexAttributes: safeZipAttributes(complexAttributeKeys, complexAttributeValues, i), |
Spotted by Diamond
Is this helpful? React 👍 or 👎 to let us know.
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
yurishkuro
approved these changes
Sep 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which problem is this PR solving?
Description of the changes
How was this change tested?
Started the ClickHouse server on my local machine using
Initialized the tables using
Seeded the database using test.sql
Wrote a Go unit test to run
GetTracesand got the following outputChecklist
jaeger:make lint testjaeger-ui:npm run lintandnpm run test