[processor/routing] Collector crash #26462
Assignees
Labels
Comments
jpkrohling
added
bug
|
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
jpkrohling
added
processor/routing
jpkrohling
changed the title
jpkrohling
added a commit
to jpkrohling/opentelemetry-collector-contrib
that referenced
this issue
Sep 5, 2023
This PR changes the routing processor so that it properly creates new instances of pdata.Span and InstrumentationScope, avoiding crashes later on due to nil pointers. Fixes open-telemetry#26462 Signed-off-by: Juraci Paixão Kröhling <[email protected]>
This was referenced Sep 5, 2023
jpkrohling
added a commit
that referenced
this issue
Sep 6, 2023
This PR changes the routing processor so that it properly creates new instances of pdata.Span and InstrumentationScope, avoiding crashes later on due to nil pointers. Fixes #26462 Signed-off-by: Juraci Paixão Kröhling <[email protected]> --------- Signed-off-by: Juraci Paixão Kröhling <[email protected]>
This was referenced Sep 12, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Component(s)
processor/routing
What happened?
Description
Collector crash, caused by misuse of the ptrace.Span construct by the routing processor.
Steps to Reproduce
telemetrygen traces --traces 10000 --otlp-insecure --otlp-attributes='attr1="demo",attr2="xxx"'Expected Result
No crashes.
Actual Result
Collector version
0.84.0
Environment information
Environment
OS: Fedora release 38 (Thirty Eight)
OpenTelemetry Collector configuration
Log output
2023-09-05T07:02:35.045-0300 info service/telemetry.go:84 Setting up own telemetry... 2023-09-05T07:02:35.045-0300 info service/telemetry.go:201 Serving Prometheus metrics {"address": ":8888", "level": "Basic"} 2023-09-05T07:02:35.045-0300 info [email protected]/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "logging/route-2"} 2023-09-05T07:02:35.045-0300 info [email protected]/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "logging"} 2023-09-05T07:02:35.045-0300 info [email protected]/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "logging/route-1"} 2023-09-05T07:02:35.047-0300 info service/service.go:138 Starting otelcol-contrib... {"Version": "0.84.0", "NumCPU": 16} 2023-09-05T07:02:35.047-0300 info extensions/extensions.go:31 Starting extensions... 2023-09-05T07:02:35.048-0300 warn [email protected]/warning.go:40 Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks {"kind": "receiver", "name": "otlp", "data_type": "traces", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"} 2023-09-05T07:02:35.048-0300 info [email protected]/otlp.go:83 Starting GRPC server {"kind": "receiver", "name": "otlp", "data_type": "traces", "endpoint": "0.0.0.0:4317"} 2023-09-05T07:02:35.048-0300 info service/service.go:161 Everything is ready. Begin running and processing data. panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb1d720] goroutine 196 [running]: go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End.func1() go.opentelemetry.io/otel/[email protected]/trace/span.go:383 +0x25 go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End(0xc002e7a180, {0x0, 0x0, 0xd5c01e0?}) go.opentelemetry.io/otel/[email protected]/trace/span.go:415 +0x975 panic({0x698d520?, 0xd52b350?}) runtime/panic.go:914 +0x21f go.opentelemetry.io/collector/pdata/ptrace.Span.Attributes({0x0?}) go.opentelemetry.io/collector/[email protected]/ptrace/generated_span.go:124 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/internal.SpanPathGetSetter[...].func31({{0x0?}, {0x0?}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/contexts/internal/span.go:408 +0x5b github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.StandardGetSetter[...].Get(...) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/expression.go:49 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.(*Parser[...]).newComparisonEvaluator.func2({{0x0}, {0x0}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/boolean_value.go:87 +0xcd github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.BoolExpr[...].Eval(0xc001114fa0?, {0x88e5f18?, 0xc001d90ea0?}, {{0x0?}, {0x0?}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/boolean_value.go:19 +0x47 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.(*Parser[...].func2({{0x0}, {0x0}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/boolean_value.go:43 +0xec github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.BoolExpr[...].Eval(0x1?, {0x88e5f18?, 0xc001d90ea0?}, {{0x0?}, {0x0?}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/boolean_value.go:19 +0x47 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.(*Parser[...].func2({{0x0}, {0x0}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/boolean_value.go:60 +0xec github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.BoolExpr[...].Eval(...) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/boolean_value.go:19 github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl.(*Statement[...]).Execute(0xc002e87120, {0x88e5f18?, 0xc001d90ea0}, {{0x0}, {0x0?}, {0xc002991278?}, {0xc002b00d20?}}) github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/parser.go:55 +0x76 github.com/open-telemetry/opentelemetry-collector-contrib/processor/routingprocessor.(*tracesProcessor).route(0xc002932460, {0x88e5f18, 0xc001d90ea0}, {0x7a7910f?}) github.com/open-telemetry/opentelemetry-collector-contrib/processor/[email protected]/traces.go:116 +0x3db github.com/open-telemetry/opentelemetry-collector-contrib/processor/routingprocessor.(*tracesProcessor).ConsumeTraces(0x0?, {0x88e5f18?, 0xc001d90ea0?}, {0x6992e00?}) github.com/open-telemetry/opentelemetry-collector-contrib/processor/[email protected]/traces.go:81 +0x28 go.opentelemetry.io/collector/consumer.ConsumeTracesFunc.ConsumeTraces(...) go.opentelemetry.io/collector/[email protected]/traces.go:25 go.opentelemetry.io/collector/receiver/otlpreceiver/internal/trace.(*Receiver).Export(0xc002925470, {0x88e5f18, 0xc001d90e10}, {0x0?}) go.opentelemetry.io/collector/receiver/[email protected]/internal/trace/otlp.go:41 +0xc5 go.opentelemetry.io/collector/pdata/ptrace/ptraceotlp.rawTracesServer.Export({{0x889a548?, 0xc002925470?}}, {0x88e5f18?, 0xc001d90e10?}, 0xc002e7a100?) go.opentelemetry.io/collector/[email protected]/ptrace/ptraceotlp/grpc.go:83 +0xb0 go.opentelemetry.io/collector/pdata/internal/data/protogen/collector/trace/v1._TraceService_Export_Handler.func1({0x88e5f18, 0xc001d90e10}, {0x769a460?, 0xc002b00a08}) go.opentelemetry.io/collector/[email protected]/internal/data/protogen/collector/trace/v1/trace_service.pb.go:310 +0x72 go.opentelemetry.io/collector/config/configgrpc.(*GRPCServerSettings).toServerOption.enhanceWithClientInformation.func9({0x88e5f18?, 0xc001d90db0?}, {0x769a460, 0xc002b00a08}, 0xa?, 0xc002b00cd8) go.opentelemetry.io/collector/config/[email protected]/configgrpc.go:400 +0x46 google.golang.org/grpc.getChainUnaryHandler.func1({0x88e5f18, 0xc001d90db0}, {0x769a460, 0xc002b00a08}) google.golang.org/[email protected]/server.go:1179 +0xb2 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryServerInterceptor.func1({0x88e5f18, 0xc001d907e0}, {0x769a460, 0xc002b00a08}, 0xc002e692a0, 0xc002e73100) go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/[email protected]/interceptor.go:344 +0x52b google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1({0x88e5f18, 0xc001d907e0}, {0x769a460, 0xc002b00a08}, 0xc002088a50?, 0x698d9a0?) google.golang.org/[email protected]/server.go:1170 +0x85 go.opentelemetry.io/collector/pdata/internal/data/protogen/collector/trace/v1._TraceService_Export_Handler({0x683fd60?, 0xc00293d060}, {0x88e5f18, 0xc001d907e0}, 0xc001faf180, 0xc00292d4a0) go.opentelemetry.io/collector/[email protected]/internal/data/protogen/collector/trace/v1/trace_service.pb.go:312 +0x135 google.golang.org/grpc.(*Server).processUnaryRPC(0xc00293a1e0, {0x8916aa0, 0xc0027009c0}, 0xc002923320, 0xc002966d50, 0xd539870, 0x0) google.golang.org/[email protected]/server.go:1360 +0xe15 google.golang.org/grpc.(*Server).handleStream(0xc00293a1e0, {0x8916aa0, 0xc0027009c0}, 0xc002923320, 0x0) google.golang.org/[email protected]/server.go:1737 +0x9e7 google.golang.org/grpc.(*Server).serveStreams.func1.1() google.golang.org/[email protected]/server.go:982 +0x8d created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 195 google.golang.org/[email protected]/server.go:980 +0x165Additional context
Logs from telemetrygen:
The text was updated successfully, but these errors were encountered: