[exporterhelper] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc99570]

[ceevaaa]

Component(s)

extension/storage/filestorage

What happened?

Description

I am trying to run the collector with file_storage extension so that I can test the persistent storage feature in otlp exporter.

Actual Result

The collector doesn't start and throws this error (Check below for full error).

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc99570]

Collector version

v0.86.0

Environment information

Environment

OS: Amazon Linux 2
Running the collector as a Docker container.

OpenTelemetry Collector configuration

extensions:
  file_storage:

receivers:

  hostmetrics:
    scrapers:
      cpu:
      load:
      memory:
      disk:
        metrics:
          system.disk.io_time:
            enabled: false
          system.disk.merged:
            enabled: false
          system.disk.operation_time:
            enabled: false
          system.disk.operations:
            enabled: false
          system.disk.pending_operations:
            enabled: false
          system.disk.weighted_io_time:
            enabled: false
      network:
    collection_interval: 60s

processors:

  batch:

exporters:

  otlp:
    endpoint: ${env:CENTRAL_OTEL_COLLECTOR_ENDPOINT}
    tls:
      insecure: true
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 3600s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 100000
      storage: file_storage
    timeout: 5s

  debug:
    verbosity: detailed


service:

  extensions:
    - file_storage

  pipelines:

    metrics:
      receivers:
        - hostmetrics
      processors:
        - batch
      exporters:
        - otlp
        - debug

Log output

2023-10-21T10:26:28.066Z	info	service@v0.86.0/telemetry.go:84	Setting up own telemetry...
2023-10-21T10:26:28.066Z	info	service@v0.86.0/telemetry.go:201	Serving Prometheus metrics	{"address": ":8888", "level": "Basic"}
2023-10-21T10:26:28.067Z	info	exporter@v0.86.0/exporter.go:275	Development component. May change in the future.	{"kind": "exporter", "data_type": "metrics", "name": "debug"}
2023-10-21T10:26:28.067Z	info	service@v0.86.0/service.go:138	Starting otelcol-contrib...	{"Version": "0.86.0", "NumCPU": 1}
2023-10-21T10:26:28.067Z	info	extensions/extensions.go:31	Starting extensions...
2023-10-21T10:26:28.067Z	info	extensions/extensions.go:34	Extension is starting...	{"kind": "extension", "name": "file_storage"}
2023-10-21T10:26:28.068Z	info	extensions/extensions.go:38	Extension started.	{"kind": "extension", "name": "file_storage"}
2023-10-21T10:26:28.068Z	info	service@v0.86.0/service.go:170	Starting shutdown...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc99570]

goroutine 1 [running]:
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*persistentContiguousStorage).stop(0x0)
	go.opentelemetry.io/collector/exporter@v0.86.0/exporterhelper/internal/persistent_storage.go:196 +0x20
go.opentelemetry.io/collector/exporter/exporterhelper/internal.glob..func1(0x40028da180?)
	go.opentelemetry.io/collector/exporter@v0.86.0/exporterhelper/internal/persistent_queue.go:19 +0x20
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*persistentQueue).Stop.func1()
	go.opentelemetry.io/collector/exporter@v0.86.0/exporterhelper/internal/persistent_queue.go:94 +0x44
sync.(*Once).doSlow(0x400237f0d8?, 0x0?)
	sync/once.go:74 +0x100
sync.(*Once).Do(...)
	sync/once.go:65
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*persistentQueue).Stop(0x40028da178?)
	go.opentelemetry.io/collector/exporter@v0.86.0/exporterhelper/internal/persistent_queue.go:90 +0x48
go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).shutdown(0x400280b130)
	go.opentelemetry.io/collector/exporter@v0.86.0/exporterhelper/queue_sender.go:159 +0x80
go.opentelemetry.io/collector/exporter/exporterhelper.(*baseExporter).Shutdown(0x400292c3c0, {0x7e69738, 0xca44d40})
	go.opentelemetry.io/collector/exporter@v0.86.0/exporterhelper/common.go:227 +0x4c
go.opentelemetry.io/collector/service/internal/graph.(*Graph).ShutdownAll(0x0?, {0x7e69738, 0xca44d40})
	go.opentelemetry.io/collector/service@v0.86.0/internal/graph/graph.go:358 +0xd4
go.opentelemetry.io/collector/service.(*Service).Shutdown(0x40028e6510, {0x7e69738, 0xca44d40})
	go.opentelemetry.io/collector/service@v0.86.0/service.go:176 +0xd0
go.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents(0x400275cc00, {0x7e69738, 0xca44d40})
	go.opentelemetry.io/collector/otelcol@v0.86.0/collector.go:187 +0x4f4
go.opentelemetry.io/collector/otelcol.(*Collector).Run(0x400275cc00, {0x7e69738, 0xca44d40})
	go.opentelemetry.io/collector/otelcol@v0.86.0/collector.go:221 +0x3c
go.opentelemetry.io/collector/otelcol.NewCommand.func1(0x4002231200, {0x7007964?, 0x7?, 0x70001d1?})
	go.opentelemetry.io/collector/otelcol@v0.86.0/command.go:27 +0x70
github.com/spf13/cobra.(*Command).execute(0x4002231200, {0x40000b0bb0, 0x2, 0x2})
	github.com/spf13/cobra@v1.7.0/command.go:940 +0x658
github.com/spf13/cobra.(*Command).ExecuteC(0x4002231200)
	github.com/spf13/cobra@v1.7.0/command.go:1068 +0x320
github.com/spf13/cobra.(*Command).Execute(0x4002801050?)
	github.com/spf13/cobra@v1.7.0/command.go:992 +0x1c
main.runInteractive({{0x4002801050, 0x4002826330, 0x40028014d0, 0x4002800c00, 0x40028263f0}, {{0x70314b6, 0xf}, {0x70c1983, 0x1f}, {0x70040dc, ...}}, ...})
	github.com/open-telemetry/opentelemetry-collector-releases/contrib/main.go:31 +0x40
main.run(...)
	github.com/open-telemetry/opentelemetry-collector-releases/contrib/main_others.go:11
main.main()
	github.com/open-telemetry/opentelemetry-collector-releases/contrib/main.go:24 +0x134

Additional context

The docker compose

version: '3.7'
services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.86.0
    volumes:
      - /home/ec2-user/otelcol/file_storage/:/var/lib/otelcol/file_storage/:rw
    network_mode: host
    env_file:
      - .env
    restart: always

No response

[github-actions]

Pinging code owners:

• extension/storage/filestorage: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[ceevaaa]

I also have a small question regarding the
2023-10-21T10:26:28.067Z info service@v0.86.0/service.go:138 Starting otelcol-contrib... {"Version": "0.86.0", "NumCPU": 1}

What does it represent ? Does it mean the collector only uses 1 CPU max.
What if I want to increase it ? Can someone point me to the relevant doc around this ?

Thanks and Regards.

[djaglowski]

I've transferred this issue to opentelemetry-collector repository as it appears to be related to exporterhelper.

[HeavenTonight]

/assign ,,Maybe i can try resolve this issue. @djaglowski

[atoulme]

It's yours @HeavenTonight

[ceevaaa]

Not sure, but is this the same issue guys.

https://github.com/open-telemetry/opentelemetry-collector/releases/tag/v0.88.0

"exporterhelper: Fix nil pointer dereference when stopping persistent queue after a start encountered an error (#8718)"

[HeavenTonight]

Not sure, but is this the same issue guys.↳

https://github.com/open-telemetry/opentelemetry-collector/releases/tag/v0.88.0

"exporterhelper: Fix nil pointer dereference when stopping persistent queue after a start encountered an error (#8718)"↳

Yes,i thank this pr #8718 have already been solved this issue！！

[ceevaaa]

Closing as this seems to fixed already.

Thank you everyone.