Can using access and secret key to login in aws account

[matheusrosmaninho]

Component(s)

exporter/awss3

Is your feature request related to a problem? Please describe.

I would like enter the credentials of AWS for login in account

Describe the solution you'd like

The credential are optional, if don't use don't need pass nothing

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/awss3: @atoulme @pdelewski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

From the README there's a link that explains how to specify credentials to AWS. This looks like the best practice for accomplishing what you want.

Can you share why it would be better to specify credentials in the config file instead of how it's currently done?

[matheusrosmaninho]

From the README there's a link that explains how to specify credentials to AWS. This looks like the best practice for accomplishing what you want.

Can you share why it would be better to specify credentials in the config file instead of how it's currently done?

Hi,
I'm using the container otel/opentelemetry-collector-contrib and need describe the credentials in the config.yaml

I tried copy ~/.aws folder inside the container, but doesn't work
I tried create the environment AWS_PROFILE, doesn't work too

If you could have the option of putting the credentials in the file, it would be a solution. Equals the other parameters

[crobert-1]

I believe you can accomplish this by passing environment variables into your docker container.

Are you passing the environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN) into the container?

[matheusrosmaninho]

In the aws sdk for go the default login use the environment AWS_PROFILE and read ~/.aws/credentials file

I tried copy ~/.aws folder inside the container, but doesn't work
I tried create the environment AWS_PROFILE, doesn't work too

[crobert-1]

It looks like there are some ways to mount the credentials file if required, or specify the environment variables. Here's a stack overflow thread on this.

[matheusrosmaninho]

In the code I can't pass the access key and secret key. The code get this credentials in default profile

opentelemetry-collector-contrib/exporter/awss3exporter/s3_writer.go

Line 49 in 511b04b

func getSessionConfig(config *Config) *aws.Config {

[crobert-1]

Can you share your configuration? I might be missing something so it would help to know exactly how you're trying to run the collector in a container. The collector's config file, any kind of dockerfile/CLI command being used would be helpful.

[matheusrosmaninho]

Sure :)

docker-compose.yml

version: "3.8"
services:
  collector:
    container_name: collector
    build:
      context: .
      args:
        COLLECTOR_VERSION: 0.88.0
    ports:
      - 4318:4318 # OTLP http receiver
      - 4317:4317 # OTLP http receiver
    volumes:
      - ./aws:/.aws
    environment:
      AWS_PROFILE: "default"

Dockerfile

ARG COLLECTOR_VERSION

FROM otel/opentelemetry-collector-contrib:${COLLECTOR_VERSION}

COPY ./otel-collector-config.yaml /etc/otel-collector-config

COPY ./aws ~/.aws

CMD [ "--config=/etc/otel-collector-config" ]

EXPOSE 4317
EXPOSE 4318

Config.yaml

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317


processors:
  batch:
    timeout: 5s

extensions:
  health_check:
  basicauth:
    client_auth:
      username: <grafana_cloud_username>
      password: <grafana_cloud_password>


exporters:
  logging:
    verbosity: detailed

  otlphttp:
    auth:
      authenticator: basicauth
    endpoint: <grafana_endpoint>

  awss3:
    s3uploader:
      region: 'us-east-1'
      s3_bucket: 'bucket-teste-otel'
      s3_prefix: 'metric'
      s3_partition: 'minute'

service:
  extensions: [basicauth, health_check]
  pipelines:
    # traces:
    #   receivers: [otlp]
    #   processors: batch
    #   exporters: [otlphttp]

    # metrics:
    #   receivers: [otlp]
    #   processors: batch
    #   exporters: [logging, otlphttp]

    logs:
      receivers: [otlp]
      processors: batch
      exporters: [logging, otlphttp, awss3]

The error message:

[crobert-1]

I'll defer to the code owners at this point since they're the best suited to help. I believe the authentication can be done with the correct mounting path for the aws/credentials file, or adding the right environment variables, but I'll let others add more information here.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/awss3: @atoulme @pdelewski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[tkanhe-karini]

According to https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/#specifying-credentials
We can also give environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION) in docker environment like this:

docker-compose.yaml

version: '3'

services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
    ports:
      - "4317:4317" # OTLP gRPC receiver
      - "4318:4318" # OTLP http receiver
    environment:
      AWS_ACCESS_KEY_ID: "AK*************"
      AWS_SECRET_ACCESS_KEY: "MA**************"
      AWS_REGION: "us-east-1"

otel-collector-config.yaml

receivers:
  otlp: # the OTLP receiver the app is sending logs to
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 10s

exporters:
  awss3:
    s3uploader:
      region: "us-east-1"
      s3_bucket: "otel-traces-test"
      s3_prefix: "traces"
      s3_partition: "minute"
      disable_ssl: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [awss3]

[crobert-1]

@matheusrosmaninho Is there anything else needed for this issue, or can we resolve it?

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/awss3: @atoulme @pdelewski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.