-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Add Queuing, retry and timeout settings to AWS S3 exporter #36264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
Sounds good. |
@stephenhong I can work on this, unless you're planning to implement it. Let me know! |
Yes, please do. Thank you |
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
Queueing implemented here: |
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Implements the common queueing feature into the `awss3exporter`, as suggested [here](#36264).
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Implements the common queueing feature into the `awss3exporter`, as suggested [here](open-telemetry#36264).
Thank you. I am going to close this issue now, but if anything needs surfacing, please reach out with a new issue. |
@atoulme, @pdelewski: I just came across this issue as I noticed that the timeout still isn't implemented for the s3 exporter, can that be added as well? |
This was delivered as part of the PR adding sending queue support to the exporter. You can leverage the sending queue that is offered as part of this component to configure retries and timeouts. See the options of the sending queue here: https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md |
#### Description Implements the common timeout feature into the awss3exporter, as suggested #36264.
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Implements the common queueing feature into the `awss3exporter`, as suggested [here](open-telemetry#36264).
#### Description Implements the common timeout feature into the awss3exporter, as suggested open-telemetry#36264.
…koff settings (#39509) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description The queue and timeout settings have been added in #36264. However, there is no retry settings added. Initially I thought we should rely on the standard `configretry.BackOffConfig`, and then realized that the AWS S3 client has a "standard" retryer implementation enabled by default. It also handles the definition of retryable errors https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/aws/retry#hdr-Standard, which makes more sense to leverage the AWS SDK instead of shoehorning it to the standard `configretry.BackOffConfig`. Explicitly mention the default retry mode in the documentation to avoid confusion. Also expose the retry settings so the user can have control. <!-- Issue number (e.g. #1234) or full URL to issue, if applicable. --> #### Link to tracking issue Mentioned in #36264. <!--Describe what testing was performed and which tests were added.--> #### Testing <!--Describe the documentation added.--> #### Documentation Updated the exporter documentation to include the retry settings. Signed-off-by: Mengnan Gong <[email protected]>
…koff settings (open-telemetry#39509) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description The queue and timeout settings have been added in open-telemetry#36264. However, there is no retry settings added. Initially I thought we should rely on the standard `configretry.BackOffConfig`, and then realized that the AWS S3 client has a "standard" retryer implementation enabled by default. It also handles the definition of retryable errors https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/aws/retry#hdr-Standard, which makes more sense to leverage the AWS SDK instead of shoehorning it to the standard `configretry.BackOffConfig`. Explicitly mention the default retry mode in the documentation to avoid confusion. Also expose the retry settings so the user can have control. <!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. --> #### Link to tracking issue Mentioned in open-telemetry#36264. <!--Describe what testing was performed and which tests were added.--> #### Testing <!--Describe the documentation added.--> #### Documentation Updated the exporter documentation to include the retry settings. Signed-off-by: Mengnan Gong <[email protected]>
Component(s)
exporter/awss3
Is your feature request related to a problem? Please describe.
There is no configuration option for queuing, retry and timeout for the AWS S3 exporter. I see that the exporterhelper is part of the factory.go but when I tried to use the options in my config.yaml, the collector wouldn't start
Describe the solution you'd like
I believe adding it in the config.go like this would add the options available
Describe alternatives you've considered
No response
Additional context
If the queuing, retry and timeout settings were left out intentionally, may I ask why they had to be taken out?
The text was updated successfully, but these errors were encountered: