Skip to content

Provide a built-in health check binary to support Amazon ECS #866

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
garysassano opened this issue Feb 2, 2025 · 6 comments
Open

Provide a built-in health check binary to support Amazon ECS #866

garysassano opened this issue Feb 2, 2025 · 6 comments
Labels
enhancement New feature or request stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed

Comments

@garysassano
Copy link
Member

Component(s)

No response

Is your feature request related to a problem? Please describe.

Amazon ECS relies on health checks to verify the availability of running services. These checks typically require executing a command inside the container, such as:

[ "CMD", "/healthcheck" ]

Relevant docs: ECS task definition parameters

Since both the ADOT collector and the otel-contrib collector images are built from scratch (ADOT Dockerfile, otel-contrib Dockerfile), they do not include a shell or curl. This exclusion is intentional, as curl has known security vulnerabilities, making it unsuitable for inclusion in production containers.

To address this, AWS designed the ADOT collector with a custom healthcheck binary. This binary replicates curl-like functionality by making an HTTP request to a predefined endpoint and returning an appropriate exit code, allowing ECS to assess the container's health without relying on external dependencies.

However, the otel-contrib collector does not provide an equivalent built-in health check mechanism. Without a native solution, users must implement their own workaround, which complicates deployments and deviates from the streamlined approach AWS provides with ADOT.

Describe the solution you'd like

The otel-contrib collector should include a built-in health check binary similar to the one in the ADOT collector. This would provide a consistent, secure, and dependency-free method for performing health checks in ECS environments.

Describe alternatives you've considered

No response

Additional context

No response

@mowies
Copy link
Member

mowies commented Mar 10, 2025

Pinging @open-telemetry/collector-contrib-triagers here since I don't think this is just a release issue.
Ideally this could somehow be implemented in form of a collector extension. The current health check extension doesn't work with ECS since it requires a command to be called inside the collector container.

@garysassano
Copy link
Member Author

The current health check extension doesn't work with ECS since it requires a command to be called inside the collector container.

This isn't entirely accurate. You can still use the health_check extension in ECS and make use of it through an ALB target group. However, you cannot use the ECS task built-in mechanism to assess its health since that requires executing a command inside the container.

@atoulme
Copy link
Contributor

atoulme commented Mar 11, 2025

An extension won't do because extensions cannot change the way the collector executes. You need a separate executable or have a set of arguments to start the collector that makes it behave differently. A separate executable seems like the cheapest option.

@mowies
Copy link
Member

mowies commented Mar 11, 2025

What if the collector binary itself had an extra flag? Maybe that could be packaged as an extension somehow?

@atoulme
Copy link
Contributor

atoulme commented Mar 11, 2025

Extensions don't have access to CLI flags.

@mowies
Copy link
Member

mowies commented Mar 12, 2025

Ok, so then what do you suggest? I am kind of opposed to just have the extra binary in every distribution per default.

@github-actions github-actions bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed
Projects
None yet
Development

No branches or pull requests

3 participants