Skip to content

Commit 49a05ab

Browse files
XuechunHoubraydonk
andauthored
Added a New Provider Component - Google Secrets Provider (#39790)
#### Description This PR adds the full implementation of the Google Secret Provider component. It allows users to securely reference secrets in Otel configurations. This is achieved by replacing plaintext secrets with placeholders in config files; the actual secrets are then retrieved dynamically from Google Secret Manager during the Otel Collector initialization. #### Link to tracking issue #39665 #### Testing Unit tests are added for the `Retrieve` method, and cover the following scenarios: 1. Given an identifier that matches a secret entry stored in Google Secret Manager, Retrieve() returns the corresponding plaintext secret. 2. Given an identifier that does not match any stored secret entry in Google Secret Manager, Retrieve() returns an error. 3. Given an identifier with an invalid scheme, `Retrieve()` should return an error. 4. Given an identifier with invalid opaque_data, `Retrieve()` should return an error. #### Documentation README.md --------- Co-authored-by: Braydon Kains <[email protected]>
1 parent 8965825 commit 49a05ab

File tree

18 files changed

+492
-0
lines changed

18 files changed

+492
-0
lines changed
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
2+
change_type: new_component
3+
4+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
5+
component: confmap/googlesecretmanagerprovider
6+
7+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
8+
note: Initial implementation of secrets manager provider. Allows fetch secrets from Google Secrets Manager
9+
10+
# One or more tracking issues related to the change
11+
issues: [39665]
12+
13+
# (Optional) One or more lines of additional information to render under the primary note.
14+
# These lines will be padded with 2 spaces and then inserted directly into the document.
15+
# Use pipe (|) for multiline entries.
16+
subtext:

.codecov.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ component_management:
3232
name: confmap_provider_aesprovider
3333
paths:
3434
- confmap/provider/aesprovider/**
35+
- component_id: confmap_provider_googlesecretmanagerprovider
36+
name: confmap_provider_googlesecretmanagerprovider
37+
paths:
38+
- confmap/provider/googlesecretmanagerprovider/**
3539
- component_id: confmap_provider_s3provider
3640
name: confmap_provider_s3provider
3741
paths:

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ cmd/otelcontribcol/ @open-telemetry
2525
cmd/oteltestbedcol/ @open-telemetry/collector-contrib-approvers
2626
cmd/telemetrygen/ @open-telemetry/collector-contrib-approvers @mx-psi @codeboten @Erog38
2727
confmap/provider/aesprovider/ @open-telemetry/collector-contrib-approvers @djaglowski
28+
confmap/provider/googlesecretmanagerprovider/ @open-telemetry/collector-contrib-approvers @aabmass @dashpole @jsuereth @psx95 @braydonk @ridwanmsharif
2829
confmap/provider/s3provider/ @open-telemetry/collector-contrib-approvers @Aneurysm9
2930
confmap/provider/secretsmanagerprovider/ @open-telemetry/collector-contrib-approvers @atoulme
3031
connector/countconnector/ @open-telemetry/collector-contrib-approvers @djaglowski

.github/ISSUE_TEMPLATE/bug_report.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ body:
2525
- cmd/oteltestbedcol
2626
- cmd/telemetrygen
2727
- confmap/provider/aesprovider
28+
- confmap/provider/googlesecretmanagerprovider
2829
- confmap/provider/s3provider
2930
- confmap/provider/secretsmanagerprovider
3031
- connector/count

.github/ISSUE_TEMPLATE/feature_request.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ body:
1919
- cmd/oteltestbedcol
2020
- cmd/telemetrygen
2121
- confmap/provider/aesprovider
22+
- confmap/provider/googlesecretmanagerprovider
2223
- confmap/provider/s3provider
2324
- confmap/provider/secretsmanagerprovider
2425
- connector/count

.github/ISSUE_TEMPLATE/other.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ body:
1919
- cmd/oteltestbedcol
2020
- cmd/telemetrygen
2121
- confmap/provider/aesprovider
22+
- confmap/provider/googlesecretmanagerprovider
2223
- confmap/provider/s3provider
2324
- confmap/provider/secretsmanagerprovider
2425
- connector/count

.github/ISSUE_TEMPLATE/unmaintained.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ body:
2424
- cmd/oteltestbedcol
2525
- cmd/telemetrygen
2626
- confmap/provider/aesprovider
27+
- confmap/provider/googlesecretmanagerprovider
2728
- confmap/provider/s3provider
2829
- confmap/provider/secretsmanagerprovider
2930
- connector/count

.github/component_labels.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ cmd/otelcontribcol cmd/otelcontribcol
66
cmd/oteltestbedcol cmd/oteltestbedcol
77
cmd/telemetrygen cmd/telemetrygen
88
confmap/provider/aesprovider confmap/provider/aesprovider
9+
confmap/provider/googlesecretmanagerprovider confmap/provider/googlesecretmanagerprovider
910
confmap/provider/s3provider confmap/provider/s3provider
1011
confmap/provider/secretsmanagerprovider confmap/provider/secretsmanagerprovider
1112
connector/countconnector connector/count
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
include ../../../Makefile.Common
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Google Secrets Provider
2+
<!-- status autogenerated section -->
3+
| Status | |
4+
| ------------- |-----------|
5+
| Stability | [development] |
6+
| Distributions | [] |
7+
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aprovider%2Fgooglesecretmanagerprovider%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aprovider%2Fgooglesecretmanagerprovider) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aprovider%2Fgooglesecretmanagerprovider%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aprovider%2Fgooglesecretmanagerprovider) |
8+
| Code coverage | [![codecov](https://codecov.io/github/open-telemetry/opentelemetry-collector-contrib/graph/main/badge.svg?component=provider_googlesecretmanagerprovider)](https://app.codecov.io/gh/open-telemetry/opentelemetry-collector-contrib/tree/main/?components%5B0%5D=provider_googlesecretmanagerprovider&displayType=list) |
9+
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@aabmass](https://www.github.com/aabmass), [@dashpole](https://www.github.com/dashpole), [@jsuereth](https://www.github.com/jsuereth), [@psx95](https://www.github.com/psx95), [@braydonk](https://www.github.com/braydonk), [@ridwanmsharif](https://www.github.com/ridwanmsharif) |
10+
11+
[development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development
12+
<!-- end autogenerated section -->
13+
14+
## Summary
15+
16+
This Provider component offers a secure way to reference secrets or sensitive information in collector configurations using [Google Secret Manager](https://cloud.google.com/security/products/secret-manager). Use a placeholder in the format `${googlesecretmanagerprovider:projects/<project Id>/secrets/<secret Id>/versions/<version Id>}` within your configuration. The actual secrets will then be fetched dynamically from [Google Secret Manager](https://cloud.google.com/security/products/secret-manager) during collector initialization.
17+
## Usage
18+
19+
- Simply replace plaintext secrets within your collector configuration with the placeholder: `${googlesecretmanagerprovider:projects/<project Id>/secrets/<secret Id>/versions/<version Id>}`
20+
21+
An example collector configuration:
22+
23+
```
24+
receivers:
25+
otlp:
26+
protocols:
27+
grpc:
28+
http:
29+
processors:
30+
batch:
31+
32+
exporters:
33+
logging:
34+
loglevel: debug
35+
http:
36+
endpoint: "https://example.com/api/metrics"
37+
headers:
38+
X-API-Key: ${googlesecretmanagerprovider:projects/12345/secrets/my-secret/versions/1}
39+
service:
40+
pipelines:
41+
traces:
42+
receivers: [otlp]
43+
processors: [batch]
44+
exporters: [logging, http]
45+
metrics:
46+
receivers: [otlp]
47+
processors: [batch]
48+
exporters: [logging, http]
49+
logs:
50+
receivers: [otlp]
51+
processors: [batch]
52+
exporters: [logging, http]
53+
54+
```
55+
56+
### Prerequisites
57+
1. Make sure to enable access to the [Secret Manager API](https://cloud.google.com/secret-manager/docs/accessing-the-api).
58+
2. Make sure to [add the secret entries to Google Secret Manager](https://cloud.google.com/secret-manager/docs/create-secret-quickstart) before referencing them in the collector configurations.
59+
3. This Provider interacts with Google Secret Manager using the Secret Manager client library. This library uses [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials) to locate authentication credentials for Secret Manager. Therefore, if you run your collector in a local environment, execute the [`gcloud auth application-default login`](https://cloud.google.com/secret-manager/docs/authentication#client-libs) command to generate the necessary credential file to provide to ADC.
60+
4. However, if your collector runs on Google Compute Engine (GCE) or Google Kubernetes Engine (GKE), running `gcloud auth application-default login` is optional. This is because ADC can retrieve credentials via [the metadata server](https://cloud.google.com/docs/authentication/application-default-credentials#order). However, ensure that your GKE or GCE instance [has enabled the cloud-platform OAuth scope](https://cloud.google.com/secret-manager/docs/accessing-the-api#oauth-scopes). Additionally, verify that the Service Account attached to the GCE or GKE instance has been granted at least the [roles/secretmanager.secretAccessor](https://cloud.google.com/secret-manager/docs/access-control#secret-manager-roles) IAM role to access secret entries in Google Secret Manager.
61+

0 commit comments

Comments
 (0)