Skip to content

[receiver/mongodbatlas] add Project Config to metrics #28865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JonathanWamsley opened this issue Nov 2, 2023 · 3 comments
Closed

[receiver/mongodbatlas] add Project Config to metrics #28865

JonathanWamsley opened this issue Nov 2, 2023 · 3 comments
Labels

Comments

@JonathanWamsley
Copy link
Contributor

JonathanWamsley commented Nov 2, 2023

Component(s)

receiver/mongodbatlas

Is your feature request related to a problem? Please describe.

Add a Project Config to metrics like the Alerts, Events and Logs configs

type ProjectConfig struct {
	Name            string   `mapstructure:"name"`
	ExcludeClusters []string `mapstructure:"exclude_clusters"`
	IncludeClusters []string `mapstructure:"include_clusters"`

	includesByClusterName map[string]struct{}
	excludesByClusterName map[string]struct{}
}

Adding this feature will help improve performance by selecting specific projects and or clusters instead of collecting every one which may cause rate limiting by MongoDB Atlas.

Describe the solution you'd like

Add in a Project Config for metrics in the config.

Describe alternatives you've considered

The existing config has a MetricsBuilderConfig in the base, so to not creating breaking changes a the ProjectConfig would go directly in the base config for metrics which would like like:

type Config struct {
	scraperhelper.ScraperControllerSettings `mapstructure:",squash"`
	PublicKey                               string                        `mapstructure:"public_key"`
	PrivateKey                              configopaque.String           `mapstructure:"private_key"`
	Granularity                             string                        `mapstructure:"granularity"`
	MetricsBuilderConfig                    metadata.MetricsBuilderConfig `mapstructure:",squash"`
	Projects                                []*ProjectConfig              `mapstructure:"projects"`
	Alerts                                  AlertConfig                   `mapstructure:"alerts"`
	Events                                  *EventsConfig                 `mapstructure:"events"`
	Logs                                    LogConfig                     `mapstructure:"logs"`
	RetrySettings                           exporterhelper.RetrySettings  `mapstructure:"retry_on_failure"`
	StorageID                               *component.ID                 `mapstructure:"storage"`
}

However, an alternative breaking change would create a new Metrics section which would include the ProjectConfig and possibly moving the MetricsBuilderConfig into the Metrics that would look like:

type Config struct {
	scraperhelper.ScraperControllerSettings `mapstructure:",squash"`
	PublicKey                               string                        `mapstructure:"public_key"`
	PrivateKey                              configopaque.String           `mapstructure:"private_key"`
	Granularity                             string                        `mapstructure:"granularity"`
	Metrics                               MetricConfig                 `mapstructure:"metrics"`
	Alerts                                  AlertConfig                   `mapstructure:"alerts"`
	Events                                  *EventsConfig                 `mapstructure:"events"`
	Logs                                    LogConfig                     `mapstructure:"logs"`
	RetrySettings                           exporterhelper.RetrySettings  `mapstructure:"retry_on_failure"`
	StorageID                               *component.ID                 `mapstructure:"storage"`
}

Additional context

No response

@JonathanWamsley JonathanWamsley added enhancement New feature or request needs triage New item requiring triage labels Nov 2, 2023
Copy link
Contributor

github-actions bot commented Nov 2, 2023

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

I'm not very familiar with MongoDB, but this seems like a good idea to me. Adding a filter to the query in a receiver is a good way to help performance and reduce resource usage.

Thanks for submitting the PR as well @JonathanWamsley!

@crobert-1 crobert-1 removed the needs triage New item requiring triage label Nov 10, 2023
djaglowski added a commit that referenced this issue Nov 17, 2023
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This feature adds a Project Config for the metrics to filter by Project
name and or clusters.

**Link to tracking Issue:** <Issue number if applicable>
#28865 

**Testing:** <Describe what testing was performed and which tests were
added.>
- Added test for cluster filtering
- Tested project name alone, project name with IncludeClusters and
project name with ExcludeClusters on a live environment with success.

**Documentation:** <Describe the documentation added.>
Added optional project config fields to README

---------

Co-authored-by: Daniel Jaglowski <[email protected]>
graphaelli pushed a commit to graphaelli/opentelemetry-collector-contrib that referenced this issue Nov 17, 2023
…metry#28866)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This feature adds a Project Config for the metrics to filter by Project
name and or clusters.

**Link to tracking Issue:** <Issue number if applicable>
open-telemetry#28865 

**Testing:** <Describe what testing was performed and which tests were
added.>
- Added test for cluster filtering
- Tested project name alone, project name with IncludeClusters and
project name with ExcludeClusters on a live environment with success.

**Documentation:** <Describe the documentation added.>
Added optional project config fields to README

---------

Co-authored-by: Daniel Jaglowski <[email protected]>
@JonathanWamsley
Copy link
Contributor Author

Resolved with #28866

aishyandapalli pushed a commit to aishyandapalli/opentelemetry-collector-contrib that referenced this issue Nov 17, 2023
…metry#28866)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This feature adds a Project Config for the metrics to filter by Project
name and or clusters.

**Link to tracking Issue:** <Issue number if applicable>
open-telemetry#28865 

**Testing:** <Describe what testing was performed and which tests were
added.>
- Added test for cluster filtering
- Tested project name alone, project name with IncludeClusters and
project name with ExcludeClusters on a live environment with success.

**Documentation:** <Describe the documentation added.>
Added optional project config fields to README

---------

Co-authored-by: Daniel Jaglowski <[email protected]>
RoryCrispin pushed a commit to ClickHouse/opentelemetry-collector-contrib that referenced this issue Nov 24, 2023
…metry#28866)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This feature adds a Project Config for the metrics to filter by Project
name and or clusters.

**Link to tracking Issue:** <Issue number if applicable>
open-telemetry#28865 

**Testing:** <Describe what testing was performed and which tests were
added.>
- Added test for cluster filtering
- Tested project name alone, project name with IncludeClusters and
project name with ExcludeClusters on a live environment with success.

**Documentation:** <Describe the documentation added.>
Added optional project config fields to README

---------

Co-authored-by: Daniel Jaglowski <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants