Skip to content

[receiver/k8sobjectsreceiver] k8sobject receiver support leader election #39054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 12, 2025

Conversation

Frapschen
Copy link
Contributor

Description

As #38426 move the k8sleaderelector from development to alpha, we can use it in k8sobjectsreceiver.

@Frapschen
Copy link
Contributor Author

I have test the code in my enviroment. I use this config:

exporters:
  debug: {}
extensions:
  k8s_leader_elector/k8sobjects:
    auth_type: kubeConfig
    context: cluster-bm-admin@cluster-bm
    lease_name: local-test-k8sobjects
    lease_namespace: default
processors:
  batch: {}
receivers:
  k8sobjects:
    auth_type: kubeConfig
    context: cluster-bm-admin@cluster-bm
    k8s_leader_elector: k8s_leader_elector/k8sobjects
    objects:
      - name: events
        mode: pull
        group: events.k8s.io/v1
        interval: 10s
service:
  telemetry:
    logs:
      level: info
    metrics:
      address: 0.0.0.0:1233
  extensions:
    - k8s_leader_elector/k8sobjects
  pipelines:
    logs:
      exporters:
#        - file
        - debug
      processors:
        - batch
      receivers:
        - k8sobjects

test case 1: start two collector instances with leader electon
image
we can see only the first collector successfully acquired lease and start pull k8s event data

test case 1: The lease resource was accidentally deleted
image
The first collector print error logs but stil get lease and continue pull event from k8s

test case 1: shutdown the first and wait 3s to restart
image
The second collector successfully acquired the lease 20 seconds after the first collector shut down.

@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from 4bb004b to 81d5190 Compare March 31, 2025 11:05
@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from 81d5190 to 00284d7 Compare April 1, 2025 06:22
@atoulme
Copy link
Contributor

atoulme commented Apr 5, 2025

Please resolve conflicts and add tests. It would be ideal to have a unit test that covers an election with receivers joining and leaving.

@atoulme atoulme marked this pull request as draft April 5, 2025 05:58
@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from 1bed805 to c7a8865 Compare April 6, 2025 16:01
@Frapschen
Copy link
Contributor Author

@atoulme A normal unit test can‘t achieve your suggestion. I will try to add a new e2e test to address this.

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Apr 24, 2025
@Frapschen Frapschen removed the Stale label Apr 24, 2025
@rakesh-garimella
Copy link
Contributor

@Frapschen
Copy link
Contributor Author

@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from c7a8865 to a02815c Compare May 7, 2025 08:06
@Frapschen Frapschen marked this pull request as ready for review May 7, 2025 08:22
@Frapschen Frapschen requested a review from atoulme May 7, 2025 08:22
@Frapschen
Copy link
Contributor Author

@atoulme Please review

@Frapschen Frapschen requested a review from ChrsMark May 7, 2025 17:30
@atoulme atoulme marked this pull request as draft May 8, 2025 00:49
@atoulme
Copy link
Contributor

atoulme commented May 8, 2025

Moving to draft for now while you address comments, please mark ready for review once done.

@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from 1ceb3a2 to 2bbd0e5 Compare May 8, 2025 02:41
@Frapschen Frapschen requested a review from dmitryax May 8, 2025 02:42
@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from 8d2ba99 to 2461088 Compare May 8, 2025 07:27
@Frapschen Frapschen requested a review from dmitryax May 8, 2025 07:28
@Frapschen Frapschen force-pushed the k8s0bject-leader-election branch from 8c62ff5 to 3867efa Compare May 10, 2025 15:54
@Frapschen Frapschen marked this pull request as ready for review May 12, 2025 16:19
@dmitryax dmitryax merged commit a7d283d into open-telemetry:main May 12, 2025
176 checks passed
@github-actions github-actions bot added this to the next release milestone May 12, 2025
@Frapschen Frapschen deleted the k8s0bject-leader-election branch May 14, 2025 06:05
dragonlord93 pushed a commit to dragonlord93/opentelemetry-collector-contrib that referenced this pull request May 23, 2025
…ion (open-telemetry#39054)

#### Description

As
open-telemetry#38426
move the k8sleaderelector from development to alpha, we can use it in
k8sobjectsreceiver.

---------

Co-authored-by: Antoine Toulme <[email protected]>
Co-authored-by: Christos Markou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants