Skip to content

[v2][remote-storage] Implement remote storage extension #7043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Apr 20, 2025

Conversation

mahadzaryab1
Copy link
Collaborator

@mahadzaryab1 mahadzaryab1 commented Apr 18, 2025

Which problem is this PR solving?

Description of the changes

  • Introduces a new remote_storage extension that wraps the existing remote-storage binary. This extension allows a remote storage gRPC server to be started within jaeger-v2, delegating to one of the configured backends.
  • Adds a new configuration file (config-remote-storage-backend.yaml) used by the integration tests. This replaces the need to manually start the test remote storage server. Integration tests are updated to use this config and remove related boilerplate.

How was this change tested?

  • Updated the integration tests and added new unit tests

Checklist

@mahadzaryab1 mahadzaryab1 added the changelog:exprimental Change to an experimental part of the code label Apr 18, 2025
Copy link

codecov bot commented Apr 18, 2025

Codecov Report

Attention: Patch coverage is 93.18182% with 6 lines in your changes missing coverage. Please review.

Project coverage is 96.03%. Comparing base (d6c760b) to head (f0c22c8).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
.../jaeger/internal/extension/remotestorage/server.go 89.09% 4 Missing and 2 partials ⚠️

❌ Your patch status has failed because the patch coverage (93.18%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7043      +/-   ##
==========================================
- Coverage   96.06%   96.03%   -0.03%     
==========================================
  Files         352      355       +3     
  Lines       20890    20973      +83     
==========================================
+ Hits        20067    20141      +74     
- Misses        620      626       +6     
- Partials      203      206       +3     
Flag Coverage Δ
badger_v1 9.97% <0.00%> (-0.01%) ⬇️
badger_v2 2.07% <0.00%> (-0.01%) ⬇️
cassandra-4.x-v1-manual 15.00% <0.00%> (-0.01%) ⬇️
cassandra-4.x-v2-auto 2.06% <0.00%> (-0.01%) ⬇️
cassandra-4.x-v2-manual 2.06% <0.00%> (-0.01%) ⬇️
cassandra-5.x-v1-manual 15.00% <0.00%> (-0.01%) ⬇️
cassandra-5.x-v2-auto 2.06% <0.00%> (-0.01%) ⬇️
cassandra-5.x-v2-manual 2.06% <0.00%> (-0.01%) ⬇️
elasticsearch-6.x-v1 19.95% <0.00%> (-0.01%) ⬇️
elasticsearch-7.x-v1 20.02% <0.00%> (-0.01%) ⬇️
elasticsearch-8.x-v1 20.19% <0.00%> (-0.01%) ⬇️
elasticsearch-8.x-v2 2.07% <0.00%> (-0.12%) ⬇️
grpc_v1 11.52% <71.42%> (+<0.01%) ⬆️
grpc_v2 2.07% <0.00%> (-7.08%) ⬇️
kafka-3.x-v1 10.24% <0.00%> (-0.01%) ⬇️
kafka-3.x-v2 2.07% <0.00%> (-0.01%) ⬇️
memory_v2 2.07% <0.00%> (-0.01%) ⬇️
opensearch-1.x-v1 20.07% <0.00%> (-0.01%) ⬇️
opensearch-2.x-v1 20.07% <0.00%> (-0.01%) ⬇️
opensearch-2.x-v2 2.07% <0.00%> (-0.01%) ⬇️
query 2.07% <0.00%> (-7.08%) ⬇️
tailsampling-processor 0.56% <0.00%> (-0.01%) ⬇️
unittests 94.80% <93.18%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

}

tm := tenancy.NewManager(&s.config.Tenancy)
s.server, err = app.NewServer(&s.config.Options, tf, df, tm, telset)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we avoid depending on cmd/remote-storage ? What are you getting from there?

Copy link
Collaborator Author

@mahadzaryab1 mahadzaryab1 Apr 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yurishkuro The server implementation is already defined there so I was reusing it. We do the same thing in the jaeger_query extension as well. Let me know if you want to take a different approach here though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@mahadzaryab1 mahadzaryab1 changed the title [WIP] Implement remote storage extension [WIP][v2][remote-storage] Implement remote storage extension Apr 18, 2025
Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: Mahad Zaryab <[email protected]>
@mahadzaryab1 mahadzaryab1 changed the title [WIP][v2][remote-storage] Implement remote storage extension [v2][remote-storage] Implement remote storage extension Apr 20, 2025
@mahadzaryab1 mahadzaryab1 marked this pull request as ready for review April 20, 2025 01:08
@mahadzaryab1 mahadzaryab1 requested a review from a team as a code owner April 20, 2025 01:08
@mahadzaryab1 mahadzaryab1 requested a review from jkowall April 20, 2025 01:08
@yurishkuro yurishkuro merged commit b3c329b into jaegertracing:main Apr 20, 2025
56 of 57 checks passed
amilbcahat pushed a commit to amilbcahat/jaeger that referenced this pull request May 4, 2025
…g#7043)

## Which problem is this PR solving?
- Towards jaegertracing#7038

## Description of the changes
- Introduces a new remote_storage extension that wraps the existing
remote-storage binary. This extension allows a remote storage gRPC
server to be started within jaeger-v2, delegating to one of the
configured backends.
- Adds a new configuration file (config-remote-storage-backend.yaml)
used by the integration tests. This replaces the need to manually start
the test remote storage server. Integration tests are updated to use
this config and remove related boilerplate.

## How was this change tested?
- Updated the integration tests and added new unit tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `npm run lint` and `npm run test`

---------

Signed-off-by: Mahad Zaryab <[email protected]>
amilbcahat pushed a commit to amilbcahat/jaeger that referenced this pull request May 4, 2025
…g#7043)

## Which problem is this PR solving?
- Towards jaegertracing#7038

## Description of the changes
- Introduces a new remote_storage extension that wraps the existing
remote-storage binary. This extension allows a remote storage gRPC
server to be started within jaeger-v2, delegating to one of the
configured backends.
- Adds a new configuration file (config-remote-storage-backend.yaml)
used by the integration tests. This replaces the need to manually start
the test remote storage server. Integration tests are updated to use
this config and remove related boilerplate.

## How was this change tested?
- Updated the integration tests and added new unit tests

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `npm run lint` and `npm run test`

---------

Signed-off-by: Mahad Zaryab <[email protected]>
Signed-off-by: amol-verma-allen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/storage changelog:exprimental Change to an experimental part of the code v2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants