Skip to content

Conversation

@0xnm
Copy link
Member

@0xnm 0xnm commented Mar 7, 2025

What does this PR do?

Mapping file can read few hundred MBs and so this PR improves its handling in such case by:

  • Using buffered writing to the disk instead of doing I/O operation for each line
  • Avoiding loading the whole file in the memory and using the stream to read the file line by line.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@0xnm 0xnm requested review from a team as code owners March 7, 2025 13:33
shrinkedFile.bufferedWriter(bufferSize = MAPPING_OPTIMIZED_FILE_WRITE_BUFFER_SIZE)
.use { shrinkedFileWriter ->
mappingFile.useLines { lines ->
lines.run { if (mappingFileTrimIndents) map { it.trimStart() } else this }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

if/else statements should use braces. (...read more)

View in Datadog  Leave us feedback  Documentation

Comment on lines +133 to +135
if (shrinkedFile.exists()) {
shrinkedFile.delete()
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Single line if statements should not use braces. (...read more)

View in Datadog  Leave us feedback  Documentation

shrinkedFile.bufferedWriter(bufferSize = MAPPING_OPTIMIZED_FILE_WRITE_BUFFER_SIZE)
.use { shrinkedFileWriter ->
mappingFile.useLines { lines ->
lines.run { if (mappingFileTrimIndents) map { it.trimStart() } else this }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Braces are required for multiline if else statements. (...read more)

View in Datadog  Leave us feedback  Documentation

@codecov-commenter
Copy link

codecov-commenter commented Mar 7, 2025

Codecov Report

Attention: Patch coverage is 91.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 77.88%. Comparing base (4041e2d) to head (64fcf0e).
Report is 2 commits behind head on develop.

Files with missing lines Patch % Lines
...com/datadog/gradle/plugin/MappingFileUploadTask.kt 91.67% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #326      +/-   ##
=============================================
- Coverage      77.94%   77.88%   -0.06%     
- Complexity       258      260       +2     
=============================================
  Files             31       31              
  Lines            943      945       +2     
  Branches         113      113              
=============================================
+ Hits             735      736       +1     
  Misses           151      151              
- Partials          57       58       +1     
Files with missing lines Coverage Δ
...com/datadog/gradle/plugin/MappingFileUploadTask.kt 83.78% <91.67%> (-0.94%) ⬇️
🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@0xnm 0xnm merged commit 1ea0e1a into develop Mar 10, 2025
12 checks passed
@0xnm 0xnm deleted the nogorodnikov/rum-8910/improve-big-file-handling-for-mapping-file-shrinking branch March 10, 2025 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants