Skip to content

Conversation

@reltuk
Copy link
Contributor

@reltuk reltuk commented Aug 25, 2025

During something like an mysqldump import, Dolt disk utilization can be unexpectedly high if we do run GC. This change enables Auto GC by default when running dolt sql in local mode.

Auto GC can be optionally disabled by passing the --disable-auto-gc flag to the dolt sql invocation.

Auto GC behavior for dolt sql-server is not changed. It is still enabled through the config.yaml file, and is off by default.

reltuk and others added 4 commits August 24, 2025 20:50
This allows things like bulk imports of mysqldumps to take advantage of auto-gc.

In this change, this setting is not currently configurable. It may represent a
performance regression for some workloads.
@coffeegoddd
Copy link
Contributor

@reltuk DOLT

comparing_percentages
100.000000 to 100.000000
version result total
2fb5423 ok 5937471
version total_tests
2fb5423 5937471
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
4e6b1af ok 5937471
version total_tests
4e6b1af 5937471
correctness_percentage
100.0

…hutdown and a database hook still gets called.
@coffeegoddd
Copy link
Contributor

@reltuk DOLT

comparing_percentages
100.000000 to 100.000000
version result total
09bcb61 ok 5937471
version total_tests
09bcb61 5937471
correctness_percentage
100.0

@reltuk reltuk requested a review from zachmu August 28, 2025 07:50
Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting those typos in there so I could add value

@coffeegoddd
Copy link
Contributor

@reltuk DOLT

comparing_percentages
100.000000 to 100.000000
version result total
f45ad2a ok 5937471
version total_tests
f45ad2a 5937471
correctness_percentage
100.0

@reltuk reltuk merged commit 544f298 into main Sep 1, 2025
22 of 24 checks passed
@github-actions
Copy link

github-actions bot commented Sep 2, 2025

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.06 1.5
batching batch sql 10000 1 0.1 1.3
batching by line sql 10000 1 0.08 1.63
blob 1 blob 200000 1 0.89 4.04 4.75
blob 2 blobs 200000 1 0.88 4.49 4.9
blob no blob 200000 1 0.97 2.46 2.84
col type datetime 200000 1 0.79 2.68 3.13
col type varchar 200000 1 0.69 3.59 4.01
config width 2 cols 200000 1 0.87 2.44 2.83
config width 32 cols 200000 1 1.9 2.54 2.84
config width 8 cols 200000 1 0.98 2.69 3.16
pk type float 200000 1 0.87 2.48 3.21
pk type int 200000 1 0.89 2.39 2.76
pk type varchar 200000 1 2.85 0.95 1.1
row count 1.6mm 1600000 1 5.65 3.13 3.31
row count 400k 400000 1 1.52 2.84 3.09
row count 800k 800000 1 2.91 3.01 3.22
secondary index four index 200000 1 3.65 1.47 1.44
secondary index no secondary 200000 1 0.97 2.47 2.87
secondary index one index 200000 1 1.16 2.59 2.84
secondary index two index 200000 1 1.99 1.89 1.98
sorting shuffled 1mm 1000000 0 5.25 2.91 2.94
sorting sorted 1mm 1000000 1 5.38 2.87 2.82

@github-actions
Copy link

github-actions bot commented Sep 2, 2025

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.13
dolt_blame_commit_filter system table 2.89
dolt_commit_ancestors_commit_filter system table 0.65
dolt_commits_commit_filter system table 1.05
dolt_diff_log_join_from_commit system table 2.68
dolt_diff_log_join_to_commit system table 2.71
dolt_diff_table_from_commit_filter system table 1.23
dolt_diff_table_to_commit_filter system table 1.19
dolt_diffs_commit_filter system table 1
dolt_history_commit_filter system table 1.5
dolt_log_commit_filter system table 1.1

@github-actions
Copy link

github-actions bot commented Sep 2, 2025

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 1.09
adds_updates_deletes 60000 60000 60000 4.47
deletes_only 0 60000 0 2.36
updates_only 0 0 60000 3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants