Skip to content

Modified incremental safe check which does not depend on snapshots list order #665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ashvina merged 1 commit into from
Mar 11, 2025
Merged

Modified incremental safe check which does not depend on snapshots list order #665

ashvina merged 1 commit into from
Mar 11, 2025

Conversation

@brishi19791
Copy link
Contributor

@brishi19791 brishi19791 commented Mar 10, 2025

Fixes #660

@brishi19791 brishi19791 changed the title Modified incremental safe check which does not depend on snapshots li… Modified incremental safe check which does not depend on snapshots list order Mar 10, 2025
@brishi19791
Copy link
Contributor Author

brishi19791 commented Mar 10, 2025 •
edited
Loading

@the-other-tim-brown this PR will fix the isIncrementalSafe check if the snapshots() list is not in ascending order. For the iceberg tables I am working on, snapshots() list not returning in ascending order and this check is not working as expected. I sometimes see the current snapshot or snapshot with timestamp > last sync timestamp as the first entry in the snapshots() list and check is failing but I have snapshots with timestamp < last sync timestamp. I took inspiration from getCommitsBacklog() method and created this PR. Let me know if this fixes your issue too?

@brishi19791 brishi19791 mentioned this pull request Mar 10, 2025
Closed
@the-other-tim-brown
Copy link
Contributor

the-other-tim-brown commented Mar 10, 2025

@brishi19791 can you run mvn spotless:apply to fix the style issues?

@brishi19791 brishi19791 force-pushed the users/rbokka/incrementalSafeCheckFix branch from e57299f to 0070697 Compare March 10, 2025 23:49
@brishi19791
Copy link
Contributor Author

brishi19791 commented Mar 10, 2025

@the-other-tim-brown I have added the test changes from PR #663. let me know if I missed anything.

ashvina
ashvina approved these changes Mar 11, 2025
View reviewed changes
Copy link
Contributor

@ashvina ashvina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍
One minor suggestion

@brishi19791 brishi19791 force-pushed the users/rbokka/incrementalSafeCheckFix branch from f3900ab to 77d7e4c Compare March 11, 2025 16:39
@brishi19791 brishi19791 force-pushed the users/rbokka/incrementalSafeCheckFix branch from 77d7e4c to 75396f9 Compare March 11, 2025 16:44
the-other-tim-brown
the-other-tim-brown approved these changes Mar 11, 2025
View reviewed changes
Copy link
Contributor

@the-other-tim-brown the-other-tim-brown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! Can you squash down to 1 commit? Then we can merge it for you.

@ashvina ashvina merged commit 23d870e into apache:main Mar 11, 2025
2 checks passed
@vinishjail97 vinishjail97 mentioned this pull request Apr 7, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@the-other-tim-brown the-other-tim-brown the-other-tim-brown approved these changes

@ashvina ashvina ashvina approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

XTable is doing a full sync instead of incremental sync in certain cases

3 participants

@brishi19791 @the-other-tim-brown @ashvina