Release Notes for release-0.3.0-incubating-rc3

Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

✨ What's Changed

🔄 Core Functionality Enhancements

Catalog Sync:
- Introduced CatalogSyncClient and CatalogSync interfaces.
- Implemented Glue and HMS catalog sync for Iceberg, Delta, and Hudi.
- Additional information can be found in docs.
Table Format Sync Improvements:
- Added support for continuous sync using RunSync.
- Support restore/rollback sync during conversion across all three formats.
- Added support for bucket partition transform (Iceberg → Delta).
- Synced schema field comments and record counts in Delta source.
- Improved decimal field scale handling in iceberg and fixed bug in column stats.
- Supporting timestamp_ntz type in conversion logic.
Breaking changes
- We have refactored SyncStatusCode and ErrorDetails into dedicated classes, refactor code by fixing the import path if you are using these classes.

🧪 Testing & Quality

Fixed flaky tests and normalized base paths for test stability.
Addressed minor issues found via static analysis tooling.
Dockerfile syntax fix and test class name corrections.

📘 Documentation & Community

Added downloads page for 0.2.0-incubating release.
Introduced RFC template and new change proposals.
Updated community sync and blog pages.

🧰 Infrastructure & Maintenance

Upgraded parquet-avro to 1.15.1 to fix critical vulnerabilities found recently.
Upgraded Hadoop and AWS SDK dependencies.
Refactored RunSync for better modularity and reusability.

Detailed Change Log

Iceberg and Delta target: sync schema field comments by @emilie-wang in Iceberg and Delta target: sync schema field comments #575
[551] Add downloads page for 0.2.0-incubating release by @vinishjail97 in [551] Add downloads page for 0.2.0-incubating release #576
[551] Try enabling keep_files as false in GH action for downloads by @vinishjail97 in [551] Try enabling keep_files as false in GH action for downloads #577
[DOCS] Remove info on snowflake public preview for Iceberg by @sagarlakshmipathy in [DOCS] Remove info on snowflake public preview for Iceberg #580
Add 2 New Community Blogs to the Apache XTable Blogs section by @dipankarmazumdar in Add 2 New Community Blogs to the Apache XTable Blogs section #585
Site content is not deployed due to missing .asf.yaml file by @zabetak in Site content is not deployed due to missing .asf.yaml file #587
Execute correct delete operation in delta tests by @ashvina in Execute correct delete operation in delta tests #589
Fix commit log parsing of Delta tables with delete vector by @ashvina in Fix commit log parsing of Delta tables with delete vector #596
Normalize basePath in targetTables in ConversionController by @vinishjail97 in Normalize basePath in targetTables in ConversionController #602
[590] Add RFC template for XTable by @vinishjail97 in [590] Add RFC template for XTable #604
[590] Add interfaces for CatalogSyncClient and CatalogSync by @vinishjail97 in [590] Add interfaces for CatalogSyncClient and CatalogSync #603
[590] Add RFC template for XTable by @vinishjail97 in [590] Add RFC template for XTable #610
[590] Hadoop and AWS sdk version upgrade by @kroushan-nit in [590] Hadoop and AWS sdk version upgrade #614
[619] Maintenance: Update dependencies, fix test class names by @the-other-tim-brown in [619] Maintenance: Update dependencies, fix test class names #620
[608] Ensure column stats for decimal fields have proper scale set by @the-other-tim-brown in [608] Ensure column stats for decimal fields have proper scale set #617
[615] Add Iceberg stats when reading snapshot by @the-other-tim-brown in [615] Add Iceberg stats when reading snapshot #616
Add community sync page for XTable by @vinishjail97 in Add community sync page for XTable #625
Update GMeet link and add today's recording by @vinishjail97 in Update GMeet link and add today's recording #628
[590] Add RFC for XCatalogSync - Synchronize tables across catalogs by @vinishjail97 in [590] Add RFC for XCatalogSync - Synchronize tables across catalogs #605
[590] Add RunCatalogSync utility for synchronizing tables across catalogs by @vinishjail97 in [590] Add RunCatalogSync utility for synchronizing tables across catalogs #591
Delta source: add record count for DataFile by @emilie-wang in Delta source: add record count for DataFile #623
[site] Update blogs by @bhasudha in [site] Update blogs #635
[590] Add Iceberg Glue Catalog Sync implementation by @kroushan-nit in [590] Add Iceberg Glue Catalog Sync implementation #632
[590] Add Iceberg HMS Catalog Sync implementation by @kroushan-nit in [590] Add Iceberg HMS Catalog Sync implementation #633
[590] Add Delta Glue Catalog Sync implementation by @kroushan-nit in [590] Add Delta Glue Catalog Sync implementation #637
[590] Add Delta HMS Catalog Sync implementation by @kroushan-nit in [590] Add Delta HMS Catalog Sync implementation #638
[590] Add Hudi HMS Catalog Sync implementation by @vamsikarnika in [590] Add Hudi HMS Catalog Sync implementation #639
Update xtable-api sources by @emmanuel-ferdman in Update xtable-api sources #651
[Enhancement] Support restore/rollback sync during conversion (1/2) by @danielhumanmod in [Enhancement] Support restore/rollback sync during conversion (1/2) #569
[590] Add Hudi HMS Catalog Sync Implementation by @vamsikarnika in [590] Add Hudi HMS Catalog Sync Implementation #648
[590] Add Hudi Glue Catalog Sync Implementation by @vamsikarnika in [590] Add Hudi Glue Catalog Sync Implementation #649
Add change proposal for conversion of deletion vectors by @ashvina in Add change proposal for conversion of deletion vectors #634
Add base representation for storage files by @ashvina in Add base representation for storage files #653
Fix flaky test, ignore order of elements in list validation by @ashvina in Fix flaky test, ignore order of elements in list validation #662
[594] Support continuous conversion with the RunSync tool by @the-other-tim-brown in [594] Support continuous conversion with the RunSync tool #593
[MINOR] fixes recommended from static analysis tooling by @the-other-tim-brown in [MINOR] fixes recommended from static analysis tooling #664
Modified incremental safe check which does not depend on snapshots list order by @brishi19791 in Modified incremental safe check which does not depend on snapshots list order #665
[668] Support bucket partition transform for Iceberg Sources and Delta Targets by @the-other-tim-brown in [668] Support bucket partition transform for Iceberg Sources and Delta Targets #670
Handle timestamp_ntz in delta and iceberg by @vinishjail97 in Handle timestamp_ntz in delta and iceberg #647
[MINOR] Fix docker file syntax issue by @the-other-tim-brown in [MINOR] Fix docker file syntax issue #674
[528] Refactor RunSync Class for Improved Modularity and Reusability by @vaibhavk1992 in [528] Refactor RunSync Class for Improved Modularity and Reusability #675
Add how-to docs for catalog sync by @vinishjail97 in Add how-to docs for catalog sync #673
[MINOR] Add mailing list for discussions, update issues list in asf.yaml by @the-other-tim-brown in [MINOR] Add mailing list for discussions, update issues list in asf.yaml #676
[MINOR] Fix mailing list in asf.yaml by @the-other-tim-brown in [MINOR] Fix mailing list in asf.yaml #677
[678] Add contribution guidelines to reflect stale PR's guidelines by @vinishjail97 in [678] Add contribution guidelines to reflect stale PR's guidelines #684
[678] Avoid skipping xtable-aws and xtable-hive-metastore in mvn deploy by @vinishjail97 in [678] Avoid skipping xtable-aws and xtable-hive-metastore in mvn deploy #679
[678] Introduce maturity-level in xtable-api and Refactor SyncStatusCode into separate class by @vinishjail97 in [678] Introduce maturity-level in xtable-api and Refactor SyncStatusCode into separate class #685
Bump org.apache.parquet:parquet-avro from 1.12.3 to 1.15.1 by @dependabot in Bump org.apache.parquet:parquet-avro from 1.12.3 to 1.15.1 #686
Enable squash and merge in .asf.yaml for xtable by @vinishjail97 in Enable squash and merge in .asf.yaml for xtable #688
[678] Add LICENSE for xtable-aws module by @vinishjail97 in [678] Add LICENSE for xtable-aws module #690
[678] Add LICENSE for xtable-hive-metastore module by @vinishjail97 in [678] Add LICENSE for xtable-hive-metastore module #691
Docs: Update Footer Copyright Year by @omkenge in Docs: Update Footer Copyright Year #654
[692] Include 2025 in NOTICE file by @vinishjail97 in [692] Include 2025 in NOTICE file #695
[692] Exclude jol-core in xtable bundled dependencies by @vinishjail97 in [692] Exclude jol-core in xtable bundled dependencies #696
Remove parameters from Glue partition columns by @kroushan-nit in Remove parameters from Glue partition columns #697
[692] Ensure LICENSE content is different for bundled jars by @vinishjail97 in [692] Ensure LICENSE content is different for bundled jars #699
[DOCS]: Add 3 New Community Blogs by @dipankarmazumdar in [DOCS]: Add 3 New Community Blogs #706
Add open api REST spec for XTable Service by @rahil-c in Add open api REST spec for XTable Service #705
[BLOG]: Add the new Catalog sync blog by @dipankarmazumdar in [BLOG]: Add the new Catalog sync blog #708
[700] Update license files for bundled jars by @vinishjail97 in [700] Update license files for bundled jars #707
Add latest metadata path field in InternalTable by @rahil-c in Add latest metadata path field in InternalTable #710
[700] Update NOTICE file for bundled jars by @vinishjail97 in [700] Update NOTICE file for bundled jars #709
Add implementation for XTable REST Service by @rahil-c in Add implementation for XTable REST Service #704

New Contributors

@kroushan-nit made their first contribution in [590] Hadoop and AWS sdk version upgrade #614
@bhasudha made their first contribution in [site] Update blogs #635
@brishi19791 made their first contribution in Modified incremental safe check which does not depend on snapshots list order #665
@vaibhavk1992 made their first contribution in [528] Refactor RunSync Class for Improved Modularity and Reusability #675
@omkenge made their first contribution in Docs: Update Footer Copyright Year #654
@rahil-c made their first contribution in Add open api REST spec for XTable Service #705

Full Changelog: 0.2.0-incubating...0.3.0-incubating-rc3

XTable 3rd Release #678

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions