Skip to content

Pull upstream changes until 4.19.0 #467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Bouncheck
Copy link
Collaborator

@Bouncheck Bouncheck commented Mar 14, 2025

Copied from merge commit:

Merged "CASSANDRA-19635: Run integration tests with C* 5.x":

  • InventoryITBase slight adjustments
  • CcmBridge: skip updateJVMVersion changes

Merged "CASSANDRA-19635: Configure Jenkins to run integration tests with C* 5.x":

  • No impact on the fork. Modifies only jenkinsfile.

Skipped "update badge URL to org.apache.cassandra/java-driver-core":

  • Fork's badge remains as is.

Merged "Limit calls to Conversions.resolveExecutionProfile"
Skipped "autolink JIRA tickets in commit messages"
Merged "Don't return empty routing key when partition key is unbound"
Merged "JAVA-3167: CompletableFutures.allSuccessful() may return never completed future"
Merged "ninja-fix Various test fixes"
Merged "Run integration tests with DSE 6.9.0":

  • Allowed new Version definitions
  • Rejected some methods from CcmBridge for jvm version overrides

Merged "JAVA-3117: Call CcmCustomRule#after if CcmCustomRule#before fails to allow subsequent tests to run"
Merged "JAVA-3149: Support request cancellation in request throttler"
Merged "Fix C* 3.0 tests failing on Jenkins"
Merged "Reduce lock held duration in ConcurrencyLimitingRequestThrottler"
Merged "Annotate BatchStatement, Statement, SimpleStatement methods with CheckReturnValue"
Merged "Remove "beta" support for Java17 from docs"
Merged "Fix uncaught exception during graceful channel shutdown"
Merged "Build a public CI for Apache Cassandra Java Driver":

  • No impact on the fork. Changes unused jenkinsfiles and scripts.

Merged "CASSANDRA-19932: Allow to define extensions while creating table"
Merged "Fix DefaultSslEngineFactory missing null check on close"
Merged "Query builder support for NOT CQL syntax":

  • This is C* 5.1 feature. Not yet supported in ScyllaDB.

Merged "Fix CustomCcmRule to drop CURRENT flag no matter what"
Merged "JAVA-3051: Memory leak"
Merged "Automate latest Cassandra versions when running CI":

  • Similarly, has no impact on the fork.

Merged "Refactor integration tests to support multiple C* distributions. Test with DataStax HCD 1.0.0":

  • Added BackendType.SCYLLA
  • CcmBridge was adjusted to use a mix of variables made for our CI and upstream additions

Merged "Fix TableMetadata.describe() when containing a vector column"
Merged "Move Apache Cassandra 5.x off of beta1 and remove some older Apache Cassandra versions."

  • No impact on the fork.

Skipped "Update link to Jira to be CASSJAVA"
Merged "Move DataStax shaded Guava module into Java driver":

  • Adjusted groupId to be com.scylladb in our case
  • Adjusted plugins so that driver builds using same commands as before.

Merged "JAVA-3057 Allow decoding a UDT that has more fields than expected"
Merged "CASSJAVA-55 Remove setting "Host" header for metadata requests."
Merged "JAVA-3118: Add support for vector data type in Schema Builder, QueryBuilder"
Merged "Upgrade Guava to 33.3.1-jre"
Skipped "Conditionally compile shaded Guava module":

  • This was needed in the upstream's CI. Conflicted with fork's setup.

Merged "JAVA-3143: Extend driver vector support to arbitrary subtypes and fix handling of variable length types (OSS C* 5.0)"
Merged "JAVA-3168 Copy node info for contact points on initial node refresh only from first match by endpoint"
Merged "JAVA-3055: Prevent PreparedStatement cache to be polluted if a request is cancelled."
Merged "Expose a decorator for CqlPrepareAsyncProcessor cache rather than the ability to specify an arbitrary cache from scratch."
Merged "ninja-fix Using shaded Guava classes for import in order to make OSGi class paths happy."
Merged "Changelog updates for 4.19.0"
Skipped "[maven-release-plugin] prepare release 4.19.0"
Merged(*) "[maven-release-plugin] prepare for next development iteration":

  • Versions were manually set to 4.19.0.0-SNAPSHOT, overriding the changes in this commit
  • This commit will be the next merge base.

lukasz-antoniak and others added 30 commits June 3, 2024 13:48
patch by Lukasz Antoniak; reviewed by Andy Tolbert, and Bret McGuire for CASSANDRA-19635
patch by Lukasz Antoniak; reviewed by Bret McGuire for CASSANDRA-19635
Those repeated calls account for a non-negligible portion of my application
CPU (0.6%) and can definitly be a final field so that it gets resolved only
once per CqlRequestHandler.

patch by Benoit Tellier; reviewed by Andy Tolbert, and Bret McGuire
reference: apache#1623
patch by Stefan Miklosovic; reviewed by Michael Semb Wever for CASSANDRA-19854
DefaultBoundStatement#getRoutingKey has logic to infer the
routing key when no one has explicitly called setRoutingKey
or otherwise set the routing key on the statement.
It however doesn't check for cases where nothing has been
bound yet on the statement.
This causes more problems if the user decides to get a
BoundStatementBuilder from the PreparedStatement, set some
fields on it, and then copy it by constructing new
BoundStatementBuilder objects with the BoundStatement as a
parameter, since the empty ByteBuffer gets copied to all
bound statements, resulting in all requests being targeted
to the same Cassandra node in a token-aware load balancing
policy.

patch by Ammar Khaku; reviewed by Andy Tolbert, and Bret McGuire
reference: apache#1620
…ted future

patch by Lukasz Antoniak; reviewed by Andy Tolbert, and Bret McGuire for JAVA-3167
patch by Lukasz Antoniak; reviewed by Bret McGuire
reference: apache#1955
…allow subsequent tests to run

patch by Henry Hughes; reviewed by Alexandre Dutra and Andy Tolbert for JAVA-3117
patch by Lukasz Antoniak; reviewed by Andy Tolbert and Chris Lohfink for JAVA-3149
patch by Lukasz Antoniak; reviewed by Bret McGuire
reference: apache#1939
It might take some (small) time for callback handling when the
throttler request proceeds to submission.

Before this change, the throttler proceed request will happen while
holding the lock, preventing other tasks from proceeding when there is
spare capacity and even preventing tasks from enqueuing until the
callback completes.

By tracking the expected outcome, we can perform the callback outside
of the lock. This means that request registration and submission can
proceed even when a long callback is being processed.

patch by Jason Koch; Reviewed by Andy Tolbert and Chris Lohfink for CASSANDRA-19922
…kReturnValue

Since the driver's default implementation is for
BatchStatement and SimpleStatement methods to be immutable,
we should annotate those methods with @CheckReturnValue.
Statement#setNowInSeconds implementations are immutable so
annotate that too.

patch by Ammar Khaku; reviewed by Andy Tolbert and Bret McGuire
reference: apache#1607
patch by Bret McGuire; reviewed by Andy Tolbert and Alexandre Dutra
reference: apache#1962
after exceeding max orphan ids

patch by Christian Aistleitner; reviewed by Andy Tolbert, and Bret McGuire for apache#1938
 patch by Siyao (Jane) He; reviewed by Mick Semb Wever for CASSANDRA-19832
patch by Lukasz Antoniak; reviewed by Bret McGuire and Chris Lohfink
patch by Abe Ratnofsky; reviewed by Andy Tolbert and Chris Lohfink for CASSANDRA-20001
patch by Bret McGuire; reviewed by Bret McGuire and Andy Tolbert for CASSANDRA-19930
If super.after() throws an Exception `CURRENT` flag is never dropped
which leads next tests to fail with IllegalStateException("Attempting to use a Ccm rule while another is in use.  This is disallowed")

Patch by Dmitry Kropachev; reviewed by Andy Tolbert and Bret McGuire for JAVA-3117
patch by Jane He; reviewed by Alexandre Dutra and Bret McGuire for JAVA-3051
 patch by Siyao (Jane) He; reviewed by Mick Semb Wever for CASSJAVA-25
… with DataStax HCD 1.0.0

patch by Lukasz Antoniak; reviewed by Bret McGuire
reference: apache#1958
patch by Stefan Miklosovic; reviewed by Bret McGuire for CASSJAVA-2
…assandra versions.

patch by Bret McGuire; reviewed by Bret McGuire for CASSJAVA-54
Updating the link to Jira.

Previously we had a component in the CASSANDRA Jira project but now we have a project for each driver - in the case of Java, it's CASSJAVA.

Added CASSJAVA to .asf.yaml

patch by Jeremy Hanna; reviewed by Bret McGuire for CASSJAVA-61
patch by Lukasz Antoniak; reviewed by Alexandre Dutra and Bret McGuire for CASSJAVA-52
patch by Ammar Khaku; reviewed by Andy Tolbert and Bret McGuire
reference: apache#1635
With some sysprops enabled this will actually be respected which completely borks Astra routing.

patch by Bret McGuire; reviewed by Alexandre Dutra and Bret McGuire for CASSJAVA-55
SiyaoIsHiding and others added 14 commits January 7, 2025 09:05
…uilder

patch by Jane He; reviewed by Mick Semb Wever and Bret McGuire for JAVA-3118
reference: apache#1931
patch by Lukasz Antoniak; reviewed by Alexandre Dutra and Bret McGuire for CASSJAVA-53
… handling of variable length types (OSS C* 5.0)

patch by Jane He; reviewed by Bret McGuire and João Reis
reference: apache#1952
…nly from first match by endpoint

patch by Alex Sasnouskikh; reviewed by Andy Tolbert and Alexandre Dura for JAVA-3168
…t is cancelled.

There was a critical issue when the external code cancels a request, indeed the cached CompletableFuture will then always throw a CancellationException.
This may happens, for example, when used by reactive like Mono.zip or Mono.firstWithValue.

patch by Luc Boutier; reviewed by Alexandre Dutra and Bret McGuire
reference: apache#1757
… ability to specify an

arbitrary cache from scratch.

Also bringing tests from apache#2003 forward
with a few minor changes due to this implementation

patch by Bret McGuire; reviewed by Bret McGuire and Andy Tolbert
reference: apache#2008
… class paths happy.

Major hat tip to Dmitry Konstantinov for the find here!
Merge commit '90612f6758eb0f0ba964daf054f397a47a90a736'
of https://github.com/apache/cassandra-java-driver
into scylla-4.x-pull-upstream-4.19.0-cleaned

Following changes were made:

Merged "CASSANDRA-19635: Run integration tests with C* 5.x":
  - InventoryITBase slight adjustments
  - CcmBridge: skip updateJVMVersion changes
Merged "CASSANDRA-19635: Configure Jenkins to run integration tests with C* 5.x":
  - No impact on the fork. Modifies only jenkinsfile.
Skipped "update badge URL to org.apache.cassandra/java-driver-core":
  - Fork's badge remains as is.
Merged "Limit calls to Conversions.resolveExecutionProfile"
Skipped "autolink JIRA tickets in commit messages"
Merged "Don't return empty routing key when partition key is unbound"
Merged "JAVA-3167: CompletableFutures.allSuccessful() may return never completed future"
Merged "ninja-fix Various test fixes"
Merged "Run integration tests with DSE 6.9.0":
  - Allowed new Version definitions
  - Rejected some methods from CcmBridge for jvm version overrides
Merged "JAVA-3117: Call CcmCustomRule#after if CcmCustomRule#before fails to allow subsequent tests to run"
Merged "JAVA-3149: Support request cancellation in request throttler"
Merged "Fix C* 3.0 tests failing on Jenkins"
Merged "Reduce lock held duration in ConcurrencyLimitingRequestThrottler"
Merged "Annotate BatchStatement, Statement, SimpleStatement methods with CheckReturnValue"
Merged "Remove "beta" support for Java17 from docs"
Merged "Fix uncaught exception during graceful channel shutdown"
Merged "Build a public CI for Apache Cassandra Java Driver":
  - No impact on the fork. Changes unused jenkinsfiles and scripts.
Merged "CASSANDRA-19932: Allow to define extensions while creating table"
Merged "Fix DefaultSslEngineFactory missing null check on close"
Merged "Query builder support for NOT CQL syntax":
  - This is C* 5.1 feature. Not yet supported in ScyllaDB.
Merged "Fix CustomCcmRule to drop CURRENT flag no matter what"
Merged "JAVA-3051: Memory leak"
Merged "Automate latest Cassandra versions when running CI":
  - Similarly, has no impact on the fork.
Merged "Refactor integration tests to support multiple C* distributions. Test with DataStax HCD 1.0.0":
  - Added `BackendType.SCYLLA`
  - CcmBridge was adjusted to use a mix of variables made for our CI and upstream additions
Merged "Fix TableMetadata.describe() when containing a vector column"
Merged "Move Apache Cassandra 5.x off of beta1 and remove some older Apache Cassandra versions."
  - No impact on the fork.
Skipped "Update link to Jira to be CASSJAVA"
Merged "Move DataStax shaded Guava module into Java driver":
  - Adjusted groupId to be com.scylladb in our case
  - Adjusted plugins so that driver builds using same commands as before.
Merged "JAVA-3057 Allow decoding a UDT that has more fields than expected"
Merged "CASSJAVA-55 Remove setting "Host" header for metadata requests."
Merged "JAVA-3118: Add support for vector data type in Schema Builder, QueryBuilder"
Merged "Upgrade Guava to 33.3.1-jre"
Skipped "Conditionally compile shaded Guava module":
  - This was needed in the upstream's CI. Conflicted with fork's setup.
Merged "JAVA-3143: Extend driver vector support to arbitrary subtypes and fix handling of variable length types (OSS C* 5.0)"
Merged "JAVA-3168 Copy node info for contact points on initial node refresh only from first match by endpoint"
Merged "JAVA-3055: Prevent PreparedStatement cache to be polluted if a request is cancelled."
Merged "Expose a decorator for CqlPrepareAsyncProcessor cache rather than the ability to specify an arbitrary cache from scratch."
Merged "ninja-fix Using shaded Guava classes for import in order to make OSGi class paths happy."
Merged "Changelog updates for 4.19.0"
Skipped "[maven-release-plugin] prepare release 4.19.0"
Merged(*) "[maven-release-plugin] prepare for next development iteration":
  - Versions were manually set to 4.19.0.0-SNAPSHOT, overriding the changes in this commit
  - This (90612f6) will be the next merge base.
@Bouncheck Bouncheck self-assigned this Mar 14, 2025
@Bouncheck Bouncheck marked this pull request as draft March 14, 2025 15:03
Sets the version to 3.4.1 in parent pom.
Having it unspecified causes the following warning:
```
[WARNING] Some problems were encountered while building the effective model for com.scylladb:java-driver-guava-shaded:jar:4.19.0.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-clean-plugin is missing. @ line 97, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
```
Without this module available "Build" and "Unit tests" job fail with
"package <x> does not exist" or "cannot find symbol" pointing to
`[...].shaded.guava.[...]` packages.
Since incorporating "JAVA-3057 Allow decoding a UDT that has more fields than
expected" the asuumptions of removed check are no longer valid.
@Bouncheck Bouncheck force-pushed the scylla-4.x-pull-upstream-4.19.0-cleaned branch from 1d08256 to 823d4fe Compare March 17, 2025 12:20
Switches from `org.apache.cassandra` to `com.scylladb`
in `BundleOptions#commonBundles()`.
@Bouncheck Bouncheck marked this pull request as ready for review March 17, 2025 16:22
@Bouncheck
Copy link
Collaborator Author

Added some post rebase adjustments.
Had to rerun CI because of flaky tests.

@Bouncheck Bouncheck requested a review from dkropachev March 17, 2025 16:26
Copy link
Collaborator

@dkropachev dkropachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Bouncheck , great job.

@dkropachev dkropachev merged commit cf9f8f0 into scylladb:scylla-4.x Mar 19, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.