Skip to content

Shareable function for partitioning integration tests #17223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yaauie merged 1 commit into from
Mar 12, 2025
Merged

Shareable function for partitioning integration tests #17223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions ci/get-test-half.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash

# get_test_half returns either the first or second half of integration tests
# Usage: get_test_half <half_number>
# half_number: 0 for first half, 1 for second half
get_test_half() {
Copy link
Contributor

@mashhurs mashhurs Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making separate function with file looks cleaner from maintenance perspective. Can we make this function a bit abstracted in a way to split specs into N chunks?
If it is something initial version, LGTM!

Copy link
Member

@yaauie yaauie Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mashhurs yes, that is the plan.

@donoghuc and I discovered we were both improving the flow, so he PR'd this so that I can rebase my #17219 on it and get a clean merge into the feature branch.

local half_number=$1
# Ensure only spec files go to stdout
pushd qa/integration >/dev/null 2>&1

# Collect all spec files
local glob1=(specs/*spec.rb)
local glob2=(specs/**/*spec.rb)
local all_specs=("${glob1[@]}" "${glob2[@]}")

# Calculate the split point
local split_point=$((${#all_specs[@]} / 2))

# Get the requested half (:: is "up to", : is "from")
if [[ $half_number -eq 0 ]]; then
local specs="${all_specs[@]::$split_point}"
else
local specs="${all_specs[@]:$split_point}"
fi
popd >/dev/null 2>&1
echo "$specs"
}
21 changes: 7 additions & 14 deletions ci/integration_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ export GRADLE_OPTS="-Xmx2g -Dorg.gradle.jvmargs=-Xmx2g -Dorg.gradle.daemon=false
export SPEC_OPTS="--order rand --format documentation"
export CI=true

# Source shared function for splitting integration tests
source "$(dirname "${BASH_SOURCE[0]}")/get-test-half.sh"

if [ -n "$BUILD_JAVA_HOME" ]; then
GRADLE_OPTS="$GRADLE_OPTS -Dorg.gradle.java.home=$BUILD_JAVA_HOME"
fi
Expand All @@ -19,20 +22,10 @@ if [[ $1 = "setup" ]]; then
exit 0

elif [[ $1 == "split" ]]; then
cd qa/integration
glob1=(specs/*spec.rb)
glob2=(specs/**/*spec.rb)
all_specs=("${glob1[@]}" "${glob2[@]}")

specs0=${all_specs[@]::$((${#all_specs[@]} / 2 ))}
specs1=${all_specs[@]:$((${#all_specs[@]} / 2 ))}
cd ../..
if [[ $2 == 0 ]]; then
echo "Running the first half of integration specs: $specs0"
./gradlew runIntegrationTests -PrubyIntegrationSpecs="$specs0" --console=plain
elif [[ $2 == 1 ]]; then
echo "Running the second half of integration specs: $specs1"
./gradlew runIntegrationTests -PrubyIntegrationSpecs="$specs1" --console=plain
if [[ $2 =~ ^[01]$ ]]; then
specs=$(get_test_half "$2")
echo "Running half $2 of integration specs: $specs"
./gradlew runIntegrationTests -PrubyIntegrationSpecs="$specs" --console=plain
else
echo "Error, must specify 0 or 1 after the split. For example ci/integration_tests.sh split 0"
exit 1
Expand Down