Skip to content

feat(ci): Partition workflow scripts updated to work #6900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jun 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/layers_partition_verify.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Parition Layer Verification
# Partition Layer Verification
# ---
# This workflow queries the Parition layer info in production only
# This workflow queries the Partition layer info in production only

on:
workflow_dispatch:
Expand Down Expand Up @@ -51,11 +51,11 @@ jobs:
runs-on: ubuntu-latest
outputs:
regions: ${{ format('{0}{1}', steps.regions_china.outputs.regions, steps.regions_govcloud.outputs.regions) }}
parition: ${{ format('{0}{1}', steps.regions_china.outputs.partition, steps.regions_govcloud.outputs.parition) }}
partition: ${{ format('{0}{1}', steps.regions_china.outputs.partition, steps.regions_govcloud.outputs.partition) }}
aud: ${{ format('{0}{1}', steps.regions_china.outputs.aud, steps.regions_govcloud.outputs.aud) }}
steps:
- id: regions_china
name: Parition (China)
name: Partition (China)
if: ${{ inputs.partition == 'China' }}
run: |
echo regions='["cn-north-1"]'>> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -95,7 +95,7 @@ jobs:
- name: Output ${{ matrix.layer }}-${{ matrix.arch }}
# fetch the specific layer version information from the us-east-1 commercial region
run: |
aws --region us-east-1 lambda get-layer-version-by-arn --arn 'arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }}' | jq -r '{"Layer Version Arn": .LayerVersionArn, "Version": .Version, "Description": .Description, "Compatible Runtimes": .CompatibleRuntimes[0], "Compatible Architectures": .CompatibleArchitectures[0], "SHA": .Content.CodeSha256} | keys[] as $k | [$k, .[$k]] | @tsv' | column -t -s $'\t'
aws --region us-east-1 lambda get-layer-version-by-arn --arn 'arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }}' > '${{ matrix.layer }}-${{ matrix.arch }}.json'
- name: Store Metadata
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
Expand Down Expand Up @@ -149,7 +149,7 @@ jobs:
- name: Verify Layer
run: |
export layer_output='${{ matrix.layer }}-${{ matrix.arch }}-${{matrix.region}}.json'
aws --region ${{ matrix.region}} lambda get-layer-version-by-arn --arn "arn:${{ needs.setup.outputs.parition }}:lambda:${{ matrix.region}}:${{ secrets[format('AWS_ACCOUNT_{0}', steps.transform.outputs.CONVERTED_REGION)] }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ steps.partition_version.outputs.partition_version }}" > $layer_output
aws --region ${{ matrix.region}} lambda get-layer-version-by-arn --arn "arn:${{ needs.setup.outputs.partition }}:lambda:${{ matrix.region}}:${{ secrets[format('AWS_ACCOUNT_{0}', steps.transform.outputs.CONVERTED_REGION)] }}:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ steps.partition_version.outputs.partition_version }}" > $layer_output
REMOTE_SHA=$(jq -r '.Content.CodeSha256' $layer_output)
LOCAL_SHA=$(jq -r '.Content.CodeSha256' ${{ matrix.layer }}-${{ matrix.arch }}.json)
test "$REMOTE_SHA" == "$LOCAL_SHA" && echo "SHA OK: ${LOCAL_SHA}" || exit 1
Expand Down
20 changes: 10 additions & 10 deletions .github/workflows/layers_partitions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,20 +91,20 @@ jobs:
mask-aws-account-id: true
- name: Grab Zip
run: |
aws --region us-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} --query 'Content.Location' | xargs curl -L -o ${{ matrix.layer }}_${{ matrix.arch }}.zip
aws --region us-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} > ${{ matrix.layer }}_${{ matrix.arch }}.json
aws --region us-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} --query 'Content.Location' | xargs curl -L -o ${{ matrix.layer }}-${{ matrix.arch }}.zip
aws --region us-east-1 lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:017000801446:layer:${{ matrix.layer }}-${{ matrix.arch }}:${{ inputs.version }} > ${{ matrix.layer }}-${{ matrix.arch }}.json
- name: Store Zip
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.zip
path: ${{ matrix.layer }}_${{ matrix.arch }}.zip
name: ${{ matrix.layer }}-${{ matrix.arch }}.zip
path: ${{ matrix.layer }}-${{ matrix.arch }}.zip
retention-days: 1
if-no-files-found: error
- name: Store Metadata
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.json
path: ${{ matrix.layer }}_${{ matrix.arch }}.json
name: ${{ matrix.layer }}-${{ matrix.arch }}.json
path: ${{ matrix.layer }}-${{ matrix.arch }}.json
retention-days: 1
if-no-files-found: error

Expand Down Expand Up @@ -135,15 +135,15 @@ jobs:
- name: Download Zip
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.zip
name: ${{ matrix.layer }}-${{ matrix.arch }}.zip
- name: Download Metadata
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: ${{ matrix.layer }}_${{ matrix.arch }}.json
name: ${{ matrix.layer }}-${{ matrix.arch }}.json
- name: Verify Layer Signature
run: |
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}_${{ matrix.arch }}.json')
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}_${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
SHA=$(jq -r '.Content.CodeSha256' '${{ matrix.layer }}-${{ matrix.arch }}.json')
test "$(openssl dgst -sha256 -binary ${{ matrix.layer }}-${{ matrix.arch }}.zip | openssl enc -base64)" == "$SHA" && echo "SHA OK: ${SHA}" || exit 1
- id: transform
run: |
echo 'CONVERTED_REGION=${{ matrix.region }}' | tr 'a-z\-' 'A-Z_' >> "$GITHUB_OUTPUT"
Expand Down
13 changes: 13 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,19 @@ You can install Powertools for AWS Lambda (Python) using your favorite dependenc

You'll find the pre-signed URL under `Location` key as part of the CLI command output.

=== "Lambda Layer (China)"

[Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html){target="_blank"} is a .zip file archive that can contain additional code, pre-packaged dependencies, data, or configuration files. We compile and optimize [all dependencies](#install), and remove duplicate dependencies [already available in the Lambda runtime](https://github.com/aws-powertools/powertools-lambda-python/blob/develop/layer_v3/docker/Dockerfile#L34){target="_blank"} to achieve the most optimal size.

For the latter, make sure to replace `{python_version}` without the period (.), e.g., `python313` for `Python 3.13`.

**AWS China Beijing (cn-north-1)**

| Architecture | Layer ARN |
| ------------ | --------------------------------------------------------------------------------------------------------- |
| x86_64 | __arn:aws-cn:lambda:cn-north-1:498634801083:layer:AWSLambdaPowertoolsPythonV3-{python_version}-x86_64:18__{: .copyMe}:clipboard: |
| ARM | __arn:aws-cn:lambda:cn-north-1:498634801083:layer:AWSLambdaPowertoolsPythonV3-{python_version}-arm64:18__{: .copyMe}:clipboard: |

=== "Lambda Layer (GovCloud)"

[Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html){target="_blank"} is a .zip file archive that can contain additional code, pre-packaged dependencies, data, or configuration files. We compile and optimize [all dependencies](#install), and remove duplicate dependencies [already available in the Lambda runtime](https://github.com/aws-powertools/powertools-lambda-python/blob/develop/layer_v3/docker/Dockerfile#L34){target="_blank"} to achieve the most optimal size.
Expand Down