fixed conflicts

keithrozario · Jun 2, 2024 · d5e698d · d5e698d
2 parents 51d89dc + 45497b1
commit d5e698d
Show file tree

Hide file tree

Showing 23 changed files with 1,076 additions and 426 deletions.
diff --git a/.github/workflows/container_builds.yml b/.github/workflows/container_builds.yml
@@ -108,6 +108,7 @@ jobs:
           echo BUILD_DIR=$BUILD_DIR >> $GITHUB_ENV
   
           cp ./pipeline/container_images/build_images/common/build.py ./$BUILD_DIR
+          cp ./pipeline/container_images/build_images/common/build_optimized.py ./$BUILD_DIR
 
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2

diff --git a/README.MD b/README.MD
@@ -2,7 +2,7 @@
 
 🐍 A collection of Python Packages as AWS Lambda(λ) Layers 🐍
 
-[![Python 3.8](https://img.shields.io/badge/python-3.8-green.svg)](https://www.python.org/downloads/release/python-380/) [![Python 3.8](https://img.shields.io/badge/python-3.9-green.svg)](https://www.python.org/downloads/release/python-390/) [![Python 3.10](https://img.shields.io/badge/python-3.10-green.svg)](https://www.python.org/downloads/release/python-3100/)[![Python 3.11](https://img.shields.io/badge/python-3.11-green.svg)](https://www.python.org/downloads/release/python-3110/) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![Python 3.10](https://img.shields.io/badge/python-3.10-green?style=for-the-badge)](https://www.python.org/downloads/release/python-3100/)  [![Python 3.11](https://img.shields.io/badge/python-3.11-green?style=for-the-badge)](https://www.python.org/downloads/release/python-3110/)  [![Python 3.12](https://img.shields.io/badge/python-3.12-green?style=for-the-badge)](https://www.python.org/downloads/release/python-3120/)  [![Code style: black](https://img.shields.io/badge/code%20style-black-000000?style=for-the-badge)](https://github.com/psf/black)
 
   -   [List of ARNs](#list-of-arns)
   -   [Python Packages](#python-packages)
@@ -12,6 +12,7 @@
       -   [Option 3: Using Serverless Framework](#option-3-using-serverless-framework)
       -   [Option 4: Using AWS Serverless Application Model (SAM)](#option-4-using-aws-serverless-application-model-sam)
       -   [Option 5: Using Terraform with the Klayer provider](#option-5-using-terraform-with-the-klayer-provider)
+      -   [Option 6: Using CDK with cdk-klayers](#option-6-using-cdk-with-cdk-klayers)
   -   [Status of layers](#status-of-layers)
   -   [Layer expiry](#layer-expiry)
   -   [Architecture Diagram](#architecture-diagram)
@@ -25,25 +26,25 @@
 
 List of the latest layer version arns are available by region:
 
-* [Python 3.8](deployments/python3.8)
-* [Python 3.9](deployments/python3.9)
 * [Python 3.10](deployments/python3.10)
 * [Python 3.11](deployments/python3.11)
+* [Python 3.12](deployments/python3.12)
 * [Python 3.10-arm64](deployments/python3.10-arm64)
 * [Python 3.11-arm64](deployments/python3.11-arm64)
+* [Python 3.12-arm64](deployments/python3.12-arm64)
 
-*Note: We have deprecated layers for python3.6 and python 3.7 and will deprecate 3.8 shortly. Please use python 3.10 going forward.*
+*Note: We have deprecated layers for python3.6, python3.7, python 3.8 and python3.9, Please use the latest version of python (currently python3.12) going forward*
 
 ## Python Packages
 
 For the full list of Python packages, refer to the following, feel free to make a `pull requests` modifying the files below to requests for a specific package.
 
-* [Python 3.8](pipeline/config/packages_p38.csv)
-* [Python 3.9](pipeline/config/packages_p39.csv)
 * [Python 3.10](pipeline/config/packages_p310.csv)
 * [Python 3.11](pipeline/config/packages_p311.csv)
+* [Python 3.12](pipeline/config/packages_p312.csv)
 * [Python 3.10-arm64](pipeline/config/packages_p310-arm64.csv)
 * [Python 3.11-arm64](pipeline/config/packages_p311-arm64.csv)
+* [Python 3.12-arm64](pipeline/config/packages_p312-arm64.csv)
 
 ## Using the Layers
 
@@ -131,6 +132,42 @@ resource "aws_lambda_function" "test_lambda" {
 }
 ```
 
+### Option 6: Using CDK with cdk-klayers
+
+Using CDK, you can use the [cdk-klayers](https://github.com/keithrozario/cdk-klayers) package to help you pull in the latest layers for your Stack or App.
+
+```python
+
+from cdk_klayers import Klayers
+
+class MockStack(Stack):
+
+    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
+        super().__init__(scope, construct_id, **kwargs)
+
+        runtime = aws_lambda.Runtime.PYTHON_3_12
+
+        # Initialize Klayers Class
+        klayers = Klayers(
+            self,
+            python_version = runtime
+            region = "ap-southeast-1"
+        )
+
+        # get the latest layer version for the requests package
+        requests_layer = klayers.layer_version(self, "requests")
+
+        lambda_function = aws_lambda.Function(
+            self, 'HelloHandler',
+            runtime=runtime,
+            layers=[requests_layer],
+            code=aws_lambda.Code.from_asset('lambda'),
+            handler='hello.handler'
+        )
+
+```
+
+
 ## Status of layers
 
 Layers are built with the latest package version at 2am UTC on the first day of the Month.

diff --git a/deployments/python3.12-arm64/README.MD b/deployments/python3.12-arm64/README.MD
@@ -0,0 +1,26 @@
+# Layers for Python 3.12-arm64
+
+
+| Region (Code) | Region (Name)| ARNs|
+| :------------- |:--------|:--------|
+| af-south-1 |Africa (Cape Town)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/af-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/af-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/af-south-1/html)|
+| ap-east-1 | Asia Pacific (Hong Kong)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-east-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-east-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-east-1/html)|
+| ap-northeast-1 |Asia Pacific (Tokyo)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-northeast-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-northeast-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-northeast-1/html)|
+| ap-northeast-2 |Asia Pacific (Seoul)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-northeast-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-northeast-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-northeast-2/html)|
+| ap-south-1 |Asia Pacific (Mumbai)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-south-1/html)|
+| ap-southeast-1 |Asia Pacific (Singapore)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-1/html)|
+| ap-southeast-2 |Asia Pacific (Sydney)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-2/html)|
+| ap-southeast-3 |Asia Pacific (Jakarta)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-3/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-3/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ap-southeast-3/html)|
+| ca-central-1 |Canada (Central)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ca-central-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ca-central-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/ca-central-1/html)|
+| eu-central-1 |EU (Frankfurt)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-central-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-central-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-central-1/html)|
+| eu-north-1 |EU (Stockholm)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-north-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-north-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-north-1/html)|
+| eu-south-1 |EU (Milan)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-south-1/html)|
+| eu-west-1 |EU (Ireland)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-1/html)|
+| eu-west-2 |EU (London)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-2/html)|
+| eu-west-3 |EU (Paris)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-3/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-3/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/eu-west-3/html)|
+| me-south-1 |Middle East (Bahrain)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/me-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/me-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/me-south-1/html)|
+| sa-east-1 |South America (São Paulo)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/sa-east-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/sa-east-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/sa-east-1/html)|
+| us-east-1 |US East (N. Virginia)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-east-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-east-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-east-1/html)|
+| us-east-2 |US East (Ohio)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-east-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-east-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-east-2/html)|
+| us-west-1 |US West (N. California)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-west-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-west-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-west-1/html)|
+| us-west-2 |US West (Oregon)| [json](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-west-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-west-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12-arm64/layers/latest/us-west-2/html)|
diff --git a/deployments/python3.12/README.MD b/deployments/python3.12/README.MD
@@ -0,0 +1,26 @@
+# Layers for Python 3.11
+
+
+| Region (Code) | Region (Name)| ARNs|
+| :------------- |:--------|:--------|
+| af-south-1 |Africa (Cape Town)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/af-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/af-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/af-south-1/html)|
+| ap-east-1 | Asia Pacific (Hong Kong)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-east-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-east-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-east-1/html)|
+| ap-northeast-1 |Asia Pacific (Tokyo)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-northeast-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-northeast-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-northeast-1/html)|
+| ap-northeast-2 |Asia Pacific (Seoul)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-northeast-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-northeast-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-northeast-2/html)|
+| ap-south-1 |Asia Pacific (Mumbai)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-south-1/html)|
+| ap-southeast-1 |Asia Pacific (Singapore)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-1/html)|
+| ap-southeast-2 |Asia Pacific (Sydney)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-2/html)|
+| ap-southeast-3 |Asia Pacific (Jakarta)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-3/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-3/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ap-southeast-3/html)|
+| ca-central-1 |Canada (Central)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ca-central-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ca-central-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/ca-central-1/html)|
+| eu-central-1 |EU (Frankfurt)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-central-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-central-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-central-1/html)|
+| eu-north-1 |EU (Stockholm)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-north-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-north-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-north-1/html)|
+| eu-south-1 |EU (Milan)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-south-1/html)|
+| eu-west-1 |EU (Ireland)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-1/html)|
+| eu-west-2 |EU (London)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-2/html)|
+| eu-west-3 |EU (Paris)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-3/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-3/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/eu-west-3/html)|
+| me-south-1 |Middle East (Bahrain)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/me-south-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/me-south-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/me-south-1/html)|
+| sa-east-1 |South America (São Paulo)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/sa-east-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/sa-east-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/sa-east-1/html)|
+| us-east-1 |US East (N. Virginia)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-east-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-east-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-east-1/html)|
+| us-east-2 |US East (Ohio)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-east-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-east-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-east-2/html)|
+| us-west-1 |US West (N. California)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-west-1/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-west-1/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-west-1/html)|
+| us-west-2 |US West (Oregon)| [json](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-west-2/json) \| [csv](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-west-2/csv) \| [html](https://api.klayers.cloud/api/v2/p3.12/layers/latest/us-west-2/html)|
diff --git a/pipeline/Serverless/README.MD b/pipeline/Serverless/README.MD
@@ -9,3 +9,11 @@ Functions and their corresponding `.yml` files are stored in individual folders.
 ## Step Functions
 
 Step function configuration uses the `serverless-step-functions` plugin, with corresponding `.yml` files stored in the /state_machines folder.
+
+## Installation
+
+serverless plugin install -n serverless-step-functions
+serverless plugin install -n serverless-iam-roles-per-function
+serverless plugin install -n serverless-better-credentials
+
+export AWS_SDK_LOAD_CONFIG=1