AWS Nitro Enclaves with Kubernetes

This repository contains a collection of tools that can be used to build and run AWS Nitro Enclaves applications with Amazon Elastic Kubernetes Service (EKS).

The userguide for AWS Nitro Enclaves with Kubernetes (K8s) can be found here.

Overview

There are two NE (Nitro Enclaves) applications in this repository which can be built and deployed in a Kubernetes deployment.

• hello: A simple application that prints periodically on the Nitro Enclave debug console
• kmstool: A kms application that is able to connect to KMS from inside the Nitro Enclave and decrypt an encrypted KMS message received from the outside world

Dependencies like docker, jq, eksctl and kubectl are required for configuring, building and deploying various NE applications.

Use the enclavectl tool from this repository to create EKS clusters, build NE applications and do deployments.

To add the tool to your $PATH variable, use:

source env.sh

See enclavectl help for all the supported options.

The default settings for enclavectl are stored in the local settings.json file.

In this file the following input can be provided:

• AWS region
• Instance type
• EKS cluster name
• EKS nodegroup name
• EKS nodegroup desired capacity
• K8s version
• CPUs per node to reserve for Nitro Enclaves
• Memory per node to reserve for Nitro Enclaves

Here is an example of a configuration file:

{
  "region" : "eu-central-1",
  "instance_type" : "m5.2xlarge",
  "eks_cluster_name" : "eks-ne-cluster",
  "eks_worker_node_name" : "eks-ne-nodegroup",
  "eks_worker_node_capacity" : "1",
  "k8s_version" : "1.22",
  "node_enclave_cpu_limit": 2,
  "node_enclave_memory_limit_mib": 768
}

Building and running the hello example

1. Adapt the configuration and apply it to the project:

enclavectl configure --file settings.json

After finishing, the tool confirms a successful configuration like below

[enclavectl] Configuration finished successfully.

2. Create a Nitro Enclave aware EKS cluster. This will use the EnclaveOptions=true parameter in the EC2 launch template that shall be used on the cluster nodegroup:

enclavectl setup

This high-level command consists of three internal steps:

• Generates a basic EC2 Launch Template for Nitro Enclaves and UserData
• Creates an EKS cluster with a managed node-group of configured capacity
• Deploys the Nitro Enclaves K8s Device plugin: This plugin enables Kubernetes pods to access Nitro Enclaves device driver. As part of this step, the plugin is deployed as a daemonset to the cluster.

3. Build the hello enclave application

Usually, applications run on EKS clusters in containers. A Nitro Enclave applications need one more step for running in an enclave - it needs to be packaged in an Enclave Image File (EIF). To get more information about building EIFs, please take a look at this user guide.

To trigger a build, use:

enclavectl build --image hello

This phase makes use of a builder docker container which builds the targeted application if it is present in the container directory and packages it in a ready-to-deploy container. All application deliverables, including the Nitro Enclave EIF, are put in the container/bin/ folder.

4. Push the hello enclave application to a remote repository For deploying, a docker repository shall be required. We will be using Amazon Elastic Container Registry (ECR) for this purpose.

enclavectl push --image hello

Unless it has already been created, this also creates a repository under your private ECR and pushes the hello image to it. For the subsequent uses, the command will always use the previously created repository.

5. Deploy the hello enclave application

This command does necessary pre-initialization (if exists) for the application before deployment and generates deployment specification.

enclavectl run --image hello --prepare-only

To see the contents of the deployment specification, use:

cat hello_deployment.yaml

Finally, to start the actual deployment, use:

kubectl apply -f hello_deployment.yaml

The above steps can be done via a single command:

enclavectl run --image hello

6. Check the application logs

To check deployment status of the hello application, use

kubectl get pods --selector app=hello --watch

After a while the command is expected to report a similar output like below after a short while:

NAME                               READY   STATUS              RESTARTS   AGE
hello-deployment-7bf5d9b79-qv8vm   0/1     Pending             0          2s
hello-deployment-7bf5d9b79-qv8vm   0/1     Pending             0          27s
hello-deployment-7bf5d9b79-qv8vm   0/1     ContainerCreating   0          27s
hello-deployment-7bf5d9b79-qv8vm   1/1     Running             0          30s

When you ensure that the application is running, press "Ctrl + C" to terminate kubectl and check the logs:

kubectl logs hello-deployment-7bf5d9b79-qv8vm

You can find the <deployment name> from your terminal logs. After successful execution of this command, you will see output like this below.

[   1] Hello from the enclave side!
[   2] Hello from the enclave side!
[   3] Hello from the enclave side!
[   4] Hello from the enclave side!
[   5] Hello from the enclave side!

The application keeps printing "Hello from the enclave side!" message every 5 seconds.

7. Stop the application logs

To clear the previous deployment, use:

enclavectl stop --image hello

This function not only executes kubectl -f delete hello_deployment.yaml in the background, but also uninitalizes resources if any were initialized after enclavectl run command.

Building and running the kmstool example

KMS Tool is an example application that uses is able to connect to KMS and decrypt an encrypted KMS message.

NOTE: The user would be required to create a role which is associated with the EC2 instance that has permissions to access the KMS service in order to create a key, encrypt a message and decrypt the message inside the enclave. This is the way and the recommended way Nitro Enclaves are used today by users. More information in the doc listed at the beginning of the section.

For this demo application in EKS, we already have a role associated with the instance but those permissions do not apply to the Kubernetes containers. In order to resolve this, we require a service account that has all the required permissions with KMS in order to issue a successful KMS Decrypt from inside the Nitro Enclave.

As an important note, AWS currently supports one enclave per EC2 instance. Before moving on, please ensure you stopped the previous hello deployment.

To run KMS example, please follow the similar steps below as you did for the hello application.

enclavectl build --image kms
enclavectl push --image kms
enclavectl run --image kms
kubectl get pods --selector app=kms --watch

And check the logs to see that the enclave has decrypted the message:

kubectl logs <kms-deployment-name>

After successful execution of this command, you will see output like this below. (User specific data has been truncated)

[kms-example] Creating a KMS key...
[kms-example] Encrypting message...
[kms-example] ******************************
[kms-example] KMS Key ARN: arn:aws:kms:[...]
[kms-example] Account ID: [...]
[kms-example] Unencrypted message: Hello, KMS\!
[kms-example] Ciphertext: AQICAHg7LT9PYQzAhL3hhzA4N15Lsok7f4DEEPGiNf8fyUM+5QHHy85xZXBek7uFPtNX+vJyAAAAZDBiBgkqhkiG9w0BBwagVTBTAgEAME4GCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMspJf9GEN1DqaJ55sAgEQgCGorI4UgmAAwmhfgsuXIud/PcTKwt8K8L/aPyj8Hq6KIVo=
[kms-example] ******************************
Start allocating memory...
Started enclave with enclave-cid: 18, memory: 128 MiB, cpu-ids: [1, 5]
[kms-example] Requesting from the enclave to decrypt message...
[kms-example] ------------------------
[kms-example] > Got response from the enclave!
[kms-example] Object = { "Status": "Ok" } Object = { "Status": "Ok", "Message": "HelloKMS" }
[kms-example] ------------------------
Successfully terminated enclave i-[...]-enc[...]

Creating your own example application

To quickly create your own application within this tutorial, you need to perform a few more steps. All application specific data is stored under the container folder. The hello can be a good example to see what kind of files are required for your application. To see more information, please check this document.

Cleaning up AWS resources

If you followed this tutorial partially or entirely, it must have created some AWS resources. To clean them up, use:

enclavectl cleanup

Security issue notifications

If you discover a potential security issue, we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public GitHub issue.