AWS S3 server-side encryption

Recently, a client asked me to develop standards for them for AWS S3. They are primarily an Azure shop but, like most enterprises these days, they have some apps and business units which use AWS. As I documented what I already knew and researched what I didn’t, I was (once again) blown away by the capabilities of this massive cloud-scale network file system.

AWS S3 is the original AWS service. It was released in March, 2006. With over 15 years of usage, it has become not only the reference cloud storage service, it has had a lot — a whole lot — of features added to it over the 15 years it has been available.

And that brings us to the reason for this post. Rather than documenting everything one could do in AWS S3 to secure and encrypt their data, I decided to mandate very specific S3 encryption and security requirements for my client. Even narrowed to what I assert are S3 encryption best practices, app designers might find the choices confusing. So, I wrote a couple of “reference” scripts that create buckets that comply with the security and encryption standards that are most useful for everyday usage in the typical enterprise. Here, I’ve combined and expanded those scripts. I’ll bet they’ll work for you, too. I’ve tried to put the “simple” back in Simple Storage Service when it comes to data encryption and bucket security.

Opinions are like…

But be aware: this script is opinionated. It can only create S3 buckets with server-side encryption (SSE). It will turn off all S3 bucket public access. It doesn’t care that you want client-side encryption because I have decided that’s a mistake for the vast majority of enterprise users. It doesn’t care that you want to use an AWS S3-managed AWS Key Management Service (KMS) encryption key with server-side KMS encryption; it forces you to use a customer master key (CMK) because I assumed you would only use KMS encryption if you needed to permit cross-AWS-account access to the bucket.

IOW, if you don’t like what this script does, go ahead and tell me I’m full of it in the comments. But I ain’t changing my mind. 🙂

So, here’s what this script will do. And because it’s in AWS PowerShell, you could directly map the calls to the AWS CLI and/or pretty much any other AWS SDK. I think AWS PowerShell works as both a pseudo-language and an implementable language to make the concepts clear. (I do wish that AWS PowerShell cmdlets would respect -ErrorAction.)

Let’s EnScript: create a secure S3 bucket

The AWS PowerShell script below:

  • Creates an S3 bucket
  • Turns off all public access to that bucket
  • Enables either S3 server-side encryption with S3 managed keys (SSE-S3) or S3 server-side encryption with KMS using a CMK (SSE-KMS)
  • Enables bucket keys to reduce KMS API call costs
  • Retrieves the S3 server-side encryption and bucket keys settings and displays them on the PowerShell console.

The best way to explore this S3 bucket encryption script is to save it to disk (I called my script CreateSecureS3Bucket.ps1, then enter help .\CreateSecureS3Bucket.ps1. The PowerShell comment-based help should explain what’s going on. I also made lots of comments in the code. Most of the script is about showing what’s going on in the PowerShell console. The actual work of creating, securing and encrypting S3 buckets with server-side encryption is pretty simple.

I think this script does what you should do if you are thinking about S3 encryption — use server-side encryption, use S3 bucket keys that AWS manages and for gosh sakes, turn off public access.

AWS PowerShell script to create, secure and encrypt a new S3 bucket

<#
.SYNOPSIS
Creates an S3 bucket using either SSE-S3 or SSE-KMS encryption and makes the bucket non-public.

.DESCRIPTION
From among the many encryption and security options for S3 buckets, this script has an opinionated function. It will create an S3 bucket in the currently set default AWS region with either SSE-S3 or SSE-KMS encyption. If SSE-KMS is selected, a valid ARN for a KMS CMK must be specified. In both cases, the bucket is made non-public.

.EXAMPLE
# Create a non-public S3 bucket with SSE-KMS
./CreateSecureS3Bucket -bucketName alex-test-bucket-1 -sseKms -cmkArn arn:aws:kms:us-east-1:1111222223333:key/44444-5555-6666-7777-88888888

.EXAMPLE
# Create a non-public S3 bucket with SSE-S3
./CreateSecureS3Bucket -bucketName alex-test-bucket-1 -sseS3

.PARAMETER bucketName
The name of the bucket to be created

.PARAMETER sseKms
Switch to select SSE-KMS encryption. Mutually exclusive with SSE-S3

.PARAMETER sseS3
Switch to select SSE-S3 encryption. Mutually exclusive with SSE-KMS

.PARAMETER cmkArn
String containing the full (not alias) ARN of a customer master key in the region the script is accessing

.NOTES
This script assumes you have run Initialize-AWSDefaultConfiguration to specify the region you wish to have the S3 bucket created in. 


 Alex Neihaus 2021-04-04
	(c) 2021 Air11 Technology LLC -- licensed under the Apache OpenSource 2.0 license, https://opensource.org/licenses/Apache-2.0
	Licensed under the Apache License, Version 2.0 (the "License");
	you may not use this file except in compliance with the License.
	You may obtain a copy of the License at
	http://www.apache.org/licenses/LICENSE-2.0
	
	Unless required by applicable law or agreed to in writing, software
	distributed under the License is distributed on an "AS IS" BASIS,
	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	See the License for the specific language governing permissions and
	limitations under the License.
	
	Author's blog: https://www.yobyot.com
#>
#Requires -Modules @{ModuleName = "AWS.Tools.Common"; ModuleVersion = "4.1.10.0"}, @{ModuleName = "AWS.Tools.S3"; ModuleVersion = "4.1.10.0"}
[CmdletBinding()]
param (
    # Name of the bucket to be created with secure defaults
    [Parameter(Mandatory = $true)]
    [String]
    $bucketName,
    # Switch to select SSE-KMS default encryption
    [Parameter(Mandatory = $false)]
    [Switch]
    $sseKms,
    # Switch to select SSE-S3 default encryption
    [Parameter(Mandatory = $false)]
    [Switch]
    $sseS3,
    # If $SseKms is $true, then a customer master key must be specified. This is checked in the script
    [Parameter(Mandatory = $false)]
    [string]
    $cmkArn
)

function CreateS3Bucket ($bkt)
{
    # Create the S3 bucket
    New-S3Bucket -BucketName $bkt
}
<#
This function disables all public access; nobody should be using S3 publicly, including for static websites. :-)
#>
function DisablePublicAccess ($bkt)
{
    # Turn off all public access
    $bkt.BucketName | Add-S3PublicAccessBlock `
        -PublicAccessBlockConfiguration_BlockPublicAcl $true `
        -PublicAccessBlockConfiguration_BlockPublicPolicy $true `
        -PublicAccessBlockConfiguration_IgnorePublicAcl $true `
        -PublicAccessBlockConfiguration_RestrictPublicBucket $true
}
function EnableServerSideEncryption ($bkt)
{
    # Enable SSE based on the contents of $encryptionConfig
    $newS3Bucket | Set-S3BucketEncryption `
        -ServerSideEncryptionConfiguration_ServerSideEncryptionRule $encryptionConfig
}

# Set the credentials to be used if needed
# Set-AWSCredential -ProfileName AProfileStoredIn$HOME/.aws/credentials
# Set the default region if needed
# Set-DefaultAWSRegion -Region us-east-1

# Create the the encryption options in a hasthable modeled on an object of type
#  Amazon.S3.Model.ServerSideEncryptionByDefault
# This hashtable will create a bucket with SSE-KMS or SSE-S3 encryption
# Bucket keys are always enabled
$encryptionConfig =
@{
    ServerSideEncryptionByDefault = @{
        # ServerSideEncryptionAlgorithm = "" Will be added based on parmeter specified, either "aws:kms" or "AES256"
        # ServerSideEncryptionKeyManagementServiceKeyId = $cmkArn Will be added if aws:kms is specified
    }
    BucketKeyEnabled              = $true # Bucket keys are always enabled
}


# Check the parameter switches
switch ($true) 
{
    ($sseKms.IsPresent -and $sseS3.IsPresent) 
    {
        Write-Host "Both SSE-S3 and SSE-KMS were selected. They are mutually exclusive. Exiting" -ForegroundColor Red
        Exit
    }
    $sseS3.IsPresent
    {
        "SSE-S3 was selected" 
        "Creating S3 bucket $bucketName with SSE-S3 encryption"
        # Update hashtable with encryption defaults
        $encryptionConfig.ServerSideEncryptionByDefault.Add("ServerSideEncryptionAlgorithm", "AES256")
        $newS3Bucket = CreateS3Bucket $bucketName
        DisablePublicAccess $newS3Bucket
        EnableServerSideEncryption $newS3Bucket
    }
    $sseKms.IsPresent
    {
        if ($cmkArn)
        {
            "Specified SSE-KMS"
            # Check to see if the ARN of the CMK is valid.
            # The ARN MUST BE in this format: arn:aws:kms:region:acct-id:key/key-id"
            try
            {
                # Make sure $cmkArn exists. Get-KMSKey does not respect -ErrorAction SilentlyContinue so a try block is required
                "Testing if CMK ARN is valid"
                Get-KMSKey -KeyId $cmkArn
            }
            catch
            {
                Write-Host "CMK ARN is invalid or key not found. Is it in this region? Exiting" -ForegroundColor Red
                Exit
            } 
            # If we get to here, the ARN is valid for this region.
            # Update hashtable with encryption defaults
            $encryptionConfig.ServerSideEncryptionByDefault.Add("ServerSideEncryptionAlgorithm", "aws:kms")
            $encryptionConfig.ServerSideEncryptionByDefault.Add("ServerSideEncryptionKeyManagementServiceKeyId", $cmkArn)
            "Creating S3 bucket with encryption using CMK $cmkArn"
            $newS3Bucket = CreateS3Bucket $bucketName
            DisablePublicAccess $newS3Bucket
            EnableServerSideEncryption $newS3Bucket

        }   
        else 
        {
            Write-Host "No CMK specified but SSE-KMS requested. Exiting" -ForegroundColor Red
            Exit
        }
    }
    Default
    {
        Write-Host "Neither SSE-S3 nor SSE-KMS was selected. One form of SSE must be selected. Exiting" -ForegroundColor Red
        Exit
    }
}

# Display encryption settings for the new bucket on the console for confirmation
Switch ((Get-S3BucketEncryption -BucketName $newS3Bucket.BucketName).ServerSideEncryptionRules.BucketKeyEnabled)
{
    $true
    {
        "Bucket keys are enabled for $($newS3Bucket.BucketName)"
    }
    $false
    {
        "Bucket keys are NOT enabled for $($newS3Bucket.BucketName)"
    }
    Default
    {
        "Unable to determine the state of bucket keys for $($newS3Bucket.BucketName)"
    }
}
Switch ((Get-S3BucketEncryption -BucketName $newS3Bucket.BucketName).ServerSideEncryptionRules.ServerSideEncryptionByDefault.ServerSideEncryptionAlgorithm)
{
    "AES256"
    {
        "$($newS3Bucket.BucketName) is using SSE-S3 default encryption"
    }
    "aws:kms"
    {
        "$($newS3Bucket.BucketName) is using SSE-KMS default encryption"
    }
    Default
    {
        "Unable to determine the state of encryption for $($newS3Bucket.BucketName)"
    }

Blast from the past

If you’ve made it this far, you might be interested in this now antique PowerShell script that calculates the sizes of S3 buckets.


Posted

in

, ,

by

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *