Serverless on LocalStack

I’ve recently had occasion to start writing a series of services on the AWS stack using the Serverless Framework. Serverless is a great framework, but I really don’t like having to deploy stuff to AWS to test DynamoboDB streams, SQS queues, etc. That’s where LocalStack comes in. LocalStack lets you host an entire AWS ecosystem locally so you can test “all the things” without actually deploying anything.

LocalStack works great, but I discovered there are a few undocumented things that you’ll need to know to get your stuff working correctly locally.

1. Deployment Buckets

For whatever reason, I couldn’t get a Serverless function to properly deploy to LocalStack without including the Deployment Bucket plug-in for serverless. After installing the plug-in and doing the configuration below everything “justed worked.”

serverless.yml

provider
	deploymentBucket: 
		name: ${self:service}-${opt:stage}-deployment-bucket 
		serverSideEncryption: AES256

2. LocalStack Confuses Boto3

The Boto3 documentation makes it seem like getting to AWS resources in your code is as easy as boto3.resource(…). That is definitely the case when you’re on AWS, but on LocalStack, you’ll need to do a little more.

Basically, boto3 is super confused about running locally. When you try and get to a resource, it’ll throw some authentication issues. That’s because it doesn’t actually know where to look for the resources. You need to pass an endpoint_url to boto3 to let it know where LocalStack has things.

# Helper method to return a database handle.
def get_db_handle(handle: str):
   dbh = boto3.resource("dynamodb")
   if os.getenv("RUN_ENV", "local") == "local":
       dbh = boto3.resource(
           "dynamodb",
           endpoint_url="http://" + os.getenv("LOCALSTACK_HOSTNAME") + ":4566",
       )

   table = dbh.Table(os.getenv(handle))
   return table

3. Rest Calls are Annoying

Because the ID for the gateway keeps changing each time you start up LocalStack, it’s difficult to programatically know the URL for another Lambda you want to make a REST call to. Boto3 helps here a little, but if you go to add an API key to the header… good luck.

Through some trial and error I discovered that you can build your own request object and put it into the payload in a Boto3 Lambda invocation.

import json
import os

import boto3
from botocore.config import Config

…

# Helper method to post the message to the api
def post_to_gateway(msg):
   lambda_client = get_sls_client()

   lambda_dets = lambda_client.get_function(
       FunctionName=f"special-api-{os.getenv('RUN_ENV', 'local')}-api"
   )

   request = {
       "path": "/custom-path",
       "httpMethod": "POST",
       "headers": {
           "Accept": "*/*", 
           "Content-Type": "application/json", 
           "x-api-key", os.getenv("API_KEY", "pico")
       },
       "body": json.dumps(msg),
       "isBase64Encoded": False,
   }

   response = lambda_client.invoke(
       FunctionName=lambda_dets["Configuration"]["FunctionArn"],
       InvocationType="RequestResponse",
       Payload=json.dumps(request),
   )

   result = json.loads(response.get("Payload").read())

   return result

# Helper method to return a Lambda client.
def get_sls_client():
   client = boto3.client("lambda")
   if os.getenv("RUN_ENV", "local") == "local":
       cfg = Config(
           retries={"max_attempts": 10}, read_timeout=180, connect_timeout=180
       )
       client = boto3.client(
           "lambda",
           endpoint_url="http://" + os.getenv("LOCALSTACK_HOSTNAME") + ":4566",
           config=cfg,
       )

   return client

Yes, you read that right, you’ve got to json.dumps() the body and then the whole object a second time. It doesn’t work otherwise.

Also, you’ll notice I increase the timeout when running locally. At least on my machine, LocalStack takes a good bit of resources. It’s not instant, so increasing the time out reduces false failures (due to LocalStack taking too long) when testing.

Serverless Services

LocalStack requires you to list out all the various AWS services you’ll be needing to emulate at startup. Looking at the documentation you’d think you’d just need serverless, sqs, and dynamodbstreams. WRONG! If you’re deploying a few Serverless apps working with SQS, DynamoDB tables and DynamoDB stream you’ll need the following:

SERVICES=serverless,cloudformation,sts,sqs,secretsmanager,dynamodbstreams DEFAULT_REGION=us-east-1 DEBUG=1 localstack start

I like running LocalStack in debug mode locally. The output is a little noisy, but it has really helped me debug my applications, especially when it comes to properly creating and calling resources.

Marmanold.com