Today I Learned

3 posts about #serverless

Careful sending complex objects to lambda callback

Ran into an issue today with the following code. It's the second or third time it's bitten me, and I think this time I finally am learning the general guideline: don't pass complex objects into your lambda success/failure callbacks!

export async function main(event, context, callback) {
  const id = event.pathParameters.id;
  try {
    const result = await startExecution(
      process.env.statemachine_arn, JSON.stringify({ id })
    );
    callback(null, success({ status: true, result }));
  } catch (error) {
    console.log(error.message);
    callback(error, failure({ status: false, message: error.message }));
  }
}

This lambda will fail with Converting circular structure to JSON exception, because the result object returned from the startExecution function is a big complex object that doesn't work with JSON.stringify. I've had the same thing happen to me with the results of axios requests, as well as other AWS service calls.

To fix, don't pass result but instead pass only what you need returned, or simply true.

How to properly do SNS in Serverless

Figuring this out has taken a few hours out of my day.

Assumption

Your system is comprised of multiple CloudFormation stacks (aka Serverless Framework services aka microservices)

What not to do

functions:
  dispatcher:
    handler: dispatcher.dispatch
    events:
      - sns: 'demo_submitted'

It is critically important to not create SNS topic subscriptions on the fly as lambda event triggers. The reason is that Serverless framework tries to be helpful by implicitly auto-magically creating resource declarations for those topics, effectively within the boundaries of the consumer service. Unless parts of your microservice are talking amongst themselves using SNS (which would be weird), you usually subscribe to SNS topics that originate in other services, not the one where you are consuming it.

Why not to do it

Let's say you try to subscribe to the same topic with a lambda in another service OR you go down the road of explicitly declaring topics as resources in the services that broadcast to the topics AND you've implicitly created those very same SNS topics already.

Then when you try to deploy you will get an error that looks something like this:

Serverless Error ---------------------------------------
  An error occurred: DemoSubmittedTopic - demo_submitted already exists in stack arn:aws:cloudformation:us-east-1:423:stack/producers-service-dev/5332a.

Resources must be unique and in this case, you already created that SNS topic in the consumer, not where it belongs and is expected to live.

What to do instead

When you subscribe a lambda to an SNS topic, use the ARN notation as described in https://serverless.com/framework/docs/providers/aws/events/sns#using-a-pre-existing-topic

functions:
  dispatcher:
    handler: dispatcher.dispatch
    events:
      - sns:
          arn: arn:xxx

The ARN notation in the subscription prevents the auto-magical undesirable creation of an SNS resource.

Vanity columns in join tables

Today as I was looking at a table in DynamoDB via the AWS console. This table is basically a queue of items for a particular user to look at or dismiss, and right now it only has two key columns: one for the user, and one for the item.

It struck me that it would be worthwhile to add a few extra columns to this table just to make administration and debugging a little easier. Namely, we could add the name of the user and the name of the item. At worst the tradeoff is that the read/write throughput for the table would be a little higher.