Serverless with AWS Lambda & Scala

About a year ago, I started looking at AWS's serverless offering, AWS Lambda. The premise is relatively simple, rather than a full server running that you manage and deploy your docker/web servers to, you just define a single function endpoint and map that to the API gateway and you have an infinitely* scale-able endpoint.

The appeal is fairly obvious - no maintenance or upgrading servers, fully scalable and pay per second of usage (so no cost for AWS Lambda functions that you define whilst not being called). I haven't looked into the performance of using the JVM based Lambda functions, but my assumption is that there will be potential performance costs if your function isn't frequently used, as AWS will have to start up the function, load its dependencies etc, so depending on your use case, it would be advisable to look at performance bench marking before putting into production use.

When I first looked into AWS Lambda a year ago, it was less mature than it is today, and dealing with input/output JSON objects required annotating POJOs, so I decided to start putting together a small library to make it easier to work with AWS Lambda in a more idiomatic Scala way - using Circe and it's automatic encoder/decoder generation with Shapeless. The code is all available on github.

Getting Started

To deploy on AWS I used a framework called Serverless - this is a really easy framework to setup serverless functions on a range of cloud providers. Once you have followed the pre-requisite install steps, you can simply run:

serverless create --template aws-java-gradle 

This will generate you a Java (JVM) based gradle project template, with a YML configuration file in the root that defines your endpoints and function call. If you look in the src folder as well, you will also see the classes for a very simple function that you can deploy and check your Lambda works as expected (you should also take the time at this point to login to your AWS console and have a look at what has been created in the Lambda and API Gateway sections. You should now be able to curl your API endpoint (or use the serverless cli with a command like: serverless invoke -f YOUR_FUNCTION__NAME -l).

ScaLambda - AWS Lambda with idiomatic Scala

Ok, so we have a nice simple Java based AWS Lambda function deployed and working, let's looking at moving it to Scala. As you try to build an API in this way you will need to be able to define endpoints that can receive inbound JSON being posted as well as return fixed JSON structures - AWS provides its inbuilt de/serialisation support, but inevitably you will have a type that might need further customisation of how it is de/serialized (UUIDs maybe, custom date formats etc) and there are a few nice libraries that can handle this stuff and Scala has some nice ways that can simplify this.

We can simply upgrade our new Java project to a Scala one (either convert the build.gradle to an sbt file, or just add Scala dependency/plugins to the build file as is) and then add the dependency:

We can now update the input/output classes so they are just normal Scala case classes:

Not a huge change from the POJOs we had, but is both more idiomatic and also means you can use case classes that you have in other existing Scala projects/libraries elsewhere in your tech stack.

Next we can update the request handler - this will also result in quite similar looking code to the original generated Java code, but will be in Scala and will be backed by Circe and it's automatic JSON encoder/decoder derivation.

You will see that similar to the AWS Java class we define generic parameter types for the class that represents the input case class and the output case class and then you simply implement the handleRequest method which expects the input class and returns the output response.

You might notice the return type is wrapped in the ApiResponse class - this is simply an alias for a Scala Either[Exception, T] - which means if you need to respond with an error from your function you can just return an exception rather than the TestOutput. To simplify this, there is an ApiResponse companion object that provides a success and failure method:

All the JSON serialisation/de-serialisation will use Circe's auto derived code which relies on Shapeless - if you use custom types that cannot be automatically derived, then you can just define implicit encoder/decoders for your type and they will be used.

Error handling

The library also has support for error handling - as the ApiResponse class supports returning exceptions, we need to map those exceptions back to something that can be returned by our API. To support this, the Controller class that we have implemented for our Lambda function expects (via self type annotations) to be provided an implementation of the ExceptionHandlerComponent trait and of the ResponseSerializerComponent trait.

Out of the box, the library provides a default implementation of each of these that can be used, but they can easily be replaced with custom implementations to handle any custom exception handling required:

Custom response envelopes

We mentioned above that the we also need to provide an implementation of the ResponseSerializerComponent trait. A common pattern in building APIs is the need to wrap all response messages in a custom envelope or response wrapper - we might want to include status codes or additional metadata (paging, rate limiting etc) - this is the job of the ResponseSerializerComponent. The default implementation simply wraps the response inside a basic response message with a status code included, but this could easily be extended/changed as needed.


The project is still in early stages of exploring the AWS Lambda stuff, but hopefully is starting to provide a useful approach to idiomatic Scala with AWS Lambda functions, allowing re-use of error handling and serialisation so you can just focus on the business logic required for the function.