Topstone Software Consulting

Amazon Web Services Lambda Functions in Java

Amazon Web Services

In the past supporting a web application required purchasing one or more computer systems to act as web servers. Additional hardware might include a load balancer to distribute web application load between the web servers and a database server. These computer systems would need to be housed in an air conditioned facility. An alarm system and other security measures would be needed to provide physical security. Enough Internet bandwidth would need to be purchased to support anticipated peak usage.

In addition to the capital costs for computer hardware and facilities, staff would be needed to manage the computer systems.

Amazon Web Services (AWS) has dramatically reduced the costs of web applications. There is no upfront cost for computing, internet bandwidth and database services (although money can be saved by prepaying for some services). The Amazon compute and RDS database services require zero maintenance.

Although AWS has been revolutionary in reducing the cost of web applications, the cost of running Elastic Compute Cloud (EC2) servers 24 hours a day, seven days a week (24/7) can still be significant (especially for a start-up company that is trying to conserve every penny spent). For AWS applications that are built to scale as web application demand increases, the cost of the application will increase as additional servers are added by the load balancer.

An application running on a web server will allocate multiple server threads to support the application users. When these users perform operations that consume memory or processing resources, the load balancer may allocate additional servers to handle the load. There is usually a time lag before a new server can come on-line to handle the increased load. To avoid this latency, additional servers (EC2 instances) that run 24/7 may be allocated to handle peak demand. The EC2 resources configured to handle peak load increase the cost of the web application and may be lightly utilized most of the time.

AWS Lambda

AWS Lambda allows memory or compute intensive operations to be offloaded to a Lambda function. Lambda functions are highly scalable and concurrent and can rapidly meet peak demand. Lambda function scaling does not impact the web server or cause additional EC2 resources to be added to the web application.

Depending on how frequently a Lambda function is called, the cost may be very low (in some cases, there is no cost). For example, at the time this was written, the free tier" (e.g., zero cost) for Amazon Lambda allows 1 million calls a month and 400,000 Gigabyte-seconds of compute. The free tier for Lambda is available indefinitely, for both new and existing AWS customers.

AWS Lambda can be applied to support a variety of AWS cloud functions. For example, Lambda can be used to provide mobile app services, removing the need for a web server (supporting so called "serverless computing"). Lambda functions can also be called as remote procedure calls, off-loading memory or compute intensive operations. This is the use-case that is discussed here.

Lambda Functions in Java

The AWS documentation on Lambda Java functions focuses on Lambda functions that are invoked in response to a defined AWS event. For example, a Lambda function could be invoked when an entry is made in a DynamoDB table or when a file is stored on AWS S3 storage. Although event driven Lambda functions are the use case often described in the Amazon documentation, Lambda functions do not have to be invoked in response to an event. A Lambda function can be invoked via a remote call.

Amazon provides the AWS Toolkit for Eclipse which makes it easy to build Java Lambda functions and download them to the Lambda service. The toolkit properly builds and downloads the code for the Lambda function, with all of the necessary Java "jar" files.

A simple Lambda function that echos an annotated copy of it's input is shown below. 

package com.amazonaws.lambda.lambda_example;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

/**
 * <p>
 * SimpleLambdaFunctionExample
 * </p>
 * <p>
 * An example of a minimal Lambda function that returns an annotated argument string.
 * </p>
 * <p>
 * Feb 16, 2018
 * </p>
 * 
 * @author Ian Kaplan, iank@bearcave.com
 */
public class SimpleLambdaFunctionExample implements RequestHandler<SimpleLambdaMessage, String> {
    public final static String lambdaFunctionName = "SimpleLambdaFunctionExample";

    @Override
    public String handleRequest(SimpleLambdaMessage message, Context context) {
        final String resultStr = "Returned from AWS Lambda. Input string was " + message.getMessage();
        return resultStr;
    }

}

The argument to a Lambda function must always be a Java object that can be serialized into a JSON string. In this case, the Java object is the SimpleLambdaMessage object shown below. 

package com.amazonaws.lambda.lambda_example;
/**
 * <h4>
 * SimpleLambdaMessage
 * </h4>
 * <p>
 * A class for sending message strings to an Amazon Web Services Lambda Function
 * </p>
 * <p>
 * Feb 19, 2018
 * </p>
 * 
 * @author Ian Kaplan, iank@bearcave.com
 */
public class SimpleLambdaMessage {
    private String mMessage;

    /**
     * @param message the message contents for the object
     */
    public void setMessage(final String message) {
        this.mMessage = message;
    }

    /**
     * @return the message string
     */
    public String getMessage() {
        return this.mMessage;
    }

    @Override
    public String toString() {
        return this.mMessage;
    }

}

When Eclipse is configured with the AWS Toolkit, the SimpleLambdaFunction.java file can be selected and "right clicked" to bring up an Eclipse menu. One of the menu items is "Amazon Web Services", which includes a sub-menu with the entry "Upload function to AWS Lambda". Selecting this item will invoke a set of dialogs to download the function to Lambda. Before downloading a function to Lambda, you will need to set up AWS IAM permissions and S3 storage to support the Lambda function. The details for doing this are beyond the scope of this discussion.

To call the function from a Java web application (or from jUnit test code) you will need to construct an AWS Lambda client and convert the function argument to JSON. An example is shown in the code below. To run this code you will need to fill in your own Amazon ID and secret key for the Lambda service. 

package com.amazonaws.lambda.lambda_example;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.logging.Logger;

import org.junit.Test;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.protocol.json.SdkJsonGenerator.JsonGenerationException;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.lambda.AWSLambda;
import com.amazonaws.services.lambda.AWSLambdaClientBuilder;
import com.amazonaws.services.lambda.model.InvokeRequest;
import com.amazonaws.services.lambda.model.InvokeResult;
import com.amazonaws.services.s3.model.Region;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;

/**
 * A simple test harness for invoking your Lambda function via a remote Lambda call.
 */
public class LambdaFunctionTest {

    /**
     * Build an Amazon Web Services Lambda client object.
     * 
     * @param accessID The AWS credential ID
     * @param accessKey The AWS credential secret key
     * @return An AWS Lambda client
     */
    private AWSLambda buildClient(String accessID, String accessKey) {
        Regions region = Regions.fromName(Region.US_West.toString());
        BasicAWSCredentials credentials = new BasicAWSCredentials(accessID, accessKey);
        AWSLambdaClientBuilder builder = AWSLambdaClientBuilder.standard()
                                         .withCredentials(new AWSStaticCredentialsProvider(credentials))
                                         .withRegion(region);
        AWSLambda client = builder.build();
        return client;
    }

    /**
     * See http://www.baeldung.com/jackson-inheritance
     * 
     * @param obj
     * @return
     * @throws JsonGenerationException
     * @throws JsonMappingException
     * @throws IOException
     */
    private static String objectToJSON( Object obj, Logger logger) {
        String json = "";
        try {
            ObjectMapper mapper = new ObjectMapper();
            json = mapper.writeValueAsString(obj);
        } catch (JsonGenerationException | IOException e) {
            logger.severe("Object to JSON failed: " + e.getLocalizedMessage());
        }
        return json;
    }

    @Test
    public void testLambdaFunction() {
        final String aws_access_key_id = "Your AWS ID goes here";
        final String aws_secret_access_key = "Your AWS secret key goes here";
        final Logger logger = Logger.getLogger( this.getClass().getName() );
        final String messageToLambda = "Hello Lambda Function";
        final SimpleLambdaMessage messageObj = new SimpleLambdaMessage();
        messageObj.setMessage(messageToLambda);

        AWSLambda lambdaClient = buildClient(aws_access_key_id, aws_secret_access_key);
        String lambdaMessageJSON = objectToJSON( messageObj, logger );
        InvokeRequest req = new InvokeRequest()
                               .withFunctionName(SimpleLambdaFunctionExample
                               .lambdaFunctionName)
                               .withPayload( lambdaMessageJSON );
        InvokeResult requestResult = lambdaClient.invoke(req);
        ByteBuffer byteBuf = requestResult.getPayload();
        if (byteBuf != null) {
            String result = StandardCharsets.UTF_8.decode(byteBuf).toString();
            logger.info("testLambdaFunction::Lambda result: " + result);
        } else {
            logger.severe("testLambdaFunction: result payload is null");
        }
    }
}

By default Eclipse will build the Lambda project as a Maven project. To compile the code to call the Lambda function you will need to include the following Maven dependency (the version for the Java SDK will, of course, change over time): 

        <dependency>
           <groupId>com.amazonaws</groupId>
           <artifactId>aws-java-sdk</artifactId>
           <version>1.11.253</version>
        </dependency>

Testing and Debugging Lambda Functions

Services like AWS Lambda allow memory and processing load to be moved to Lambda, which will scale as load increases. Lambda also allows mobile applications to be built with minimal server side resources. The attractive features of Lambda, Amazon's marketing and the herd mentality of the software industry has resulted in a fad for "Severless Computing". Most of the people who write about the wonders of serverless computing do not delve the fact that Lambda based applications are known by an older name: distributed applications.

Distributed applications are notoriously difficult to debug, since the application components are running on different computer systems connected via a computer network (in the case the Amazon "virtual private cloud" network).

Information about Lambda function execution and errors can be only obtained by examining the Lambda execution logs. The Context object that is an argument to a Lambda function has an associated logger that can be used to record errors and execution state. 

         LambdaLogger logger = context.getLogger();

The difficulty of diagnosing Lambda function errors by reading the Lambda execution logs should prompt the prudent developer to test the code running on Lambda as thoroughly as possible before it is integrated into an application. Java jUnit tests that run locally to test the code that is to run under Lambda should be written. Once these tests pass, tests that run the code under Lambda should be written. Only when the local and remote Lambda tests pass should the Lambda code be integrated into the application.

Java Image Processing with AWS Lambda

A common use case for AWS Lambda is image processing to generate thumbnail and scaled images from a larger digital photograph. Moving image processing from the web server to a Lambda function removes the memory and processor spikes that can be caused my image processing operations. This reduces web server load and can allow the application to run with fewer web servers. Lambda functions will smoothly scale and can support image processing operations in multiple web server threads.

The nderground social network (designed and built by Topstone Software) users can post photos on their board and in photo galleries. When a photo is downloaded to nderground, new versions of the photo are generated for thumbnail and medium scale photos.

When the photo gallery feature was added to nderground, additional servers were added to the AWS Elastic Beanstalk load balancer to manage the memory and processing load. By adding Lambda functions to scale images, we were able to reduce the number of servers, which has reduced the monthly cost for nderground.

Java Image Scaling Software on GitHub

Topstone software has published the Java image scaling software on GitHub (see IanLKaplan/LambdaImageProcessing). This software is available under the Apache 2 open source license.

The Java code was "pretty printed" for HTML display using http://hilite.me/

Amazon Cloud Architecture, Spring and Elasticsearch Consulting

Topstone Software has extensive experience building scalable web applications on Amazon Web Services. We can help you design your AWS application architecture to provide scalability and optimized cost.

We designed and built the nderground social network. nderground is a social network designed for privacy and security. nderground has been live with close to 100% uptime for over three years.

At Topstone Software we have experience building Spring framework applications that utilize a variety of AWS services. These include Elasticsearch, DynamoDB, S3, the Simple Email System and Elastic Beanstalk. We can provide the consulting help to speed your application development or we can develop applications to your specification.

Ian Kaplan, Topstone Software Consulting, February 2018