March 2020 Summaries

We've implemented a new serverless continuous integration (CI) architecture that significantly reduces test execution time while preserving the isolation of each test, without incurring additional costs. Our approach involves using multiple AWS profiles to create isolated environments instantly, and leveraging parallelism across these accounts to reduce deployment duration. We've also updated our CI tool, "the Uber deploy", to support multiple profile names, fast-failure detection, and retries, as well as improved handling of concurrent executions on limited resources. By running integration and E2E tests concurrently across a number of isolated accounts, we can significantly improve the efficiency of our deployment process while maintaining the robustness of our CI/CD pipeline.

This article discusses the importance of adding a staging environment to a CI/CD process for critical services, and how Lumigo implemented this step in their serverless-focused workflow. The staging environment allows testing with real data in a less supervised environment, ensuring that features function smoothly in production. To automate deployment to the staging environment and release using serverless, Lumigo uses CircleCI jobs to trigger deployment, release beta versions to NPM, and deploy the staging environment. They also use AWS Step Functions to create a delay between releasing beta versions and final releases, allowing for stoppability and automatic release if necessary. The article concludes that adding a staging environment is a vitally important step in critical services, and that Lumigo will continue to share their experience and learnings on serverless CI/CD workflows.

AWS has announced that its API Gateway HTTP API is now Generally Available. This marks an important milestone in the development of the service, which offers a more efficient and cost-effective alternative to traditional REST APIs. The new version includes several key features, including stage and route-level throttling, mix-and-match support for REST and HTTP APIs under the same custom domain name, payload format version 2.0, integration with private resources such as ALBs, NLBs, and CloudMap, simplified Lambda response handling, and improved security capabilities through zero-trust networking and authentication methods like AWS IAM and Cognito User Pool. The Lumigo platform also supports HTTP APIs, allowing users to monitor and optimize their serverless environments more effectively. Overall, the new features aim to reduce the disparity between REST and HTTP APIs, making it easier for developers to choose the most suitable option for their use cases.

The Serverless Framework is an application framework designed to help build, deploy, and maintain serverless code. It offers integration with multiple cloud function providers, allowing for a multi-cloud approach and flexibility in the development pipeline. The framework provides dozens of plugins that can be incorporated into the pipeline, handling various deployment situations and architectures. Some essential plugins mentioned include Python Requirements, Serverless AWS Alerts, Lumigo Serverless, IAM Roles per Function, Cognito Custom Attributes, Reqvalidator, Offline, Step Functions, Serverless Webpack, and Serverless WSGI. These plugins improve serverless delivery speed, security, monitoring, debugging, and deployment efficiency. The Serverless Framework's extensibility is a key benefit, allowing developers to choose the right tools for their specific needs.

EventBridge offers several advantages over SNS for service integration on AWS, including supporting a wider variety of targets, delivering events across accounts with ease, capturing and processing third-party events, content-based filtering capabilities, built-in schema discovery, and input transformation. These features enable more efficient and scalable event-driven architectures, reducing the need for custom glue code. While not suitable for every application integration problem, EventBridge's expanded capabilities make it a popular alternative to SNS.

The Amazon Builders' Library article "Instrumenting distributed systems for operational visibility" by David Yanacek explores the importance of instrumentation in understanding how a system works, particularly in distributed systems. The article highlights that traditional logging tools such as cat, grep, sed, and awk are not sufficient for serverless applications, instead emphasizing the need for specialized tools to create a real map of interacting resources. Instrumentation frees developers from manually logging every statement, automatically recording important data points to debug crashes or improve performance. The article also discusses the role of trace IDs, logs, alarms, and thresholds in instrumentation, as well as best practices such as trimming large requests, protecting sensitive data, and pruning unnecessary log statements.