April 2019 Summaries

Serverless from the Trenches is a series of blog posts that aims to highlight real-world problems faced by developers and DevOps teams when dealing with serverless, focusing on day-to-day issues such as deployment and stress-testing. The authors share their experience with protecting CloudFormation stacks from accidental deletion in production environments, highlighting the importance of adding an extra layer of protection beyond AWS-provided termination protection. They automate this process using a simple bash script that updates termination protection during the deployment process, making it easier to safeguard against accidental deletions and ensuring a safer environment for serverless applications.

AWS Lambda allows deployment and scaling of individual functions, but engineers often think in terms of services. A service is a cohesive group of functions deployed together as a unit through CloudFormation, making it easier to reason about large systems and maintain autonomy among teams. Deployment frameworks like Serverless or AWS SAM help configure and deploy functions in terms of services. The question of how to organize functions into repositories is crucial, with two common approaches: monorepo (everything in one repo) and one per service (each service gets its own repo). Monorepos are productive for small teams but can become unfeasible as the organization grows due to coherence penalty issues. One repo per service avoids these challenges but incurs overhead from setting up new repos. The choice between monorepo and one repo per service depends on the organization's growth stage, velocity requirements, and existing codebase. For startups with high velocity needs, a monorepo might be suitable, while enterprises should consider sticking to their established approach or splitting the monorepo if necessary.

LaunchDarkly is a feature flag management system that helps companies implement continuous deployment, A/B testing, and infrastructure migrations. It enables canary launches through its built-in support for percentage-based rollouts. However, using persistent connections to LaunchDarkly's streaming API poses concerns due to traffic-based routing limitations and potential issues with concurrent executions exceeding the maximum number of server connections allowed by the Starter or Professional packages. To mitigate these concerns, polling mode can be used, but it introduces time windows where inconsistencies may exist between functions in a call chain. LaunchDarkly also enables canary deployments using feature toggles, which differs from weighted alias-based approaches. The system is easy to integrate with and has an intuitive configuration, but its design is not optimized for Lambda-based backend systems. Workarounds like the node-dynamodb-store and ld-relay projects allow for a proxy layer that relays updates to a DynamoDB table, eliminating the need for persistent connections.

AWS introduced Lambda Layers at re:invent 2018 as a way to share code and data between functions within and across different accounts. It's a useful tool for sharing large dependencies, such as FFmpeg and Pandoc executables, or data-heavy layers like MaxMind database or SQLite database files. However, it introduces new challenges, including harder to invoke functions locally, test functions, and deal with changes in the layer. Lambda Layers require additional security considerations and introduce new tooling challenges. They do not work well with static languages such as C# and Java, which requires all the application code and dependencies during the compilation process. The versioning scheme is basic and does not support semantic versioning, making it harder to tell if it's safe to upgrade to a new version of a layer. When a layer or a version of a layer is deleted, functions that depend on them can continue to operate, but updating the functions until you upgrade to a newer version or migrate off of the layer is not straightforward. Overall, Lambda Layers should be used for specific AWS Lambda use cases, such as large dependencies or custom runtime, and are a great way to improve deployment speed by automating the bundling and publishing of dependencies.

The Serverless Architecture Conference in The Hague, Netherlands, brought together experts from around the world to discuss the latest trends and challenges in serverless technologies. Discussions centered around the relationship between serverless and DevOps, with experts emphasizing that serverless is a natural progression towards infrastructure as code. Containers and microservices were also discussed, with panelists arguing that they are not dependent on each other. Cloud vendor lock-in was debated, with some experts arguing that it's overstated and others suggesting that it can be beneficial in certain situations. The conference also explored the concept of native, a serverless environment built on Kubernetes, which aims to reduce vendor lock-in while providing flexibility and portability. Additionally, security was discussed, with experts highlighting the need for application-level security measures to protect against common attack vectors such as event-data injections and unauthorized deployment. The conference concluded with presentations on scalability, serverless microservice patterns, and building resilient serverless systems with non-serverless components. Overall, the conference provided valuable insights into the challenges and opportunities of serverless technologies.

The open source applications described in this article are designed to simplify the process of shipping logs from CloudWatch Logs to external services. The first application, auto-subscribe-log-group-to-arn, automates the subscription of log groups to a destination ARN, handling permissions complexity and allowing for customization of the subscription filter. A second application, auto-update retention policy, automates the update of retention policies for log groups, helping users save on storage costs by automatically archiving logs after a specified period. Both applications are available from Lumigo's Github repository and can be deployed to an AWS account through the Serverless Application Repository console. The code for these applications is also publicly available on Github, allowing contributors to raise feature requests and contribute to the project via pull requests.