November 2023 Summaries

Amazon CodeWhisperer now supports Terraform, providing real-time single-line or full-function code suggestions that reduce development effort and allow practitioners to focus on end-to-end workflows. The tool is trained on large language models and collaborates with HashiCorp to provide high-quality suggestions for Terraform modules and configurations written in HashiCorp Configuration Language (HCL). CodeWhisperer can be used with the latest AWS Toolkit plugin, providing code suggestions as users write new Terraform configuration files, and can also detect when errors occur, validating Terraform code within the Visual Studio Code editor. The tool is designed to enhance developer experience by reducing context-switching between editors and CLI, and offers integrations such as enhanced editor validation and a new view in the HashiCorp Terraform extension for Visual Studio Code.

HashiCorp has been awarded Amazon Web Services' Global Collaboration Partner of the Year for its significant contributions to AWS infrastructure and security collaborations. HashiCorp accelerates organizations' transition to the cloud by providing lifecycle management and automation for application infrastructure and security, leveraging identity-based controls to protect and connect users and machines to secrets and critical data. The company's tools, such as Terraform, offer a unified solution to build on AWS, replacing fragmented workflows with a standardized approach and reducing costs through enhanced security and developer productivity. HashiCorp has also integrated its solutions into AWS, enabling thousands of developers and enterprises to move faster, modernize their operations, and get the most out of AWS. Recent developments include the integration of Terraform Cloud with AWS Service Catalog, allowing customers to use a single tool to organize, govern, and distribute their configurations at scale. Additionally, HashiCorp's HCP Vault Secrets provides a centralized secrets management solution to address secret sprawl, enabling organizations to establish a single control plane for managing secrets across cloud environments and developer workflow tools.

The HashiCorp Terraform extension for Visual Studio Code has been updated with a new integration with Terraform Cloud, providing a read-only view of workspaces and runs to reduce context-switching. The update also includes enhancements such as the Module and Provider Explorer, which can be used regardless of whether or not you have Terraform Cloud, allowing users to access module information and provider details in the editor. Additionally, fixes have been released for performance issues with the Terraform language server, improving overall performance. Future plans include adding features to the Terraform Cloud integration, completing the Terraform language server's understanding of all Terraform language features, and moving towards Day 0 support for new language features as they come out.

HashiCorp's Terraform provider for AWS has launched support for Amazon S3 Express One Zone storage class, a high-performance solution for ultra-low latency and large storage volumes. This new bucket type is designed to deliver single-digit millisecond response times for frequently accessed datasets, making it suitable for compute-intensive big data workloads such as autonomous vehicle data, financial risk modeling, real-time online advertising, and machine-learning training and inference. The Terraform provider supports S3 Express through the existing S3 API, allowing for seamless integration with other AWS services. To set up S3 Express, users can use the new `aws_s3_directory_bucket` resource and create a bucket with a hierarchical namespace, low-latency zonal storage class, and fast-authorization API. The provider also offers improved support for managing resources at scale through Terraform stacks.

Vault benchmark is an open source tool designed to test the performance of HashiCorp Vault auth methods and secrets engines through load testing, giving organizations confidence in their secrets management lifecycle. The tool uses the HTTP load testing utility Vegeta and can be run against non-production Vault clusters isolated from production systems. By using Vault benchmark, engineering teams can discover aspects of their architecture that are performing well or need improvement, as well as opportunities for optimization. The benefits include understanding how different configurations impact cluster performance and identifying potential issues before they affect the end-user experience. To use Vault benchmark, users must set up infrastructure dependencies and run the binary with a configuration file, which should be used to configure resources required for testing. The tool is intended to be used in non-production environments and can help organizations improve their overall performance by following production hardening guidelines and reference architecture.

HCP Consul Central's new observability features were recently used by a customer to help troubleshoot an issue with assistance from HashiCorp engineers, illustrating the importance of observability in modern software systems. Observability enables operators to understand the internal state of a system based on data it generates, allowing them to notice issues faster, pinpoint the root cause, and fix it. HCP Consul Central's observability features provide actionable insights into the health of Consul infrastructure, including metrics, traces, and logs, helping operators troubleshoot issues more efficiently. The customer's experience with a non-responsive cluster demonstrates how these features can cut troubleshooting time significantly and increase operator confidence. HashiCorp is committed to enhancing its observability features to provide a holistic view of infrastructure, shedding light into interservice communication relationships, and reducing friction for Consul operators.

Stacks simplify infrastructure provisioning and management at scale by automating and optimizing the coordination, deployment, and lifecycle management of interdependent Terraform configurations, reducing time and overhead. The new feature aims to be a natural next step in extending infrastructure as code to a higher layer using existing Terraform shared modules, providing a more composable and manageable approach to managing infrastructure at scale.

The AWS Cloud Control provider is a joint project between HashiCorp and AWS designed to bring new services to HashiCorp's Terraform faster by automatically generating resources and data source schema, addressing coverage gaps in the main Terraform AWS provider. The provider supports over 750 AWS resources and data sources, with more support being added as AWS service teams adopt the Cloud Control API standard. It can be used alongside the existing Terraform AWS provider and is expected to be generally available in 2024. The provider integrates with AWS Cloud Control API, making it easier for developers to manage their cloud infrastructure in a consistent manner and leverage the latest AWS capabilities faster. With the introduction of sample configurations, enhanced schema-level documentation, and other user experience enhancements, customers can now easily use resources and reduce errors when provisioning infrastructure.

The latest episode of the HashiCast: Navigating the Cloud for C-Suites podcast explores strategies to build a team with the skills needed to drive cloud initiatives, highlighting the importance of understanding individual strengths and not just technical skills. Many executives expect perfection from new hires, but this can elongate the hiring process and come at a significant cost, while also impacting customer satisfaction. Best practices involve hiring for diverse teams with varied backgrounds and skill sets, and leveraging technology responsibly to attract, motivate, and retain top talent in today's dynamic market. The post-pandemic era has seen huge shifts in employee expectations, emphasizing purpose, feeling valued, flexibility, and autonomy, prompting organizations to reassess their approach to talent acquisition and retention. AI is playing a pivotal role in this transformation, offering enhanced communication, streamlined processes, and access to hidden talent pools, but also raising questions about bias and ethical usage.

HashiCorp will be a prominent presence at AWS re:Invent this year, with a full program of events and demos showcasing its products such as Terraform, Vault, Consul, and Nomad, which support various cloud environments. The company has achieved a milestone of 2 billion downloads for the Terraform AWS provider, highlighting the growing need for standardized infrastructure as code solutions. HashiCorp will also be hosting events to celebrate this milestone, including a Terraform Thursday breakfast and an executive forum, where attendees can learn from industry leaders and network with experts. Additionally, HashiCorp Co-Founder and CTO Armon Dadgar will share his insights on the future of the cloud at the upcoming HashiTalks event in February 2024.

HashiCorp has introduced 10 new integrations for its Vault ecosystem, expanding security use cases for customers. These integrations enhance identity-based security, secrets lifecycle management, and cloud security automation. HCP Vault Secrets provides a centralized platform for managing secrets, while the new integrations with GitHub, Vercel, Intel, Pure Storage, BuddyWorks, Couchbase Capella, JetBrains, OpsMx, Red Hat, and AWS provide seamless integration with various developer tools and cloud providers. The new integrations also address issues such as secret sprawl, enabling companies to securely manage secrets across multiple platforms. With these updates, HashiCorp aims to simplify cloud security automation and improve overall security and compliance for enterprises.

HashiCorp's Vault is a centralized secrets management system that provides encryption services gated by authentication and authorization methods to ensure secure, auditable, and restricted access to secrets. Organizations are advised to adopt a centralized system where all their secrets can be stored, organized, managed, and protected. The first step in resolving secret sprawl is to use Vault's secrets sync feature, which allows users to synchronize secrets from Vault to external secrets managers, ensuring that all secrets are up-to-date and reducing the risk of secret sprawl. Additionally, organizations should utilize secret scanning tools like HCP Vault Radar to detect and identify unmanaged secrets, and limit access to usable secrets by encrypting them. HashiCorp's Vault offers various platform options, including HCP Cloud Secrets, HCP Vault, Vault Enterprise, and HCP Vault Radar, each addressing different aspects of secrets management. Resolving secrets sprawl requires a comprehensive approach covering people, processes, and technology, and integrating Vault into a broader compliance strategy is crucial for achieving the best results.

The HashiCorp ecosystem continues to expand with 18 new Terraform integration partners, providing additional capabilities for users deploying and managing cloud infrastructure. These partnerships enhance the Terraform ecosystem, offering integrations with DNSimple, Airbyte, Cisco Defense Orchestrator, ClickHouse, Cloudera, Dell Technologies, Fortinet, Juniper Networks, Kestra Technologies, Materialize, Orca Security, Palo Alto Networks Prisma Cloud, Panther Lab's SIEM, Prefect, Sym, Timeplus, Vantage, and Versa Networks Director. These integrations enable users to automate and secure their cloud infrastructure management with Terraform Cloud, Enterprise, and Community edition. The new partners provide prescriptive Sentinel policies for domains and DNS in DNSimple, automates Juniper's intent-based networking software, and enables the management of Orca resources with Terraform. All integrations are available for review in the HashiCorp Terraform Registry, and users can try the free tier of Terraform Cloud to simplify their workflows and management.

HashiTalks is a 24-hour virtual knowledge sharing event that returns on February 15, 2024, where experts from the community will share their experiences with HashiCorp tools. The call for proposals is now open and will close on November 30, 2023, and is intended to be an inclusive platform where participants can share unique use cases, technical presentations, and demos featuring HashiCorp tools. The event aims to expand accessibility by adding new regions and languages, as well as bringing back a previous regional edition. It offers extensive support for first-time speakers and provides opportunities for skill improvement through speaker-enablement sessions. Talks will be recorded and posted on YouTube for self-paced viewing, and attendees can engage with speakers in real-time using the chat feature. The event also encourages participation through the HashiCorp User Group program, where users can join local chapters and review talks when they are posted to YouTube.

Boundary` is a cloud-hosted identity-aware proxy that simplifies and secures least-privileged access to cloud infrastructure. It's an alternative to traditional Kubernetes Ingress Controllers, enabling secure external access to application content and components without compromising the security of either the platform or the application. By using `Boundary`, developers can create a Zero Trust architecture for their applications, ensuring that only authorized users have access to sensitive resources. The approach uses a self-managed deployment of Red Hat's OpenShift Container Platform as the Kubernetes platform, and HashiCorp Boundary to enable secure, controlled ingress to workloads hosted on Kubernetes. The `Boundary` worker is configured using a container image and a configuration file in the HashiCorp Configuration Language (HCL), which is presented to the running container in its Pod as a Kubernetes ConfigMap. This allows for easy deployment and management of `Boundary` workers within a Kubernetes cluster, improving the security posture of application content and components hosted within Kubernetes.

Terraform Enterprise has recently introduced several features aimed at enhancing manageability and efficiency, including flexible deployment options, ephemeral workspaces, and run data retention. The new flexible deployment options allow customers to choose the platform that fits their organizational standards and preferences, while also reducing operational costs and complexity. Ephemeral workspaces enable administrators to set timeouts that automatically destroy temporary resources, eliminating manual clean-up and simplifying workspace management and testing. Additionally, Terraform Enterprise now supports multiple configurations for dynamic provider credentials and provides a configurable data retention policy for Terraform run data, which reduces storage costs and mitigates compliance risks. These features aim to improve the overall efficiency and security of Terraform Enterprise environments.

HashiCorp has released Consul 1.17, a significant update to its service networking solution that enhances unified workflow management, reliability, scale, and security for service networking. The new release introduces locality-aware routing for low-latency connections and JWT-based authentication and authorization support for API Gateway. Consul 1.17 also bolsters reliability with sameness groups and rate limiting for services, simplifying operations and increasing service availability for multi-cluster or multi-region deployments. Additionally, the update includes a new set of APIs for interacting with the catalog, authenticating traffic using workload identities, and managing Consul service mesh, which are now exposed under the new v2 API paths. The release aims to enable a consistent, enterprise-ready control plane to discover and securely connect any application, empowering organizations to build resilient and efficient applications in distributed environments.

Terraform Cloud has introduced dynamic provider credentials for Kubernetes via Amazon Elastic Kubernetes Service (EKS) and Google Kubernetes Engine (GKE), enabling automated authentication with just-in-time credentials, reducing the risk of exposure from storing long-lived static credentials and eliminating operational burdens of manual secret rotation. This enhancement supports industry-standard OpenID Connect workload identity authentication flow, simplifying Terraform Cloud setup and ensuring secure authentication across cloud environments. With this update, users can authenticate Kubernetes clusters on AWS and Google Cloud without relying on long-lived static credentials, mitigating security risks and streamlining their workflows.

The Terraform Cloud Operator v2 for Kubernetes is generally available, enhancing resource management and scaling of Terraform Cloud agents. The new version focuses on three main areas: multiple custom resources to remove scaling limitations, a --namespace option for fine-grained resource management, and faster synchronization between custom resources and Terraform Cloud. The Operator now features separate controllers for different Terraform Cloud resources, such as AgentPool, Workspace, and Module, allowing users to manage workspaces and agent pools more efficiently. This release streamlines infrastructure management, providing increased agility, reduced risk, and cost efficiency for platform teams, while offering a Kubernetes-native experience for their users.

In March 2023, Terraform Cloud announced the general availability of dynamic provider credentials, a native support for just-in-time authentication with AWS, Azure, Google Cloud, and HashiCorp Vault providers. This enhancement reduces the risk of exposure from storing long-lived static credentials and eliminates manual secret rotation burdens. Dynamic provider credentials are based on industry-standard OpenID Connect (OIDC) workload identity authentication flow, allowing Terraform to authenticate Kubernetes clusters using EKS and GKE without long-lived static credentials. With this feature, Terraform Cloud simplifies setup and ensures secure authentication across cloud environments, mitigating security risks associated with storing long-lived credentials.

Observability integrations are crucial for organizations to build and maintain a strong "security-first" posture by streamlining audit logs and performance metrics into automated threat detection and incident response workflows, allowing for identification of suspicious user activity, capacity planning, and cost optimization. HCP Vault now offers three new top-tier observability SaaS providers - Amazon Cloudwatch, Elasticsearch, and New Relic - as well as a generic HTTP endpoint for flexible metrics streaming, providing users with both security and operational insights to inform management decisions.

The integration of Consul with Apigee allows for service-to-service communication across environments, enabling authorization based on business hours and conditions such as allow/deny based on IP addresses or API keys. This combination provides a scalable solution for managing APIs, allowing operators to automate networking and offload service-to-service authorization to external tools like Apigee. Consul's Envoy extension capabilities with the Apigee adapter enable granular control over security, rate limiting, quotas, analytics, and more for all services, while also automating and offloading mTLS authentication and encryption between network services. The solution proposed here involves installing the Apigee remote Envoy service onto the Consul service mesh to enforce authorization policies provided by the central Apigee authority in Google Cloud.

Nomad 1.7 introduces improved workload identity, improved Vault and Consul integrations, NUMA support, Nomad actions, and more. This version includes a new workload identity token that can be used by third parties to authenticate the identity of Nomad tasks, allowing for dynamic credentials to be minted for Nomad tasks. The integration with Vault has been revamped, making it easier for users to set up and manage Vault without having to manually manage tokens. Consul integrations have also improved, reducing the overhead associated with static token management. Additionally, NUMA support is now available, allowing users to dedicate whole CPU cores to specific tasks. Nomad actions allow job writers to define a named command that can be executed within the context of an allocation, providing a safer and more user-friendly experience for repeated tasks. The UI has also been updated with new sections related to access control, roles, and tokens. Furthermore, a built-in mechanism for distributed locks using Nomad variables is now available, allowing applications on Nomad to perform complex locking behavior and leadership coordination using Nomad as a source of truth.

HashiCorp's Consul was a key focus at HashiConf 2023, with sessions covering practical advice on leveraging Consul for cloud migration and hybrid infrastructure expansion, delivering robust microservice review app functionality through service mesh, and enhancing observability features to monitor and debug distributed services. The event highlighted the growing trend of multi-cloud workloads, emphasizing the importance of networking tools like Consul in simplifying application network management without explicit infrastructure handling. Key announcements included upgrades to simplified workflow management, reliability, scale, and security for service networking, as well as new security lifecycle management capabilities.