November 2017 Summaries

Amazon's announcement of its new container services, Fargate and EKS (Elastic Kubernetes Service), marks its entry into the managed Kubernetes service space. EKS is a managed service that uses upstream Kubernetes and is designed to run across multiple Availability Zones, utilizing AWS's IAM for RBAC and PrivateLink for secure access. Fargate, on the other hand, allows users to run containers orchestrated by ECS or Kubernetes without managing the underlying EC2 instances, offering flexibility by charging per computing second used. While ECS currently integrates more seamlessly with other AWS services, EKS offers the advantage of utilizing the same scheduler both within AWS and elsewhere. Fargate's approach addresses the complexity of managing container orchestration layers and provides a solution to the bin packaging problem, though it comes with its own cost considerations. However, questions remain about EKS's customization capabilities and its integration with AWS services, as well as the potential costs associated with inter-zone traffic and other resources.

Kubernetes security is a critical concern for companies transitioning to containerized environments, and implementing robust run-time security measures is essential to protect against potential threats and vulnerabilities that traditional static scanning might miss. This article explores best practices for enhancing the security of core Kubernetes components like kubelet, apiserver, scheduler, and others, which are essential for the functioning of Kubernetes clusters. It emphasizes the use of Sysdig Secure, a container-native security and forensics product, to create and test run-time security policies that provide an additional layer of protection by monitoring and restricting unauthorized processes, connections, and container images. The approach involves whitelisting allowed processes and behaviors to detect and respond to suspicious activities, ensuring that only trusted containers and actions are permitted within the kube-system namespace. The article also highlights the importance of maintaining strict control over Kubernetes secrets, network connections, and user management operations to mitigate the risk of unauthorized access and modifications, advocating for a proactive and comprehensive security strategy in Kubernetes environments.

Sysdig Secure, launched in November 2017, is a comprehensive security solution designed for Docker and microservices, integrating seamlessly with container orchestration technologies like Kubernetes and Docker Swarm. The platform offers a hands-on approach to implementing run-time security policies, exemplified through the configuration of security rules in a simple use case involving Docker's example-voting-app. Users are guided to set up Kubernetes applications, create and adapt security policies, and test their effectiveness by simulating attacks. Sysdig Secure provides an infrastructure overview, allowing users to monitor security events and configure policies with varying severity levels. It includes default security rules, such as policies against unexpected privileged container launches and unauthorized process executions, which can be customized for specific scenarios. The tool also supports creating custom security policies that leverage the simplicity and specificity of container and microservices architectures, offering features like capture files for in-depth analysis. Overall, Sysdig Secure emphasizes the importance of strict and specific security policies in container environments, promoting a proactive security stance through detailed monitoring and rule customization.

Sysdig's table panels are a powerful tool for managing and troubleshooting Kubernetes environments, offering a flexible way to view and organize infrastructure data by combining physical and logical entities in a single format. This functionality extends the capabilities of the Explore page in Sysdig Monitor, allowing users to dynamically decompose their infrastructure from clusters to services and containers, and now integrates into dashboards for enhanced visibility into hosts, services, and containers. By segmenting, scoping, and selecting metrics, users can customize their view to track processes like 'navguide' across both physical hosts and Kubernetes deployments. This enables efficient monitoring of resource usage, such as JVM heap usage per deployment, and helps identify deployment status codes and errors, facilitating the troubleshooting of network issues. Ultimately, Sysdig's table panels provide a tailored approach to data visualization, enabling users to address specific operational challenges effectively.

The upcoming enforcement of the European General Data Protection Regulation (GDPR) requires organizations dealing with personal data from the EU to make significant adjustments in data handling, breach notification, and compliance. The regulation applies to any company, including those outside the EU, that processes personal data originating there, and introduces key roles such as the Data Processor and Data Controller to delineate responsibility. GDPR enforces strict penalties for non-compliance, with fines reaching up to €20 million or 4% of global annual revenue, and mandates breach notifications within 72 hours. Tools like Sysdig Secure offer solutions for breach prevention, response, and forensics, aiding organizations in achieving compliance. Comprehensive resources, including gdpr-info.eu and various compliance checklists, are available to assist with understanding and implementing GDPR requirements, while future posts will explore in-depth strategies for using Sysdig in GDPR-specific scenarios.

The blog post discusses the process of creating a custom Kubernetes scheduler using monitoring metrics from tools like Sysdig, which automatically tags metrics with Kubernetes metadata for advanced scheduling capabilities. The article highlights the limitations of the default Kubernetes scheduler and introduces the flexibility of custom schedulers available since Kubernetes 1.6. It provides an example of using Python to write a scheduler that places web server pods on nodes with optimal HTTP response times, leveraging metrics without requiring code instrumentation. The post explains the steps to configure and implement this scheduler, including using Sysdig API metrics for decision-making, and suggests deploying the scheduler as a containerized pod for better orchestration. Additionally, it mentions a more advanced Golang version of the scheduler presented at KubeCon EU 2018, which includes features like metrics caching, failover recovery, and asynchronous scheduling, while encouraging readers to explore further use cases and enhancements for production-level schedulers.