July 2013 Summaries

The text discusses how to integrate Datadog with HipChat for efficient collaboration and communication within teams. It provides step-by-step instructions on setting up the integration and explains how to use it to share graphs, events, and alerts in chat rooms. Additionally, it demonstrates how to keep team members updated about pressing issues and receive automatic alerts directly in HipChat. The integration aims to streamline communication and improve productivity within teams by providing a simple way to share information and updates.

Datadog has integrated its services with HipChat, allowing users to share graphs, events, and alerts directly in their chat rooms. To set up the integration, users need to create a notification-only API Auth Token called "Datadog" in HipChat, navigate to Datadog's Integrations page, install the integration, and add the names of the chat rooms where they want to receive updates. Once active, users can mention @hipchat-room_name from the Datadog event stream to share updates without needing to copy or paste. The integration also allows users to automatically keep their team up-to-date on pressing issues by sending alerts directly to HipChat and sharing graphs in real-time.

Elastic Block Storage (EBS) is a storage service offered by Amazon Web Services (AWS) that uses network block storage. It is essential for traditional database systems due to its large storage capacity and reasonable throughput and latency. However, EBS can cause performance issues in EC2-based systems because it relies on the network. Datadog collects and aggregates various AWS metrics and features to identify if performance problems originate from EBS. Storage performance is usually measured in input/output operations per second (IOPS). EBS performance is expressed in IOPS, with standard EBS volumes delivering 100 IOPS on average for blocks up to 16KB in size. Provisioned IOPS can deliver up to 4,000 IOPS per volume if purchased. AWS EBS performance issues occur mainly due to the slower network latency and shared storage and network hardware. To detect EBS performance issues with Datadog, track the CloudWatch metric "VolumeQueueLength." By comparing this metric for each EBS volume attached to a slow application, you can identify if an EBS issue is causing the slowness. To fix or prevent EBS performance issues, consider selecting appropriate storage and instance types, priming your EBS volumes, using Instance Store volumes instead of EBS, purchasing Provisioned IOPS, and replacing degraded EBS volumes when necessary.

Last week's OSCON conference in Portland featured numerous discussions and updates on open-source software. One key topic was Open Compute, a project initially started by Facebook to apply the open-source model to infrastructure components like server racks and data centers. Jay Parikh of Facebook shared how they achieved impressive cost savings through new data center designs and server rack innovations. Another highlight was Go programming language, which gained significant traction in 2013 as a back-end solution. Additionally, Caskey Dickson from Google presented a proven blueprint for ensuring processes stay up and running. Overall, the conference showcased the growing influence of open-source software and its potential to transform various aspects of technology.

The OSCON conference, now in its 15th edition, has attracted a large crowd of open-source users and authors, showcasing the pervasive impact of open-source software. Datadog, which relies heavily on the open-source community, contributed to the event and shared its software with attendees. Keynote speakers discussed innovative approaches to computing, including Open Compute's efforts to optimize data center design and costs by 30%. Google further emphasized the growing popularity of Go programming language, while a session on overcoming the challenges of ensuring processes stay up and running was presented as a solution to a long-standing problem in Unix systems. The conference concluded with a positive note, emphasizing the excitement for next year's event.

AWS Elastic Block Storage (EBS) offers a storage service backed by network block storage, essential for traditional database systems. However, EBS can cause performance issues due to its reliance on the network, which is shared with other customers. To detect these issues, track the CloudWatch metric "VolumeQueueLength" in Datadog, comparing it across EBS volumes attached to slow applications. A sustained increase above 1 or provisioned IOPS/100 indicates maxing out the throughput of that volume. Resolving EBS performance issues involves selecting the right storage and instance types, priming EBS volumes, using Instance Store volumes, purchasing Provisioned IOPS, and replacing degraded volumes if needed.

Amazon Web Services (AWS) offers two types of networked block storage as a service with Elastic Block Storage (EBS): Standard and Provisioned IOPS, which differ primarily by performance. EBS Provisioned IOPS can deliver up to 4,000 IOPS if certain conditions are met, including sufficient requests from the application, read/write operations applying to blocks of 256 KB or less, and no pending EBS snapshots. Monitoring metrics such as VolumeQueueLength and CPU IOWait can help maintain optimal performance for Provisioned IOPS in EBS.

AWS offers two networked block storage options as a service with Elastic Block Storage (EBS): Standard and Provisioned IOPS. The main difference between the two is performance, with EBS Provisioned IOPS being the higher-performance option but requiring specific conditions to be met for optimal performance. To optimize EBS volumes for Provisioned IOPS, it's essential to understand how performance is measured in input/output operations per second (IOPS) and ensure that the volume meets certain conditions, such as maintaining a low average queue length, handling read and write operations on blocks of 256 KB or less, and not exceeding total read/write operations above 128MB/s. Monitoring key EBS performance metrics, such as VolumeQueueLength, is crucial to maintain optimal performance for Provisioned IOPS in EBS.

StatusPage.io has introduced a new Public Metrics feature that enables its customers to create detailed status pages quickly and efficiently. This feature allows any metric from Datadog to be integrated and displayed on StatusPage.io, providing continuous status updates to both current and potential customers. Users can begin by signing up for free trials of both Datadog and StatusPage.io, after which connecting Datadog metrics to Public Metrics is a straightforward process requiring just a few clicks. This enhancement aims to improve transparency and communication regarding service statuses.

The text discusses notable sessions at two major WebOps/DevOps events, Velocity and DevOpsDays. J. Paul Reede's talk "Is Your Team Instrument Rated (or: Deploying 125,000 times a day)" highlighted key lessons from the US national airspace system to build robust operational frameworks in DevOps. Brendan Gregg's presentation at Velocity emphasized performance monitoring and eliminating guesswork through workload characterization, USE methodology, and thread-state analysis. Adam Lazur from Facebook discussed their new load balancing architecture for handling over a billion users globally, focusing on the role of Cartographer in dynamically configuring DNS servers.

The Datadog team attended notable sessions at Velocity and DevOpsDays, two prominent WebOps/DevOps events in Silicon Valley. At Velocity, Brendan Gregg presented "Stop the Guessing: Performance Methodologies for Production Systems," emphasizing the importance of clear communication, shared expectations, and a three-pronged approach to performance monitoring: workload characterization, USE (Utilization, Saturation, Errors), and thread-state analysis. Adam Lazur from Facebook discussed their billion-user load balancer architecture, which utilizes dynamic DNS configuration with Cartographer, a continuous provisioning engine. Additionally, J. Paul Reede shared lessons learned from the US national airspace system, highlighting the need for clear communication and avoiding blind faith in automation to address complexity. The Datadog team is looking forward to presenting at Velocity later this year in New York and continuing to sponsor DevOpsDays worldwide.