August 2013 Summaries

On August 27, 2013, the New York Times website was hacked by the Syrian Electronic Army. The attack compromised the domain registrar MelbourneIT, allowing the hackers to update the name servers for NYTimes.com without authorization and redirect traffic to a malware-infected site. Several other high-profile websites managed by MelbourneIT were also affected, including Twitter and the Huffington Post. The incident highlights the importance of DNS security and the potential damage that can be caused by DNS hijacking. To protect against such attacks, domain owners should consider implementing registry locks to prevent unauthorized changes to their domains.

In 2013, CloudFlare faced challenges with their DNS infrastructure using PowerDNS and decided to create a custom authoritative name server called RRDNS. The new system allowed for easy addition of application logic, leading to the creation of fun Easter Eggs such as querying for job listings via DNS request. However, concerns were raised about potential misuse of these features in DNS reflection attacks. To address this, Ian implemented a safeguard that forced RRDNS to respond with a 0-byte UDP response and the DNS message truncated flag when receiving an Easter Egg-generating query, causing clients to retry via TCP and preventing source IP spoofing. This highlights how extensibility in systems like RRDNS can be used for both fun features and enhanced security measures.

The blog post discusses the challenges of memory management in long-running network services written in garbage-collected languages like Go. It presents a simple program that creates a lot of garbage and uses the runtime.ReadMemStats function to get information about the size of the heap. The author explains how garbage collection runs frequently in a Go program, causing HeapAlloc and HeapIdle to change over time. A separate scavenger releases memory unused for more than 5 minutes, but this doesn't happen often. The post then introduces a manual memory management technique using channels to keep a separate pool of buffers that are no longer used. This recycling mechanism can be shared across goroutines and dynamically cope with bursts of demand for buffers. The author also presents an example of a recycler in Go, which keeps a linked list of returned buffers and periodically throws away old ones unlikely to be reused. These techniques can significantly reduce the amount of memory a program needs by recycling memory that the programmer knows is likely to be reused without asking the garbage collector to do the work. They can be used for more than just byte slices, as any arbitrary Go type (user-defined or not) could be recycled in a similar manner.

The text discusses how Cloudflare has developed a new Web Application Firewall (WAF) using Lua, a lightweight scripting language, and nginx, an open-source web server. The WAF is designed to protect websites from common vulnerabilities such as SQL injection, cross-site scripting, and command injection attacks. It can read existing mod_security configurations and supports its own simplified rule language. The Lua code in the waf module determines whether a request should be blocked or passed to the origin for processing. The WAF is highly optimized using techniques such as clause reordering, regular expression optimization, operator replacement, global optimizations, and Lua optimizations. It has been tested under a test harness with line-level timing information and in Cloudflare's network with detailed systemtap-based instrumentation. The resulting code is hard to read because it's essentially the WAF's assembly language and has been automatically generated. The overall goal was to get the median WAF block/allow decision made in less than 1 millisecond when running in the real world, which has been achieved.

Kenneth R. Carter, CloudFlare's in-house counsel, has announced a new privacy policy effective from August 20, 2013. The changes made to the policy include more clearly explaining how specific information is treated, removing ambiguity, and ensuring compliance with global privacy standards. Additionally, TRUSTe has been appointed for dispute resolution relating to CloudFlare's U.S.-EU Safe Harbor framework and U.S.-Swiss Safe Harbor. A dedicated Trust and Safety email contact has also been created for handling privacy-related concerns. The policy is now available in a public GitHub repo, where changes will be tracked over time. CloudFlare maintains its commitment to keeping users' personal information private and secure.

Cloudflare has updated its Web Application Firewall (WAF) to include a traditional rules-based system alongside its existing heuristics-based approach. The new rules-based WAF is designed to be fully user configurable, accepts the ModSecurity rule configuration language, and adds less than 1ms of latency to requests when processing the full OWASP ruleset or its equivalent. It can update worldwide within less than 30 seconds of a user configuration change and scales to handle Cloudflare's level of traffic. The new WAF is included by default with every Pro and better plan.

Cloudflare's CEO, Matthew Prince, has addressed concerns about providing services to websites hosting controversial content. He emphasized that the company does not confirm any user to the media without their permission and explained that they do not monitor the content flowing through their network nor make determinations on what is politically appropriate. Prince stated that Cloudflare's mission is to build a better web, and they will continue to abide by the law, serve all customers, and hold consistently to a belief that their proper role is not that of Internet censor.

CloudFlare's name servers are named after common 2- to 4-letter names such as Bob and Lola. The company initially used standard format domain names like dns1.cloudflare.com, but users tried to be clever by adding more domains, which failed verification. To resolve this issue, they created unique combinations of name server names using a list of one hundred common names. They also commissioned an artist to draw representations of the 100 name servers as ninjas and later added "Woz" in honor of Apple co-founder Steve Wozniak. The name server names are not related, but they reference potentially hundreds of servers running in all of CloudFlare's global data centers.