April 2022 Summaries

Cloudflare has expanded its global network to 275 cities with the addition of Ahmedabad, India; Chandigarh, India; Jeddah, Saudi Arabia; and Yogyakarta, Indonesia. The company is focusing on expanding in India due to its large digital consumer base. By adding new locations, Cloudflare improves internet speed and reliability for users by bringing content closer to them. In the case of Jeddah, Saudi Arabia, performance improved by 26% after the addition of a new location. The company is continuously working on expanding interconnection capacity and establishing new connections with networks in India.

In April 2022, Cloudflare detected and mitigated a massive HTTPS DDoS attack with 15.3 million request-per-second (rps), one of the largest on record. The attack targeted a crypto launchpad customer operating under the Pro plan. It was launched by a botnet consisting of approximately 6,000 unique bots from 112 countries and over 1,300 networks. Interestingly, most of the attack traffic originated from data centers rather than residential ISPs. Cloudflare's autonomous systems automatically detected and mitigated the attack without human intervention. The company offers unlimited and unmetered DDoS protection for its customers to help build a better internet.

Cloudflare has introduced a new feature that allows users to create multiple segregated virtual private networks (VPNs) over its Zero Trust platform, starting with virtualized connectivity for Cloudflare WARP and Tunnel. This enables companies to securely connect their private networks while improving performance and reducing operational burden. The solution also provides granular control over user access within the network. Additionally, users can now overcome the problem of overlapping networks by creating unique virtual networks that logically segregate IP routes. Virtual networks are only supported through the cloudflared CLI for now but will be integrated into the Zero Trust dashboard in the future.

Marcelo Affonso and Rebecca Weekly recently joined Cloudflare as VP of Infrastructure Operations and VP of Hardware Systems respectively. Both bring extensive experience in their fields, with Affonso having previously worked at Amazon and Weekly having contributed to a variety of hardware projects. They are excited about the opportunity to help build a better Internet by improving security, speed, reliability, and privacy for users worldwide. Cloudflare's global network expansion and commitment to democratizing technologies make it an ideal place for them to contribute their expertise and passion for innovation.

The French presidential run-off election held on April 24 saw a re-election of Emmanuel Macron as President of France. Similar to the first round of elections, French-language news sites outside France were impacted by local rules prohibiting exit polls until after 20:00. Internet traffic was similar on both election days (April 10 and 24), with a slight increase in traffic on April 24. Mobile devices were the primary means of accessing the internet during the election period, particularly at night. The Macron-Le Pen debate on April 20 also saw an increase in mobile requests. French news outlets experienced spikes in traffic after 20:00 local time on both election days, while Belgian and Swiss news sites had major increases in traffic earlier due to the absence of restrictions on exit polls. Official French election-related websites saw significant spikes in traffic during the week leading up to the first round of elections, with the largest spikes occurring on both election days. Major French news websites also experienced their biggest spikes at 20:00 local time when the first major polls were revealed. The Macron-Le Pen debate on April 20 also saw a significant increase in traffic to news outlets and TV station websites.

The new Overview page for Email Routing provides users with full visibility into how emails are being routed on their behalf. It includes a routing status, necessary DNS records configuration, number of custom and destination addresses, advanced statistics about the number of messages received, forwarded or dropped by Email Routing, and rejection reasons. The Activity Log allows for deeper analysis of specific email behaviors, with filtering options available. This update aims to reduce worries about email delivery and improve troubleshooting capabilities.

Cloudflare is committed to building a better internet by powering its network with 100% renewable energy and hosting events like Impact Week. The company has released an Impact Report and emissions inventory report, reduced the environmental impact of web searches via Crawler Hints, and provided customers with visibility about their Scope 3 emissions through Carbon Impact Reports. Green Compute for Workers Cron Triggers is a new feature that ensures triggers run only in facilities powered by renewable energy. This open beta aims to incentivize more data centers to adopt responsible climate policies. Developers can enable Green Compute with just one click, and the company encourages feedback through its Discord community.

On April 18, 2022, US citizens were rushing to submit their federal and state tax returns, leading to a significant spike in traffic to official tax-related websites. The traffic increased by more than 470% on Tax Day compared to the average level on March 31. This surge was even higher than the previous highest growth day in April. Additionally, there were peaks of traffic for state official tax sites and various tax services throughout the month, with a significant increase observed right before Tax Day. Overall, these trends indicate that many people tend to leave their tax returns until the last possible moment.

The Federal Communications Commission (FCC) is working on finalizing rules for broadband Internet access service providers (ISPs) to display a "label" disclosing prices, introductory rates, data allowances, broadband performance, management practices, and more. This initiative aims to provide consumers with sufficient information to make informed choices when purchasing broadband services. The FCC has invited public comments on the existing label design, which dates back to 2016, and seeks input on how the Internet has changed since then. Cloudflare's comments emphasize the importance of transparency in broadband internet performance, including metrics beyond bandwidth such as latency and jitter. The company also recommends disclosing network management techniques and providing more comprehensive privacy information to consumers.

Cloudflare has partnered with Kentik to provide an integrated solution for network security teams, combining Magic Transit, Cloudflare's industry-leading DDoS mitigation solution, and Kentik's network observability. This partnership aims to simplify the procurement process and improve business process simplicity for customers. The integration of these two services will help detect and stop DDoS attacks more effectively, as well as provide powerful global network protection with always-on DDoS protection without impacting network latency and application performance.

The first quarter of 2022 saw a massive spike in application-layer DDoS attacks but a decrease in the total number of network-layer DDoS attacks, according to Cloudflare's latest report. Despite the decrease, volumetric DDoS attacks surged by up to 645% quarter over quarter (QoQ), and they mitigated a new zero-day reflection attack with an amplification factor of 220 billion percent. The most targeted industries in Russia were Online Media and Broadcast Media, while the Telecommunications industry was the most targeted by network-layer DDoS attacks globally.

The first round of the 2022 French presidential election was held on April 10, and a run-off will be held on April 24 between Emmanuel Macron and Marine Le Pen. During the voting period, Internet traffic in France decreased compared to previous Sundays, with the largest difference at 14:00 UTC (16:00 local time), when traffic dropped by as much as 16%. After polling stations closed, there was a significant increase in traffic to news and election-related websites. Mobile usage trends showed that people were using their mobile devices more on Election Day, with mobile usage representing 60% of Internet traffic at around 09:00 local time. Official French election-related websites experienced an increase in traffic throughout the week, with major spikes occurring when the first exit polls were revealed and when 84% of the votes were counted. News outlets also saw a significant increase in traffic on Election Day, particularly around 20:00 local time when the main exit polls were announced. Social media trends did not show any clear impact from the elections.

On December 16, 2021, Cloudflare experienced a slowdown for approximately 30 minutes due to an empty Quicksilver key caused by a missing shell option called "pipefail". The issue started when the Kubernetes cron job failed to populate the key with valid data. This led to the failure of dosd, which provides protection against large attacks and relies on Quicksilver for configuration data. As a result, the Front Line's in-memory cache was flushed, causing a slowdown as requests were stuck waiting for dosd to reply. The issue was resolved by manually re-running the Kubernetes cron job. Lessons learned from this incident include scaling out services to handle high request rates and ensuring code and systems are resilient to failure.

During the first quarter of 2022, various internet disruptions were observed by Cloudflare. These included outages caused by natural events such as a volcanic eruption in Tonga and an earthquake off the coast of Japan. Infrastructure damage also led to several disruptions, including incidents in Gambia, Yemen, Tasmania, Iran, Cuba, and Venezuela. Power outages affected internet connectivity in Kazakhstan, Uzbekistan, Kyrgyzstan, Taiwan, and Cuba. A DDoS attack targeted AS8867 (E-Gov - Tehila Project) in Israel on March 14. In some cases, the underlying technical or physical causes of internet disruptions were not specified. The Russian invasion of Ukraine led to several significant disruptions observed on major Ukrainian network providers in March. Despite these incidents, the internet remains remarkably resilient.

The Russian government has been tightening control over its internet since 2017, including laws requiring ISPs to monitor and block internet activity and establishing a separate DNS system outside of ICANN's oversight. Following the invasion of Ukraine in February 2022, Russia made announcements related to implementing these sovereign internet laws, instructing agencies to switch to Russian DNS servers and reduce reliance on non-Russian providers. However, there has been no evidence that Russia has disconnected itself from the global internet entirely. At the same time, Russians have been adopting tools to maintain access to the global internet, with private and secure internet access apps dominating the Apple App Store in March 2022. Many Russians are also seeking out non-Russian media sources, as evidenced by a significant increase in DNS requests for US, French, and British news websites. Cloudflare has been providing services to protect Ukrainian government, media, financial, and nonprofit websites, including the Ukrainian top level domain (.ua). Additionally, Cloudflare's servers at the edge of the internet in-country have detected and blocked attacks originating there, preventing them from traveling outside Russian borders. While some have called for foreign companies to exit Russia completely, including internet providers, experts argue that it is imperative for the Russian internet to remain as open as possible for the Russian people. This allows them access to information about the war in Ukraine that the Russian government does not want them to see.

The blog post discusses the reasons why CAPTCHAs are still widely used despite their poor user experience and presents an alternative solution called Cloudflare Managed Challenge. It explains that CAPTCHAs remain effective in differentiating humans from bots, but at a significant cost to usability. Many alternatives have been tried, but none have seen widespread adoption like CAPTCHA. The author then introduces the Cloudflare Managed Challenge platform, which tests multiple alternatives and adapts the challenge outcome based on individual visitor behavior. This approach has led to a 91% reduction in CAPTCHAs served by Cloudflare over the past year, with plans to reduce usage further. The post encourages users of the Cloudflare WAF to switch from "Legacy CAPTCHA" to Managed Challenge for better user experience and protection against bots.