April 2020 Summaries

In April 2020, Cloudflare highlighted its growing support and training for managed service providers (MSPs) globally. The company's partner program aims to help MSPs efficiently engage with Cloudflare and join in the mission of building a better internet. The program offers hands-on education, partner support and success resources, access to account managers, and partner enablement engineers. Partners have found success throughout Cloudflare's lifecycle by helping customers understand how to transform their network in their move to hybrid and multi-cloud solutions, develop serverless applications, or manage the Cloudflare platform. One example of a successful partnership is with Zilker Technology, which engaged Cloudflare when one of its eCommerce clients faced carding attacks and other malicious activity on their sites. By partnering with Cloudflare, Zilker improved site performance and user experience while gaining enhanced security and protection from Cloudflare WAF, advanced DDOS protection, and rate-limiting.

On April 30th at 12:00 PM, Connor Peshek introduced the fifteenth edition of "The Serverlist." This publication provides updates on serverless technology, offers developer tutorials, fosters discussions among serverless developers, and highlights upcoming meetups and conferences. To receive future editions directly in your email, sign up using the provided link while ensuring your privacy is maintained.

Gartner's recent report on DDoS cloud scrubbing centers highlights the importance of DDoS protection for businesses and organizations. The nature of DDoS attacks has evolved with more frequent smaller attacks, but larger-sized attacks are still prevalent. Cloudflare received the greatest number of 'High' ratings compared to other DDoS vendors in Gartner's assessment criteria. Cloudflare offers a comprehensive DDoS protection solution with features such as massive network capacity, globally distributed architecture, fast time to mitigation, integrated security, and unlimited mitigation.

Cloudflare is doubling its summer internship program this year to support students who may have lost their internships due to COVID-19. The company believes that interns bring new energy and perspectives, contribute to major projects, and help in talent assessment. To ensure a great remote internship experience, companies should provide committed managers or mentors, define projects and goals, create an open environment for networking, and offer visibility and exposure through presentations and meetings with executives. Cloudflare's virtual internship program includes over-communication, proper onboarding, establishing a work cadence, and organizing social activities to foster connections among interns.

Global companies face challenges doing business in and out of China due to unique rules, regulations, and norms, as well as differences in logistical and technical network infrastructure. With global internet traffic up 30% during the pandemic, these hurdles are increasing for businesses worldwide. To address this issue, Cloudflare has partnered with JD Cloud & AI, a cloud and intelligent technology business unit of Chinese Internet giant JD.com. Through this partnership, they will add 150 data centers in mainland China, enabling JD to provide a Cloudflare-powered service to its customers. This collaboration aims to create a one-stop solution for companies both inside and outside of China to go truly global.

Starting April 27, 2020, users can securely manage their Kubernetes cluster with the kubectl command-line tool using Cloudflare Access and Argo Tunnel. This feature addresses an edge case that prevented some of Cloudflare's customers from disabling VPNs. By deploying this workflow, users can add SSO requirements and a zero-trust model to their Kubernetes management in under 30 minutes. The integration of kubectl with Cloudflare Access enhances security by allowing granular checks on every request without slowing down the experience. This feature is available for all plans and feedback from users will be used to improve its ease of use.

In this tutorial, we learn how to configure Cloudflare Logs to push security events directly to a logging platform using Terraform. The steps involve setting up Sumo Logic and Cloudflare with the help of Terraform, creating a challenge token, and testing the setup by sending a malicious request. This integration allows users to ingest all Firewall Events into their SIEM or logging platform without having to filter through extraneous data.

Internet traffic has seen significant growth due to the COVID-19 pandemic, with people working from home, practicing social distancing, and staying indoors. Countries such as Portugal, Spain, and the UK have experienced a 50% increase in internet usage, while Italy's usage increased by around 40%. Despite this surge in traffic, Cloudflare has not observed any impact on its network's performance, reliability, or security. The company measures TCP RTT (round trip time) to assess the speed and congestion of networks between their servers and end-users. While some countries like Italy have experienced increased network strain, overall, the internet has shown resilience in handling these sudden changes in traffic patterns.

In this blog post, Jason Farber discusses how to deploy Cloudflare Gateway from home for protection against malware, phishing, ransomware and other security threats. He explains the concept of DNS filtering and its implementation using various methods such as DNS over HTTPS (DoH) with a Raspberry Pi, Pi-hole and DNSCrypt. Farber also provides detailed instructions on how to enable DoH in different browsers like Chrome, Firefox, Opera, etc., and configure Gateway for an entire home or business network using a Raspberry Pi as the DNS and DHCP server. The post concludes with a discussion on IPv4 check and lookup based on source IPv4 address and lookup based on IPv6 destination address as alternative methods to match requests in Gateway.

The Internet has played a crucial role in responding to the COVID-19 crisis, enabling communication between researchers and sharing vital data about the spread of the virus. Cloudflare is offering free access to its Cloudflare Workers compute platform through Project Galileo for critical web projects that help fight against the pandemic. This initiative supports developers who are creating tools to assist the public during this crisis. Some examples include API-COVID19-In, a project by Amod Malviya tracking COVID-19 cases in India; MakeFaceMasks, a Belgian grassroots movement providing a DIY manual for sewing masks approved by the government; Mask A Hero NY, a volunteer-run site matching medical professionals with donors of Personal Protective Equipment (PPE); and CovidTracking API, which collects and publishes testing data available for US states and territories. Cloudflare Workers offers several benefits such as ease of setup, scalability, and no need to maintain or scale infrastructure. Developers can get started with resources like Workers Sites, tutorials, template gallery, HTMLRewriter API, and the Built with Workers gallery for inspiration on what they can build using Workers.

Border Gateway Protocol (BGP) has been a vital part of the internet since the 1980s, but its security features have not kept up with the evolving threats. Resource Public Key Infrastructure (RPKI), a security framework for routing, is considered mature enough for widespread use and can help prevent route leaks and hijacks. Major network operators need to deploy RPKI to make the internet safer. Cloudflare has released isBGPSafeYet.com, a website that tracks RPKI deployment and filtering of invalid routes by major networks. The source code for this website is available on GitHub, and users can test their ISP's implementation of RPKI using two bad prefixes announced from Cloudflare's data centers and Internet Exchange Points (IXPs). By encouraging more networks to deploy RPKI, the internet can become safer and reduce the impact of route leaks.

Cloudflare has introduced phone authentication for Enterprise customers to enhance their support experience over the phone without compromising account security. Customers can now verify their identity during a call by providing a token generated from the Cloudflare dashboard or via a 2FA app like Google Authenticator. This feature is available to eligible Enterprise customers and can be configured through the Cloudflare dashboard or an authenticator app. The system generates single-use tokens using RFC 6238 compliant time-based one-time passwords, ensuring secure authentication during phone calls. Future improvements include giving users the ability to request a callback from a support agent within the Cloudflare dashboard.

On April 16, 2020, between 1531 UTC and 1952 UTC, Cloudflare experienced an outage affecting its Dashboard and API due to the accidental disconnection of multiple redundant fiber connections from one of its two core data centers. The incident was not caused by a DDoS attack or any other external factors. During the outage, customers were unable to log into the dashboard, use the API, make configuration changes, purge cache, run automated load balancing health checks, create or maintain Argo Tunnel connections, create or update Cloudflare Workers, transfer domains to Cloudflare Registrar, access logs and analytics, encode videos on Cloudflare Stream, or log information from edge services. However, the Cloudflare network itself continued to operate normally, and all security services remained functional. The company worked simultaneously to restore connectivity and cut over to its disaster recovery core data center. Full redundant connectivity was restored at 2031 UTC. Moving forward, Cloudflare plans to address the risk of similar problems by improving design, documentation, and processes related to hardware decommissioning and cable management in their data centers.

Recently, COBOL has gained attention due to its use in unemployment claim systems during the COVID-19 pandemic. Despite being 60 years old, COBOL is still heavily used in information management systems and financial transactions. In response to this increased demand, IBM offered free online COBOL training. Cloudflare's serverless platform Workers now supports writing code in COBOL, allowing users to take advantage of its long-standing presence in the industry. The platform also provides a simple "Hello, World!" program written in COBOL as an example.

As the COVID-19 pandemic continues to affect countries worldwide, internet usage has significantly increased as people rely on it for information and communication. Cloudflare, a company providing internet services, is offering free tools and resources to businesses, organizations, and government agencies to help them manage remote work and provide accurate information to the public during this crisis. They have also seen an increase in applications for their Project Galileo initiative, which provides free services to vulnerable organizations on the internet. Additionally, Cloudflare has introduced a free package of services for state and local governments worldwide until September 1, 2020, to help them secure their web infrastructure and internal teams during this pandemic.

Cloudflare has announced support for HTTP/3, the successor to HTTP/2, during its birthday week last year. Over 113,000 zones have activated HTTP/3 and can be accessed using experimental browsers. The company launched support in partnership with Google, who also added experimental support in Chrome. Firefox, Opera, Microsoft Edge, and Safari have since followed suit. HTTP/3 is expected to offer increased performance, specifically around fetching multiple objects simultaneously. It also offers 0-RTT support, which means subsequent connections can start up much faster by eliminating the TLS acknowledgement from the server when setting up the connection. Real-world tests have shown that HTTP/3 performs better than HTTP/2 in some cases and at worst similar to it. As the standard finalizes, Cloudflare is looking forward to seeing browsers add support for HTTP/3 in mainstream versions. The company continues to support the latest drafts while exploring more ways to leverage HTTP/3 to improve performance.

Cloudflare has extended its Spectrum service for DDoS protection and traffic acceleration to cover SSH, RDP, and Minecraft protocols. This move allows Pro and Business plan customers to protect their server management services in addition to websites. By leveraging the Cloudflare network, users can also experience increased reliability and performance with lower latency. The Spectrum service is now available for free on all paid plans, with a generous free data allowance.

Three interns from Cloudflare's 2019 program developed a new feature called passive origin monitoring, which notifies customers when their origin servers are unreachable. The project involved analyzing data patterns and designing actionable notifications for various types of users. The team faced challenges such as ensuring high alert quality and making big data small. They were able to develop additional internal capabilities based on the work done for the customer-facing feature. The interns' experience was marked by a well-scoped project, working as a team, quick agile cycles, and great mentorship. Two of the interns are continuing their work with Cloudflare after graduation, while others may join in the future.

The shift towards remote work due to COVID-19 has led to a significant change in Internet usage patterns, with traffic moving from office-focused areas like city centers and business parks to residential areas such as suburbs and outlying towns. Cloudflare's analysis of this geographical migration reveals that the overall volume of traffic in these locations has increased between 10% and 40% in just four weeks. The Internet, originally conceived for communication during a crisis, is now being utilized for its intended purpose as people work from home globally. Cloudflare continues to ensure smooth operation of their services during this time of increased network strain and offers free access to their Cloudflare for Teams products to support companies with remote employees.

On its 8th birthday in 2017, Cloudflare announced free unmetered DDoS Protection for all its plans. The company has been protecting over 26 million Internet properties from various types of attacks. Recently, it observed a decline in the proportion of L3/L4 DDoS attacks that exceed a rate of 30 Gbps. It also noticed an increase in smaller, more localized attacks and highly distributed L7 HTTP DDoS attacks. To tackle these challenges, Cloudflare developed new systems like dosd and IP Jails to detect and mitigate attacks faster and more efficiently. The company has made these new protections available by default at no additional charge for all its customers across all plans.

In the summer of 2017, Watson Ladd was an intern at Cloudflare where he worked on implementing a set of three interrelated Internet drafts aimed at improving connection reuse and reducing unnecessary time in loading resources. The project involved building a prototype using Go standard library and later integrating it into production systems based on variants of NGINX. Despite the challenges faced, Ladd gained valuable experience in collaborating on large software projects, working with git, and communicating with other implementers about issues found. He has since returned to Cloudflare full-time, continuing his work on extensions for TLS and time synchronization.

Cloudflare has migrated its CAPTCHA provider from Google's reCAPTCHA to hCaptcha due to privacy concerns and the need for more flexibility in customizing CAPTCHAs. The company uses CAPTCHAs as a challenge to determine whether traffic is malicious or good, with human intervention required for some challenges. Since its early days, Cloudflare has used Google's reCAPTCHA service, which was free but raised privacy concerns among some customers and faced blocking issues in regions like China. Earlier this year, Google informed Cloudflare that it would begin charging for reCAPTCHA, prompting the company to search for a better alternative. After evaluating various vendors and building its own system, Cloudflare chose hCaptcha due to its privacy-focused approach, performance, accessibility features, support for Privacy Pass, and ability to work in regions where Google is blocked. The company worked with hCaptcha to reduce costs and ensure scalability.

Cloudflare has introduced a new feature in its 1.1.1.1 w/ WARP app that provides mobile devices with the same level of security as offered by Cloudflare Gateway on home or office WiFi networks. The app encrypts traffic leaving the device, offering more private browsing experience. By adding secure DNS filtering to the app, users can block malicious domains flagged as phishing, command and control, or spam. This feature is available for both iOS and Android clients. Users need to upgrade their 1.1.1.1 w/ WARP app version and follow a few steps to enable Cloudflare Gateway for enhanced security on any network. The company also announced the beta release of 1.1.1.1 w/ WARP for Windows and macOS, with interested users able to sign up for the beta program.

Crossbow is an internal tool at Cloudflare used by its Technical Support Engineers to debug network issues from particular locations or individual servers. The tool was transitioned from a Product Engineering team to the Support Operations team in September last year. The Support Operations team, closely aligned with Cloudflare’s Technical Support Engineers, developed diagnostic tooling and Natural Language Processing technology to drive efficiency. Crossbow allows engineers to perform various activities such as running commands like traceroutes, cURL requests, and DNS queries, debugging product features, and performance using bespoke tools. The team deprecated the CLI (command line interface) and focused on improving the UI in Cloudflare’s internal tool for its Technical Support Engineers. They also redesigned the system architecture for Crossbow to adopt a more simple pub/sub pipeline and implemented a JWT authentication scheme. Through these improvements, usage of Crossbow increased by 4.5x within a four-week period. The team also deprecated other tools and rolled them into Crossbow, consolidating tooling and reducing overhead of developing support tooling across the business.

The text discusses the "conntrack" subsystem in Linux's network stack, which is part of the firewall system. It explains how this connection tracking facility works, its limitations, and potential issues that can arise when it gets filled up. The author also provides a detailed test setup using "unshare" to experiment with iptables and conntrack without affecting the host system. They highlight the importance of correctly applying conntrack and avoiding its use on inbound connections to prevent potential problems during SYN flood mitigation.

On April 4th at 12:01 PM, Connor Peshek introduced the fourteenth edition of "The Serverlist". This publication provides updates on serverless technology, developer tutorials, discussions with fellow serverless developers, and information about upcoming meetups and conferences. To receive this content directly in your email, sign up using the provided link while ensuring that your privacy is respected.

In this blog post, Avery Harnish shares their experience as an intern at Cloudflare in 2018 and how they contributed to improving the developer experience for Cloudflare Workers. They discuss setting up a local development environment, automating steps with Wrangler CLI tool, and implementing wrangler dev feature that allows testing of Workers using any HTTP client without deploying to production. The author also talks about integrating console.log functionality into wrangler dev for debugging purposes. Currently, wrangler dev is in alpha stage and the author encourages developers to try it out and provide feedback.

Cloudflare CEO Matthew Prince shares how the company is adapting during the COVID-19 pandemic, emphasizing their commitment to maintaining secure and reliable internet services for customers. Despite the crisis, they plan to continue hiring across all teams without layoffs. In light of canceled internships at other companies, Cloudflare has decided to double the size of its summer 2020 internship class and reopen applications. The company encourages other fortunate businesses to consider expanding their internship classes as well.

On April 1st, Cloudflare launched a filtered DNS service called "1.1.1.1 for Families" that allows users to restrict certain categories of websites. However, an error occurred whereby some sites related to the LBGTQIA+ community were inadvertently blocked due to incorrect categorization data from one of their licensed providers. The company immediately rectified the issue by removing the wrong "Adult Content" category and creating a new data structure with the correct definition. They also set up checks for known sites that should not be blocked in future updates.

On April 1, 2020, Cloudflare announced the launch of "1.1.1.1 for Families," a free service designed to protect home networks from malware and adult content. This new offering leverages Cloudflare's global network to ensure fast and secure connections worldwide while maintaining strong privacy guarantees. The setup process is simple, requiring only changing two numbers in the settings of home devices or routers. 1.1.1.1 for Families offers two default options: one that blocks malware and another that also blocks adult content. In the coming months, additional configuration settings will be provided, including allowlists, blocklists, and customizable blocking times for specific categories. This service is built on top of the same site categorization and filtering technology used in Cloudflare's Gateway product.

Cloudflare has announced the start of a beta for its WireGuard-based Warp VPN client on macOS and Windows, following its launch last year on iOS and Android. The basic service will be free, with Warp+ support to leverage Cloudflare's Argo network for faster internet performance expected in the coming months. Existing Warp+ subscribers will be among the first to try Warp for macOS and Windows, while others can sign up on the linked page to join the waitlist.

Cloudflare now supports security keys as a two factor authentication (2FA) method for all users on WebAuthn-supported browsers. This feature enhances account security by using public key cryptography and hardware security keys or built-in biometric support. WebAuthn is more resistant to phishing attacks, credential-based attacks, and offers better privacy guarantees compared to other 2FA methods. It also simplifies the user experience by eliminating the need for mobile app-based authentication.