November 2023 Summaries

Cloudflare Workers provides various tools to help developers debug their applications from local environments to production. These include logs, console.log statements, DevTools for network requests and memory heap snapshots, and breakpoint debugging. Breakpoint debugging allows developers to pause code execution at specific points to evaluate its behavior. To set up breakpoint debugging in VSCode or WebStorm, developers need to create a launch configuration file with the appropriate settings. Cloudflare Workers' runtime, workerd, enables breakpoint debugging by adding an I/O thread for handling CDP messages and source maps for rendering original source code in the debugger. Future improvements include automatic attachment of IDEs to dev servers and support for Node-style logging hooks.

In November 2023, Cyber Monday was the busiest day for US Internet traffic, with Black Friday coming in second place. E-commerce DNS traffic showed a consistent rise throughout November, with peak levels maintained during Black Friday week. Electronics and fast fashion e-commerce sites saw higher traffic on Black Friday than on Cyber Monday. Thanksgiving Day saw the lowest daily fraction of DDoS attacks in November 2023.

The Forrester Wave™ has recognized Cloudflare as a leader in Edge Development Platforms, Q4 2023, with top scores in the current offering category. Over one million developers are using its platform to build applications across Workers, Pages, R2, KV, Queues, Durable Objects, D1, Stream, Images, and more. Key strengths of the platform include interoperability with Cloudflare's programmable global CDN and intelligent workload placement. Additionally, developers can leverage Smart Placement to optimize Workers for optimal performance. The ease of onboarding and adoption by developers with no prior background is consistently praised. Cloudflare continues to expand its platform capabilities, focusing on full-stack application development support. Its commitment to web standard APIs and compatibility with popular open-source frameworks make the platform accessible to a wide range of developers. The integration of GPUs and LLMs into the developer platform aims to position Cloudflare as a competitive alternative to public cloud hyperscalers for workloads. The report also highlights upcoming features like Workers AI Inference, which allows developers to use open-source AI models without any infrastructure provisioning or paying for unused capacity. Future enhancements include improved observability and secret management tools across Cloudflare products to support advanced applications and mission-critical workloads on the platform.

The author has recently joined Cloudflare as Head of Australia & New Zealand (ANZ) with a focus on extending their valuable relationships with customers and partners in the region. With over 25 years of experience in IT, the author has previously worked for notable companies like Oracle, Salesforce, and Zendesk. They are excited to join Cloudflare as it aligns with their interest in solving real-world problems through technology. The company's mission is to help build a better internet, addressing issues such as security, performance, and reliability of the modern internet by leading the transition at the network layer through its extensive global network. The author lives in Melbourne and enjoys the city's offerings, and they are looking forward to working with customers and partners on their projects using Cloudflare's solutions.

Thanksgiving in the US resulted in a 10% daily decrease in internet traffic compared to the previous week, consistent with the trend observed in 2022. The peak drop occurred around 21:00 UTC (16:00 EST and 13:00 PST), representing a 22% decrease from the same hour of the previous week. Internet traffic began to pick up again during early Black Friday morning in the US, showing an increase of as much as 18% compared to the previous week. The "Thanksgiving time" varied across states, with central US states experiencing the largest drop in traffic around 13:00 local time and coastal US states having a later Thanksgiving, around 17:00 local time. Mobile devices accounted for 54.5% of internet traffic on Thanksgiving Day, an increase from previous years. DNS traffic to food delivery sites was higher than the previous week on Tuesday and Wednesday but dropped significantly on Thanksgiving Day itself.

Cloudflare has announced that Stable Diffusion and Code Llama are now available as part of their Workers AI, running in over 100 cities across the company's global network. Stable Diffusion is an image-generation model that can create images based on text input, while Code Llama is a language model optimized for generating programming code. The integration makes it easy to generate images or code using APIs and workers. Additionally, Workers AI inference is now available in 100 cities, ensuring closer proximity to users.

In this article, we explored the basics of attention mechanisms in large language models (LLMs) such as Mistral 7B, focusing on understanding how they work and why they are important for improving model performance. We covered three main aspects of attention: constant values, vertical stacking, and horizontal stacking or multi-head attention. Constant values refer to the learned parameters in an AI model that are adjusted during training to improve its performance. These parameters control the flow of information within the model and allow it to focus on relevant parts of the input data. Vertical layered stacking is a way to build multiple layers of attention mechanisms, with each layer building upon the output of the previous one. This allows the model to focus on different aspects of the input data at various levels of abstraction, which can lead to better performance on certain tasks. Horizontal stacking or multi-head attention involves carrying out multiple attention operations in parallel using unique linear projections for each set of Q-K-V inputs. These parallel attention blocks are called "attention heads," and they allow the model to focus on different parts of the input data concurrently. There are three common arrangements of attention mechanisms used by large language models: multi-head attention, grouped-query attention, and multi-query attention. Multi-query attention uses only a single set of K and V vectors for all Q vectors, reducing memory usage but potentially impacting performance on some tasks. Grouped-query attention combines the best of both worlds by using a fixed ratio of one set of K and V vectors for every Q vector, retaining high performance while minimizing memory consumption. Mistral 7B utilizes grouped-query attention combined with sliding window attention, making it highly efficient in terms of latency and throughput while still maintaining strong performance on benchmarks compared to larger models like OpenAI's GPT-3 series. By understanding these aspects of attention mechanisms in large language models, developers can make more informed decisions when choosing a model for their specific use case or application. The Workers AI platform now supports the Mistral 7B model, allowing developers easy access to its capabilities through serverless GPU-powered inference services. This makes it easier for developers to build and deploy cutting-edge AI applications without having to manage complex infrastructure requirements themselves. As the field of large language models continues to evolve, understanding these underlying mechanisms will become increasingly important for harnessing their full potential in real-world applications.

Between November 1, 2022, and August 31, 2023, Cloudflare protected various groups working to promote democracy worldwide, including political campaigns, organizations that work on a range of topics related to voting rights, election management bodies, and others. During this period, Cloudflare mitigated a combined total of over 46 million threats to the websites and systems of these groups. We also saw a significant increase in traffic during election times and a general trend of more requests being blocked by our WAF than due to DDoS attacks. In 2023, we continued to work with election groups globally through partnerships such as Cloudflare for Campaigns, Project Galileo, and IFES. We are committed to ensuring that all groups working to promote democracy around the world have the tools they need to stay secure online.

In this blog post, we learned about how the Linux kernel handles executing a binary file and how all components are linked together. We then looked at an example of hooking into a function within a binary to change its behavior using the AArch64 architecture's assembly language. Finally, we discussed the importance of security when processing external inputs.

The article discusses the concept of Zero Trust security, which involves defining granular controls and authorization policies per application, user, and device. It highlights how administrators can now easily audit all active user sessions and associated data used by their Cloudflare One policies, enabling them to maintain an improved ability to troubleshoot and diagnose Zero Trust deployments in a single, simple control panel. The article also provides a primer on application authentication and authorization mechanisms, focusing on HTTP cookies, JSON Web Tokens (JWT), and the stateless nature of JWTs. It concludes by discussing how Cloudflare has built a better way to debug issues related to user identity in Zero Trust without sharing JWTs or HAR files back and forth.

Cloudflare has introduced Custom Lists that enable users to create and manage lists of hostnames and Autonomous System Numbers (ASNs). This feature helps in efficiently managing traffic from ISPs and cloud providers, as well as running managed or custom rulesets only on specific hosts. Enterprise accounts can have up to 10 Custom Lists with a total of 10,000 items shared across all lists. In the near future, Cloudflare plans to introduce string lists that will allow users to create more complex use cases, such as blocking certain user agents or restricting access to specific URLs.

Cloudflare has launched a serverless GPU-powered inference platform called Workers AI, which offers off-the-shelf models running seamlessly with its existing Worker service. The platform allows developers to build powerful and scalable AI applications quickly. Key features include streaming responses for all Large Language Models (LLMs), larger context and sequence windows, and a full-precision Llama-2 model variant. In addition, server-sent events are now supported in the browser API for streaming text responses with LLM models. The introduction of higher precision and longer context and sequence lengths will provide a better user experience and enable new applications using large language models.

On November 2, Fastly experienced an outage caused by a failure in their primary data center, which resulted in the loss of some services and features for customers. The team immediately began working on restoring the lost functionality while ensuring that other critical systems remained stable. By November 4, all affected services were fully restored with no data loss reported. Fastly has identified several areas for improvement and will be implementing changes to prevent similar incidents from occurring in the future.

A service disruption occurred in Workers KV, a distributed key-value store used by many applications including first-party Cloudflare products like Pages, Access, and Zero Trust. This was due to an issue with the deployment tool which resulted in production traffic being directed to a version that was not authorized for production access, leading to HTTP 401 errors. The issue affected parts of the Cloudflare dashboard and other services dependent on Workers KV. The incident lasted approximately one hour before being resolved. The team has identified the root cause and is taking several steps to prevent similar incidents in the future. These include improving the deployment tooling, enhancing the rollback process, adding pre-checks to deployments, hardening progressive deployment scripts, and ensuring compatibility between applications and their environments during deployments. The company apologizes for any inconvenience caused by this incident.