October 2018 Summaries

At the annual Demuxed conference for video developers, several key insights were shared. Firstly, it was revealed that 90% of YouTube viewers do not care about video quality, which has implications for adaptive bitrate (ABR) algorithms and machine learning approaches to optimize video delivery. Secondly, the codec war between AOM and MPEG is intensifying with the development of royalty-free video codecs like AV1. Thirdly, sub-media-segment chunked-transfer is becoming a preferred method for achieving low latency streaming at scale. Fourthly, machine learning is increasingly being used to improve streaming video experiences, such as in per-title encoding and content classification. Lastly, the impact of repeatedly watching Big Buck Bunny on video developers was humorously highlighted.

Mux, a stream-processing system built to handle large volumes of video-streaming logs, will be represented at the Scale by the Bay conference over November 15-17, 2018 at Twitter HQ in San Francisco. The company's talk will focus on the challenges and solutions associated with processing video access logs at scale. These include scalable log storage and processing, handling unexpected load spikes, supporting multiple CDNs, avoiding a thundering-herd against the asset database, and ensuring exact-once stream processing. Mux has used Apache Kafka and Flink to build a log-processing pipeline that can scale as the company's video service continues to grow.

The text discusses how Elixir and its web framework Phoenix were used to manage large real-time data sets across many channel topics for a real-time analytics product. It highlights the use of Erlang's OTP (Open Telecom Platform) and Phoenix's Channel / Presence features, which are designed for fault tolerant, concurrent systems. The text also covers how WebSockets were used to communicate between server and client, how dynamic supervision was implemented with Elixir's DynamicSupervisor feature, and how GenServer processes were used for information polling. Furthermore, it explains how process crashes were isolated, how subscription duplication across nodes was prevented using globally registered processes and process monitors, and how socket subscription state was kept server-side with Presence. The text concludes by discussing some technical considerations such as the eventually consistency of Presence, potential race conditions, zombie processes, soft real time, and resource management.

The text discusses the challenges in evaluating viewer experience for online videos and how Mux Data has improved its scoring methodology. It explains that decision-makers often face uncertainty when dealing with multiple outcomes and competing goals, and having clear data can help make smart decisions. The new scoring update for Mux Data involves mapping utility curves for different attributes, optimizing parameters of these utility curves, formulating trade-off equations to combine secondary goals into a single overall score, and evaluating the new overall score on key metrics. The improved methodology allows better understanding of the relationship between QoS metrics and end-user experience, and can be fine-tuned using machine learning for specific customers.

Mux Data's Viewer Experience Score is designed to provide meaningful data about video performance at a quick glance while allowing users to delve deeper when needed. The score is composed of four underlying scores focusing on playback failures, rebuffering, startup time, and video quality. Scores are useful as they summarize the entire story in just a few words, with the headline being a single number that describes how happy users are with system performance. Mux Data has recently released an updated version of its Viewer Experience Score, which includes changes such as moving from binary scores to a function, capturing tradeoffs, and extending the scale.