Building an experimentation platform: Assignment

In the context of gating or experimentation platforms, the primary challenge is determining user group assignments, which can be addressed through various methods, each with its own advantages and disadvantages. One approach involves maintaining a comprehensive table of users and their corresponding group assignments, simplifying analysis and updates but introducing latency issues and noise due to overexposure. A more efficient strategy involves logging "exposure" events for users who actually encounter experiments, preventing discrepancies in logging and avoiding Sample Ratio Mismatch. For user assignments, a deterministic method is employed, such as using ID-based even-odd splits; however, this lacks the necessary randomization for effective analysis. The solution is to implement a unique deterministic split for each experiment using a generated salt and a Sha256 hash, allowing for finer control over experiment exposure percentages. For gates, fixed bucket assignments ensure deterministic feature rollouts, whereas experiments benefit from random bucket assignments to ensure varied user exposure across iterations.