Kinesis consumer balancing

Knock, a company providing APIs for integrating notifications into products, previously faced issues with uneven distribution of AWS Kinesis consumers across Kubernetes pods during deployments, leading to CPU overutilization. To resolve this, they implemented a hashring approach to distribute consumers evenly across their cluster. By using consistent hashing, each shard and consumer combination is deterministically assigned to a specific pod, ensuring balanced load distribution and self-healing capabilities in case of failures. This solution significantly reduced peak CPU utilization and improved platform performance and reliability. The hashring approach also minimizes the need for manual intervention during uneven consumer distribution, contributing to a consistent and reliable deployment process. Despite the system's complexity, it enhances Knock's ability to provide a robust notification solution, ensuring efficient operation and customer satisfaction.