Making a Shard-Aware Python Driver for ScyllaDB, Part 1

In his talk at the virtual Europython 2020 Conference, Alexys Jacob, CTO of Numberly, discussed the architecture of ScyllaDB and the benefits of creating a shard-aware Python driver. He compared the token ring architectures of ScyllaDB and Apache Cassandra, explaining how both systems organize nodes into rings where each node has equal importance and data is replicated for high availability. However, ScyllaDB goes further by distributing tokens among CPU cores, offering a more granular data distribution which improves performance. Alexys detailed how client drivers query these databases, highlighting the advantages of a token-aware policy that enhances query efficiency by minimizing network hops and server load. He noted that while token awareness is beneficial, ScyllaDB's ability to shard data to CPU cores allows for even more optimized query routing. Despite existing shard-aware drivers for Java and Go, there was none for Python, motivating Alexys to advocate for its development. He shared insights on partitioning, querying, and the potential performance gains from a Python shard-aware driver, setting the stage for further exploration of its implementation and results in a subsequent presentation.