CharybdeFS: a new fault-injecting filesystem for software testing

CharybdeFS is a new FUSE-based fault-injecting filesystem designed to test how databases, particularly ScyllaDB, handle filesystem errors. It operates as a pass-through layer between a program and a real filesystem, enabling error simulation through a Thrift RPC interface that allows for easy scripting, primarily using Python. By simulating a range of filesystem errors, such as disk IO errors or latency spikes, CharybdeFS aids in ensuring that databases can maintain data integrity and proper functionality under adverse conditions. It has been instrumental in identifying and addressing ScyllaDB's lack of a consistent strategy for handling disk errors, prompting the implementation of a shutdown feature on I/O errors. The tool's flexibility extends beyond ScyllaDB, making it suitable for testing various server software, including other databases and storage systems. CharybdeFS is open-source and available on GitHub, with additional features like distributed and longevity tests for ScyllaDB planned for future updates.