ScyllaDB Developer Hackathon: ScyllaDB + S3

At the ScyllaDB Developer Hackathon, the team embarked on a project to integrate Binary Large Object (BLOB) support in ScyllaDB using the S3 API, which resulted in the development of several innovative features. The team, composed of senior engineers from various countries, faced initial challenges but succeeded in creating a proof of concept and a production-grade implementation in C++. They devised an Object-Aware Compaction Strategy that enhances the efficiency of storing and retrieving large objects by grouping data pieces belonging to the same object, reducing the need for traditional compaction. The project also included features like multipart uploads for handling very large objects and the use of Filesystem in Userspace (FUSE) to expose ScyllaDB as a file system. This allowed ScyllaDB to be accessed via tools compatible with the S3 API and enabled use cases like streaming media content directly from the database. The hackathon's achievements, driven by passion and collaboration, demonstrated the potential of ScyllaDB as a versatile storage solution for large binary objects, although the features are yet to be released in a public version.