Home / Companies / ScyllaDB / Blog / Post Details
Content Deep Dive

ScyllaDB Developer Hackathon: New IMR infrastructure

Blog post from ScyllaDB

Post Details
Company
Date Published
Author
Botond Denes
Word Count
1,104
Language
English
Hacker News Points
-
Summary

In the ScyllaDB Developer Hackathon, a new approach to in-memory representation (IMR) for cells was explored, focusing on addressing inefficiencies in the previous complex template-metaprogramming solution. Originally, ScyllaDB's IMR format used a simple buffer with a bitset to determine a cell's state, but this required linearization of the buffer, leading to challenges with memory fragmentation. The hackathon aimed to revert to a simpler, open-coded IMR format while ensuring compatibility with fragmented buffers by leveraging C++ iterators to create the illusion of contiguity. This iteration allows for efficient processing of smaller buffers and provides a fallback for larger ones. The event produced a proof of concept demonstrating the feasibility of this approach, with further work needed to polish and test the solution for broader adoption.