Company
Date Published
Author
Redis
Word count
1204
Language
English
Hacker News points
None

Summary

The author was developing a Rust program, redismodule-rs, which is a Redis module written in Rust, and encountered the need to set up a custom memory allocator. Normally, when a Rust program needs to allocate some memory, it uses the global allocator defined in the program, but this behavior is problematic for Redis modules because they are built as shared libraries that may not use the system allocator or jemalloc, which can lead to inconsistent allocation behavior and invisible memory allocations. To solve this issue, the author implemented a custom memory allocator using the GlobalAlloc trait, which provides hooks like RedisModule_Alloc and RedisModule_Free to make Redis aware of allocated memory. However, when building a module with this custom allocator and loading it into Redis, it crashed due to a null pointer dereference caused by name mangling in Rust. The author discovered that the functions of the Redis modules API are accessed via C function pointers, which are initialized explicitly by Redis as part of the module initialization process, causing a chicken-and-egg problem. After experimenting with various approaches, the author found a solution by adding a flag to the custom allocator that causes allocations to be passed through to the system allocator at startup and toggles it after the module initialization is complete, ensuring that all previously allocated memory is freed before switching.