codedamn

Redis Modules: Expanding Capabilities with Custom Extensions

Redis is an in-memory data structure store, primarily used as a database, cache, and message broker. It is known for its high performance, flexibility, and wide range of data structures. One of the lesser-known features of Redis is its support for custom modules, which can significantly expand its capabilities. In this blog post, we will explore Redis modules, understand how they can be used to enhance Redis' capabilities with custom extensions, and provide step-by-step examples for creating and integrating modules.

What are Redis Modules?

Redis modules are dynamic libraries that can be loaded into a Redis server to extend its functionality. These libraries are written in C and can introduce new data types, commands, and other features that are not natively available in Redis. Modules enable developers to tailor Redis to their specific use case, without having to modify the core Redis source code.

Why Use Redis Modules?

There are several reasons why you might want to use Redis modules:

  1. Custom data types: Redis modules allow you to create and use new data types that are not available in the core Redis implementation. These custom data types can help you store and manipulate data in ways that better suit your application's requirements.
  2. New functionality: Modules can introduce new functionality to Redis, such as additional commands or enhancements to existing features. This can help you optimize your Redis usage for specific tasks or improve the overall performance of your application.
  3. Performance improvements: Some modules are designed to optimize Redis for specific workloads, such as time-series data or graph databases. By using these specialized modules, you can achieve better performance than with the built-in Redis data structures and commands.
  4. Ease of development: Since Redis modules are written in C, you can take advantage of the extensive ecosystem of C libraries and tools when developing your modules. Additionally, the Redis module API is well-documented and straightforward to work with.

Creating Your First Redis Module

In this section, we will walk you through creating a simple Redis module that implements a new command called INCRBYFLOAT. This command will increment a key's value by a floating-point number, which is not supported by Redis' built-in INCRBY command.

Prerequisites

Before we begin, you will need the following:

  1. A Linux or macOS machine (Windows is not officially supported for Redis module development)
  2. Redis 4.0 or later installed on your machine
  3. A C compiler and build tools (such as GCC and Make)

Setting Up the Module Skeleton

First, let's create a new directory for our module and navigate to it:

mkdir incrbyfloat
cd incrbyfloat

Next, create a new file called incrbyfloat.c with the following contents:

#include "redismodule.h"

int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    // Module initialization logic goes here

    return REDISMODULE_OK;
}

This is the basic skeleton of a Redis module. The RedisModule_OnLoad function is the entry point for the module and is called when the module is loaded by Redis. Currently, our module does nothing, but we will add the INCRBYFLOAT command in the next step.

Implementing the INCRBYFLOAT Command

Now, let's implement the INCRBYFLOAT command. Add the following function to the incrbyfloat.c file:

int IncrByFloatCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    // Command implementation goes here

    return REDISMODULE_OK;
}

This function will be called when a user invokes the INCRBYFLOATINCRBYFLOAT` command in Redis. Now, let's add the implementation details.

First, we need to check if the number of arguments is correct. The INCRBYFLOAT command expects two arguments: the key to increment and the increment value. Update the IncrByFloatCommand function with the following code:

int IncrByFloatCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    if (argc != 3) {
        return RedisModule_WrongArity(ctx);
    }

    // Remaining command implementation goes here
}

Next, we need to parse the floating-point increment value from the input argument. Add the following code to the IncrByFloatCommand function:

double increment;
if (RedisModule_StringToDouble(argv[2], &increment) != REDISMODULE_OK) {
    return RedisModule_ReplyWithError(ctx, "ERR invalid increment");
}

Now that we have the increment value, let's retrieve the current value of the key. If the key does not exist, we will treat its value as 0. Add the following code to the IncrByFloatCommand function:

RedisModule_AutoMemory(ctx);
RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_READ | REDISMODULE_WRITE);

double currentValue = 0.0;
if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_EMPTY) {
    if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_STRING) {
        return RedisModule_ReplyWithError(ctx, "ERR wrong key type");
    }
    size_t valueLen;
    char *valueStr = RedisModule_StringDMA(key, &valueLen, REDISMODULE_READ);
    currentValue = strtod(valueStr, NULL);
}

Now that we have both the current value and the increment, we can calculate the new value, update the key, and return the updated value to the user. Add the following code to the IncrByFloatCommand function:

double newValue = currentValue + increment;
RedisModuleString *newValueStr = RedisModule_CreateStringPrintf(ctx, "%.17g", newValue);
RedisModule_StringSet(key, newValueStr);
RedisModule_ReplyWithDouble(ctx, newValue);

Finally, we need to register our new command with Redis when the module is loaded. Update the RedisModule_OnLoad function with the following code:

int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    if (RedisModule_Init(ctx, "incrbyfloat", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) {
        return REDISMODULE_ERR;
    }

    if (RedisModule_CreateCommand(ctx, "incrbyfloat", IncrByFloatCommand, "write fast", 1, 1, 1) == REDISMODULE_ERR) {
        return REDISMODULE_ERR;
    }

    return REDISMODULE_OK;
}

Now our module is complete! The final incrbyfloat.c file should look like this:

#include "redismodule.h"

int IncrByFloatCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    if (argc != 3) {
        return RedisModule_WrongArity(ctx);
    }

    double increment;
    if (RedisModule_StringToDouble(argv[2], &increment) != REDISMODULE_OK) {
        return RedisModule_ReplyWithError(ctx, "ERR invalid increment");
    }

    RedisModule_AutoMemory(ctx);
    RedisModuleKey *key = RedisModule_OpenKey(ctx, argv[1], REDISMODULE_READ | REDISMODULE_WRITE);

    double currentValue = 0.0;
    if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_EMPTY) {
        if (RedisModule_KeyType(key) != REDISMODULE_KEYTYPE_STRING) {
            return RedisModule_ReplyWithError(ctx, "ERR wrong key type");
        }
        size_t valueLen;
        char *valueStr = RedisModule_StringDMA(key, &valueLen, REDISMODULE_READ);
        currentValue = strtod(valueStr, NULL);
    }

    double newValue = currentValue + increment;
    RedisModuleString *newValueStr = RedisModule_CreateStringPrintf(ctx, "%.17g", newValue);
    RedisModule_StringSet(key, newValueStr);
    RedisModule_ReplyWithDouble(ctx, newValue);

    return REDISMODULE_OK;
}

int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
    if (RedisModule_Init(ctx, "incrbyfloat", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) {
        return REDISMODULE_ERR;
    }

    if (RedisModule_CreateCommand(ctx, "incrbyfloat", IncrByFloatCommand, "write fast", 1, 1, 1) == REDISMODULE_ERR) {
        return REDISMODULE_ERR;
    }

    return REDISMODULE_OK;
}

Compiling and Testing the Module

Now that our module is complete, we need to compile it into a shared library. Create a new file named Makefile in the incrbyfloat directory with the following contents:

LIBNAME = incrbyfloat.so
CC = gcc
CFLAGS = -fPIC -std=gnu99 -Wall -W -Wwrite-strings -g -ggdb
LDFLAGS = -shared

all: $(LIBNAME)

$(LIBNAME): incrbyfloat.o
    $(CC) $(LDFLAGS) -o $@ $^

%.o: %.c
    $(CC) -c $(CFLAGS) -o $@ $<

clean:
    rm -f $(LIBNAME) *.o

Now run make to compile the module:

make

This should create a incrbyfloat.so file in the current directory.

To test our module, first start a Redis server with our module loaded:

redis-server --loadmodule ./incrbyfloat.so

Now, in a separate terminal, connect to the Redis server using the redis-cli tool:

redis-cli

Inside the Redis CLI, try the new INCRBYFLOAT command:

SET testkey "3.5"
INCRBYFLOAT testkey 2.5

The output should be 6, which is the correct result of adding 3.5 and 2.5.

FAQ

Q: How do I load a Redis module at runtime?

You can load a Redis module at runtime using the MODULE LOAD command:

MODULE LOAD /path/to/your/module.so

Q: Can I use other programming languages to develop Redis modules?

Yes, you can use other languages, such as Rust or C++, to develop Redis modules. However, you may need to use a wrapper library that provides a binding to the Redis module API.

Q: Can I use Redis modules in a cluster setup?

Yes, Redis modules can be used in a cluster setup. However, some modules may require additional configuration or setup to work correctly ina clustered environment. Be sure to consult the module's documentation to ensure proper configuration and compatibility with Redis Cluster.

Q: Are there any limitations to using Redis modules?

While Redis modules offer great flexibility and customization, there are some limitations:

  1. Modules may introduce new commands and data types that are not understood by Redis clients, which may require you to update your client libraries or application code to fully utilize the module's features.
  2. Using custom modules may affect the overall performance of Redis, especially if the module's implementation is not optimized. Carefully test and benchmark the module before deploying it in a production environment.
  3. Modules can introduce new security risks, such as vulnerabilities in the module's code or dependencies. Be sure to review the module's source code and keep it up-to-date with the latest security patches.
  4. When using modules, you may encounter compatibility issues between different module versions, Redis versions, or client libraries. Always test the compatibility of your modules and libraries before deploying them in production.

Q: Can I use multiple Redis modules simultaneously?

Yes, you can use multiple Redis modules simultaneously by loading them into your Redis server, either at startup or at runtime. However, be aware that some modules may conflict with each other or introduce unintended side effects. Always test your modules together to ensure they work correctly and do not cause performance or stability issues.

Q: How do I unload a Redis module?

You can unload a Redis module using the MODULE UNLOAD command:

MODULE UNLOAD module_name

Keep in mind that unloading a module will also unload any data stored using the module's custom data types or structures. Make sure you have a backup or have migrated the data before unloading a module.

Sharing is caring

Did you like what Mehul Mohan wrote? Thank them for their work by sharing it on social media.

0/10000

No comments so far

Curious about this topic? Continue your journey with these coding courses:

Course thumbnail for Redis Caching Concepts with Node.js
4.7

1.48k students learning

Photo of Mehul Mohan

Mehul Mohan

Redis Caching Concepts with Node.js