How to Integrate Redis and Kafka in Golang: A Complete Guide in 2025

In today’s fast-paced backend systems, handling real-time data and ensuring efficient caching is a necessity. Two of the most reliable tools for this job are Redis and Apache Kafka. This guide will show you how to integrate both into a Golang project cleanly and effectively.

Whether you’re a beginner or an experienced developer, this guide breaks down every step in a simple, crisp manner — with no fluff.

kafka redis golang 2025 — *How to Integrate Redis and Kafka in Golang (2025) — Simple and Fast Guide*

What Are Redis and Kafka?

Redis is an open-source, in-memory data structure store used as a database, cache, and message broker. It’s known for its speed and simplicity.

Apache Kafka is a distributed event streaming platform capable of handling trillions of events a day. It’s used for building real-time data pipelines and streaming apps.

Why Use Redis and Kafka Together?

Redis for ultra-fast caching and temporary data storage.
Kafka for reliable, asynchronous message streaming between services.
Together, they can build scalable, real-time applications.

Example use case: A stock trading platform where Kafka streams trade events and Redis caches the latest stock prices for quick UI display.

Required Tools and Libraries (2025)

Go Modules: Use Go Modules for dependency management.

Libraries:

github.com/go-redis/redis/v9 — Official Redis client for Golang.
github.com/segmentio/kafka-go — High-performance Kafka client for Go.

Install them:

go get github.com/go-redis/redis/v9
go get github.com/segmentio/kafka-go

Redis Integration in Golang

Connect to Redis

package main

import (
    "context"
    "fmt"
    "github.com/go-redis/redis/v9"
)

var ctx = context.Background()

func main() {
    rdb := redis.NewClient(&redis.Options{
        Addr:     "localhost:6379", // Redis server address
        Password: "",               // No password set
        DB:       0,                // Default DB
    })

    // Ping the Redis server
    pong, err := rdb.Ping(ctx).Result()
    if err != nil {
        panic(err)
    }
    fmt.Println("Connected to Redis:", pong)
}

Basic Redis Operations

// Set a key
err := rdb.Set(ctx, "stock_price", "120.5", 0).Err()
if err != nil {
    panic(err)
}

// Get a key
price, err := rdb.Get(ctx, "stock_price").Result()
if err != nil {
    panic(err)
}
fmt.Println("Latest Stock Price:", price)

Note: The 0 in Set means no expiry. Set a duration like 10*time.Second for temporary caching.

Kafka Integration in Golang

Kafka Producer (Send Messages)

package main

import (
    "context"
    "log"
    "github.com/segmentio/kafka-go"
)

func main() {
    writer := kafka.NewWriter(kafka.WriterConfig{
        Brokers: []string{"localhost:9092"},
        Topic:   "stock_updates",
    })
    defer writer.Close()

    msg := kafka.Message{
        Value: []byte("AAPL 120.5"),
    }

    err := writer.WriteMessages(context.Background(), msg)
    if err != nil {
        log.Fatal("failed to write message:", err)
    }

    log.Println("Message sent to Kafka topic")
}

Kafka Consumer (Read Messages)

package main

import (
    "context"
    "log"
    "github.com/segmentio/kafka-go"
)

func main() {
    reader := kafka.NewReader(kafka.ReaderConfig{
        Brokers: []string{"localhost:9092"},
        Topic:   "stock_updates",
        GroupID: "stock_price_consumers",
    })
    defer reader.Close()

    for {
        msg, err := reader.ReadMessage(context.Background())
        if err != nil {
            log.Fatal(err)
        }
        log.Printf("Received: %s\n", string(msg.Value))
    }
}

How to Connect Redis and Kafka Together?

Concept:

Kafka producer sends a message when stock price updates.
Kafka consumer reads the message.
Consumer updates Redis cache with new stock price.

Example Workflow Code

for {
    msg, err := reader.ReadMessage(context.Background())
    if err != nil {
        log.Fatal(err)
    }
    
    err = rdb.Set(ctx, "stock_price", string(msg.Value), 0).Err()
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("Updated Redis with: %s\n", msg.Value)
}

Key Terms You Should Know

Broker: Kafka server that stores and forwards messages.
Topic: A category or feed name to which messages are published.
Consumer Group: A set of consumers sharing the load of reading messages from a topic.
Cache: A temporary data store for fast access.

Best Practices (2025)

Always set Redis expiry for cache data to avoid stale info.
Use Kafka partitions for scaling consumers.
Handle message retries and dead-letter queues.
Secure Redis and Kafka using SSL/TLS for production.
Use environment variables for config values (don’t hardcode IPs or ports).

Final Thoughts

Integrating Redis and Kafka with Golang is simpler than it seems. This setup ensures real-time, scalable, and high-performance applications. In 2025, both these technologies remain industry-standard choices for handling high-volume, real-time data workflows.

Master these integrations, and you’re set to build fast, reliable backend services.