AWS Redis Cluster

Back to: AWS-Basics-Advanced

Introduction

Redis is a super-fast, in-memory key-value store that’s widely used as a cache, session store, message broker, and even for real-time analytics.

But when applications grow, a single Redis server isn’t enough. It might run out of memory, become a performance bottleneck, or fail and bring your system down.

That’s where Redis Cluster comes in.

Redis Cluster allows you to:

• Distribute (shard) data across multiple Redis nodes.
• Replicate data for high availability.
• Handle automatic failover if a node fails.

What is Redis Cluster?

Redis Cluster is a group of Redis nodes that work together. Instead of all data living on one node, the data is split across multiple nodes (shards).

Each shard has:

• Primary node → handles writes.
• Replica node(s) → copies of the primary for backup and scaling reads.

👉 If a primary fails, its replica is automatically promoted to become the new primary.

 

What is Sharding?

Sharding = Splitting data into smaller parts and distributing it across nodes.

Redis divides data into 16,384 hash slots. Each key belongs to one slot, and slots are distributed across shards.

Example:

Imagine you have 3 shards:

• Shard 1 → slots 0–5460
• Shard 2 → slots 5461–10922
• Shard 3 → slots 10923–16383

When you store a key like “user:1”, Redis automatically calculates its slot and assigns it to the right shard.

So your keys are spread across multiple nodes, avoiding memory or CPU overload on a single node.

👉 Benefit: Horizontal scaling — you just add more shards when your dataset grows.

Redis Cluster on AWS (ElastiCache)

On AWS, you don’t have to set up Redis manually. Amazon ElastiCache for Redis provides:

• Cluster mode enabled → automatic sharding and replicas.
• Multi-AZ replication → for disaster recovery.
• Monitoring with CloudWatch → track performance and failures.
• Scaling options → add/remove shards easily.

For example, you can create a 6-node Redis cluster (3 primaries + 3 replicas) in just a few clicks in AWS.

 

If you configure 2 shards with 2 replicas each, the cluster has:

• Shard 1 →
  • Primary 1
  • Replica 1A
  • Replica 1B
• Shard 2 →
  • Primary 2
  • Replica 2A
  • Replica 2B

 

openssl rand -base64 32

XB6BH5iz23z+57WPGeNgWBiNwpglxKLnF0JeP4DDPV4=

 

Python

yum install python3-pip -y

pip install redis

What is a Global Datastore?

A Global Datastore allows you to replicate your Redis cluster across multiple AWS regions.

• It’s designed for low-latency, cross-region read access and disaster recovery.
• You can have one primary region (read/write) and multiple secondary regions (read-only replicas).
• Changes in the primary region are asynchronously replicated to all secondary regions.

2️⃣ Key Features

Feature	Description
Cross-region replication	Keep data in sync across multiple AWS regions
Disaster Recovery	If the primary region fails, a secondary region can be promoted to primary
Low-latency global reads	Applications in secondary regions can read locally
Automatic replication	AWS handles asynchronous replication automatically
Cluster mode supported	Works with Redis Cluster (sharded)

3️⃣ Architecture Example

Region US-East-1 (Primary)

├── Shard 1: Primary Node + Replicas

├── Shard 2: Primary Node + Replicas

 

Region US-West-2 (Secondary)

├── Shard 1: Read-only replica

├── Shard 2: Read-only replica

 

Replication: Async (Primary -> Secondary)

• Writes → only in primary region
• Reads → can be in primary or secondary region
• Failover → you can promote a secondary region to primary if needed

4️⃣ Use Cases

1. Disaster Recovery → automatic failover to another region in case of regional outage
2. Global Applications → provide low-latency reads to users in different regions
3. Analytics / Reporting → secondary regions can serve heavy read workloads without affecting primary

Steps to Create a Redis Global Datastore

---

🧭 Step 1: Create a Primary Redis Cluster

1. Go to AWS Management Console → ElastiCache → Redis

2. Click Create cluster

3. Choose:

   • Cluster mode enabled (recommended)

   • Name: redis-primary

   • Region: us-east-1

   • Node type: cache.t3.medium

   • Number of shards: 1

   • Replicas per shard: 1

4. Choose Subnet Group and Security Group

5. Click Create

✅ This creates your primary Redis replication group.

---

🌍 Step 2: Create Global Datastore

Once your primary cluster is ready:

1. Navigate to ElastiCache → Global Datastore

2. Click Create Global Datastore

3. Select your primary Redis replication group (created in Step 1)

4. Click Add secondary region

5. Choose Region (e.g., ap-south-1)

6. Click Create

AWS automatically:

• Creates a replica cluster in the chosen secondary region.

• Starts cross-region replication.

---

🌏 Step 3: Verify Replication

1. In Global Datastore view, select your datastore.

2. You’ll see:

   • Primary region → us-east-1

   • Secondary region → ap-south-1

   • Replication status → “Active”

3. Connect to both clusters using redis-cli:

   # Connect to primary
   redis-cli -h <primary-endpoint> set user:1 "Shanvi"
   OK

   # Connect to secondary
redis-cli -h <secondary-endpoint> get user:1
"Shanvi"


✅ Data is successfully replicated globally!

---

🔄 Step 4: Test Failover

You can simulate a failure to verify automatic promotion.

1. Stop or delete the primary cluster.

2. The secondary region cluster is promoted as new primary.

3. Check the new write access:

   redis-cli -h <secondary-endpoint> set user:2 "NewPrimary"
OK


4. Once the original region recovers, you can add it back as a secondary.

---

🧪 Step 5: Add Another Global Region (Optional)

You can add more read replicas for global reach.

1. Go to Global Datastore → your datastore → Add region

2. Choose another region (e.g., eu-west-1)

3. Click Add

✅ Redis now replicates data to multiple continents.

---

🧰 Step 6: Monitor Replication & Metrics

Use Amazon CloudWatch:

• ReplicationLag → shows delay between regions

• CurrConnections → number of active connections

• BytesUsedForCache → memory utilization

---

🧠 Important Notes

---

💡 Use Case Example

Use Case:
A global e-commerce app with customers in USA, India, and Europe.

• Deploy Primary Redis in us-east-1

• Deploy Read Replica in ap-south-1 and eu-west-1

• Users in Asia read cached product data locally (low latency)

• If the U.S. region fails → Mumbai becomes the new primary

---

✅ Summary