terça-feira, 11 de abril de 2023

[MongoDB] How the replication works in MongoDB? And how to create a Test Environment using Docker-Compose!!!

Hello all

How are you doing?

Today, I'm going to talk about replication in MongoDB and how I can create a Test Environment using Docker-Compose.

If you don't know how to install a docker-compose, click here and here.

And always remember:
1) This step-by-setp worked for me, but it might not work for you.
2) It's a basic install. As we go along, we'll get better and better.
3) This post is just for study and test, and it has no concern for Performance and Security Best Practices.

According to the manual, "The Replica Set in MongoDB is a group of mongod processes that maintain the same data set. Replica sets provide redundancy and high availability and are the basis for all production deployments. Replication provides redundancy and increases data availability. With multiple copies of data on different database servers, replication provides a level of fault tolerance against the loss of a single database server."

It's important understand that the secondaries replicas aren't open to write operation. But we can specify a read preference to send read operations to secondaries, if we want. 

If you want active nodes, you can use Sharding which are a method for distributing data across multiple machines, but this is a conversation to other article.

This is a basic schema about replication in MongoDB.



For learn more about replication, click on image above.

If you understood this, let's get started!!!

For my test, I'll create 3 containers running on AWS EC2 instance using Amazon Linux 2. The data will be persistent because I'll use the "Docker-Managed Volume" for this. 

These automation scripts were based on these original scripts here!!!

1) On compose directory, I'll create an archive call "docker-compose-replicaset.yml". 
Important: It's forbidden to use "TAB" for indentation.

version: '3.9'

services:
  mongodb01:
    container_name: mongodb01
    image: mongo:6
    volumes:
      - shared_database:/data/mongodb
      - /root/compose/scripts/rs-init.sh:/scripts/rs-init.sh
    networks:
      MongoCluster:
        ipv4_address: 172.15.0.11
    ports:
      - 27021:27017
    restart: always
    entrypoint: ["/usr/bin/mongod", "--bind_ip_all", "--replSet", "myRS"]
  mongodb02:
    container_name: mongodb02
    image: mongo:6
    volumes:
      - shared_database:/data/mongodb
    networks:
      MongoCluster:
        ipv4_address: 172.15.0.12
    ports:
      - 27022:27017
    restart: always
    entrypoint: ["/usr/bin/mongod", "--bind_ip_all", "--replSet", "myRS"]
  mongodb03:
    container_name: mongodb03
    image: mongo:6
    volumes:
      - shared_database:/data/mongodb
    networks:
      MongoCluster:
        ipv4_address: 172.15.0.13
    ports:
      - 27023:27017
    restart: always
    entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "myRS"]

volumes:
  shared_database:

networks:
  MongoCluster:
    driver: bridge
    ipam:
     config:
       - subnet: 172.15.0.0/16

The main information about this script are:
mongodb01 = Name of service
container_name = Container (machine) name
image = Docker image. I'll use the MongoDB official image 
volumes = Persistent volumes that will be shared in all nodes
networks = Network using to the cluster 
ports = Service ports and port range 
entrypoint = Override the default entrypoint. Setting entrypoint both overrides any default entrypoint set on the service’s image with the ENTRYPOINT Dockerfile instruction.

volumes:
  shared_database = Using a "Docker-Managed Volume" for share a Docker-managed volume between containers.

2) Create directory scripts in compose directory.
     mkdir -p /root/compose/scripts

3) Create script "rs-init.sh" in directory scripts

#!/bin/bash
DELAY=25
mongosh <<EOF
var config = {
    "_id": "myRS",
    "version": 1,
    "members": [
        {
            "_id": 1,
            "host": "mongodb01:27017",
            "priority": 2
        },
        {
            "_id": 2,
            "host": "mongodb02:27017",
            "priority": 1
        },
        {
            "_id": 3,
            "host": "mongodb03:27017",
            "priority": 1
        }
    ]
};
rs.initiate(config, { force: true });
EOF

echo "****** Waiting for ${DELAY} seconds for replicaset configuration to be applied ******"
sleep $DELAY
echo "****** Environment ready to use ******"

4) Create the script "startRS.sh" on compose directory.

#!/bin/bash
DELAY=10

## Cleanup older creations
docker-compose --file docker-compose-replicaset.yml down
docker rm -f $(docker ps -a -q)
docker volume rm $(docker volume ls -q)

## Start Docker-compose using my configs
docker-compose --file docker-compose-replicaset.yml up -d

echo "****** Waiting for ${DELAY} seconds for containers to go up ******"
sleep $DELAY

## Start Replica Set just on the first execution. After that, a warning will be displayed.
docker exec mongodb01 /scripts/rs-init.sh

## Check
docker ps
docker exec -it mongodb01 /bin/hostname -i
docker exec -it mongodb02 /bin/hostname -i
docker exec -it mongodb03 /bin/hostname -i
docker exec -it mongodb01 mongosh --eval "rs.status()"

4) Run chmod to to the scripts executable
chmod +x starRS.sh
chmod +x scripts/rs-init.sh

If you want to download these scripts, click here.

Finally, we just run the startRS.sh








And now, if you want test the replica set, you can use this:

On Primary (mongodb01):
docker exec -it mongo01 mongosh
rs.stepDown()

On Secondary (mongodb02):
docker exec -it mongo02 mongosh


That's all folks!!!

I hope that this blog helps you!!!

Regards
Mario

Postagem em destaque

[ORACLE] How to move a datafile to a new location in ASM.

Hello everyone! I hope you're doing well! Today, I needed to move a datafile to another ASM Disk Group. The process was simple, but reme...