Part 1: How DRBD Works
DRBD is a block device that can be used within high availability (HA) setups. It functions similarly to a network RAID-1. When you write data to the local filesystem, the data is also sent to another host over the network and recorded identically on a filesystem there. Data on the local (primary) node and the remote host (secondary node) is guaranteed to be synchronized in real time. When the local system fails, the remote host still retains an identical copy of the data and can continue to serve. Using DRBD in HA environments can replace a shared disk array. Because data exists simultaneously on both the local and remote hosts, during a failover, the remote host can simply use its backup copy to continue providing services.
Part 2: Environment Configuration
1. Operating System: RHEL 5.4
2. Primary Server:
Hostname: master
IP: 192.168.2.124
Mirror Disk: /dev/hdb
3. Backup Server
Hostname: slave
IP: 192.168.2.125
Mirror Disk: /dev/hdb
Part 3: Installing DRBD
1. Download the source code from http://oss.linbit.com/drbd/. The version installed here is drbd-8.3.5.
2. Extract the downloaded drbd-8.3.5.tar.gz
[root@master ] # tar –zxvf drbd-8.3.5.tar.gz
[root@master ] # cd drbd-8.3.5
[root@master ] # make
[root@master ] # make install
[root@master ]# modprobe drbd @ Load the DRBD kernel module
[root@master ]# lsmod |grep drbd
drbd 271736 0
@ Check if it was successful using lsmod. If output similar to the above appears, DRBD is installed successfully.
[root@master ]# mknod /dev/drbd0 b 147 0 @ Create the drbd block device
Part 4: Configuring DRBD
When DRBD runs, it reads the configuration file /etc/drbd.conf. This file describes the mapping between DRBD devices and hard disk partitions. Once data is written to disk and sent over the network, the write operation is considered complete.
1. drbd.conf Configuration Parameter Descriptions
Protocol
Protocol A @ Considers a write complete once data is written to disk and sent to the network.
Protocol B @ Considers a write complete once a receive acknowledgment is received.
Protocol C @ Considers a write complete once a write confirmation is received.
2. global
global { usage-count yes; } @ Whether to participate in DRBD user statistics. Default is yes.
3. common
common { syncer { rate 1M; } }
@ Sets the maximum network rate for synchronization between primary and secondary nodes, in bytes.
4. resource
A DRBD device (i.e., /dev/drbdX) is called a “resource”. It contains the IP info for the primary and secondary nodes, underlying storage device name, device size, meta info storage method, the device name DRBD presents to the system, etc.
resource r0 {
protocol C; @ Uses protocol C, meaning after receiving a write confirmation from the remote host, the write is considered complete.
net {
cram-hmac-alg sha1; @ Sets the message authentication algorithm used for communication between hosts.
shared-secret “FooFunFactory”;
}
@ Each host description starts with “on” followed by the hostname. Inside the following {} is the configuration for that host.
on master {
device /dev/drbd0;
disk /dev/ hdb;
address 192.168.2.124:7898; @ Sets the listening port for DRBD, used to communicate with the other host
meta-disk internal;
}
on slave{
device /dev/drbd0;
disk /dev/ hdb;
address 192.168.2.124:7898;
meta-disk internal;
}
}
5. The following is the complete configuration used for testing
global {
usage-count yes;
}
common {
protocol C;
}
resource r0 {
on master {
device /dev/drbd0;
disk /dev/hdb;
address 192.168.2.124:7789;
meta-disk internal;
}
on slave {
device /dev/drbd0;
disk /dev/hdb;
address 192.168.2.125:7789;
meta-disk internal;
}
}
6. The configuration file is identical on both nodes in a two-node cluster. There is a sample in the DRBD source directory with detailed explanations.
Part 5: Starting DRBD
1. Creating Metadata
Before starting DRBD, you need to create metadata blocks on the hdb partitions of both hosts. Execute on both hosts respectively:
[root@master ]#drbdadm create-md r0
[root@slave ]#drbdadm create-md r0
Notes:
1) “r0” is the resource name defined in drbd.conf.
2) If the following error appears when running the “drbdadm create-md r0” command:
Device size would be truncated, which
would corrupt data and result in
‘access beyond end of device’ errors.
You need to either
* use external meta data (recommended)
* shrink that filesystem first
* zero out the device (destroy the filesystem)
Operation refused.
Command ‘drbdmeta 0 v08 /dev/xvdb internal create-md’ terminated with exit code 40
drbdadm create-md r0: exited with code 40
Solution: Initialize the disk format, e.g., dd if=/dev/zero bs=1M count=1 of=/dev/sdXYZ; sync
[root@master ]# dd if=/dev/zero bs=1M count=1 of=/dev/hda3; sync
2. Start DRBD. Execute on both hosts respectively:
[root@master ]# /etc/init.d/drbd start
[root@slave ]# /etc/init.d/drbd start
Note:
If a node cannot be found during startup, check if /dev/hda3 is mounted. If so, unmount it using the umount command.
3. Check DRBD status. Execute on both hosts respectively:
[root@slave drbd]# cat /proc/drbd
Explanation of the output:
ro indicates the role. When DRBD starts for the first time, both DRBD nodes are in the Secondary state by default.
ds is the disk state. “Inconsistent/Inconsistent” means the disk data on the two nodes is in an inconsistent state.
Ns represents network sent packet info.
Dw represents disk write info.
Dr represents disk read info.
4. Setting the Primary Node
As there is no primary/secondary distinction by default, you need to designate them. Choose the host you want to make primary and run the following command:
[root@master]#drbdsetup /dev/drbd0 primary –o
Alternatively, the following command also works:
[root@ master]#drbdadm — –overwrite-data-of-peer primary all
After the first time you run this command, for subsequent promotions to primary, you can use another command:
[root@ master]#/sbin/drbdadm primary r0 or /sbin/drbdadm primary all
Check the DRBD status again.
You can see the status is now primary/secondary, and the disk info is UpToDate/Inconsistent. Synchronization of data on the corresponding disks between the two machines has started.
After a while, check the DRBD status again. It looks like this:
Now the status is primary/secondary, and the disk info is UpToDate/UpToDate, indicating that synchronization is complete.
Part 6: Testing Synchronization
Mount the DRBD device on the primary host to a directory for use. The DRBD device on the backup host cannot be mounted, as it is used to receive data from the primary and is managed by DRBD.
There are two methods for primary-standby failover: discontinuously by stopping the DRBD service, or normally. Both are described below:
1. Failover by Stopping DRBD Service
Stop the primary node’s service. The mounted DRBD partition will automatically unmount from the primary node. Then, execute the switch command on the standby node:
[root@drbd2 ~]#drbdadm primary all
This will result in an error:
2: State change failed: (-7) Refusing to be Primary while peer is not outdated
Command ‘drbdsetup 2 primary’ terminated with exit code 11
Therefore, you must execute the following on the standby node:
[root@drbd2 ~]#drbdsetup /dev/drbd0 primary –o
or
[root@drbd2~]#drbdadm — –overwrite-data-of-peer primary all
Now the switch will work normally. After executing this command on the standby node, the original primary node automatically becomes the standby node. There is no need to execute the switch-to-standby command on the original primary node again.
2. Normal Failover
Unmount the disk partition on the primary node, then execute:
[root@drbd1 ~]#drbdadm secondary all
If you do not run this command and directly execute the switch-to-primary command on the standby node, you will get this error:
2: State change failed: (-1) Multiple primaries not allowed by config
Command ‘drbdsetup 2 primary’ terminated with exit code 11
Next, execute on the standby node:
[root@drbd2 ~]#drbdadm primary all
Finally, mount the disk partition on the standby node:
[root@drbd2 ~]#mount /dev/drbd0 /mnt
3. Key Points to Note
1) Before mounting a DRBD device, you must switch it to the primary state.
2) Between the two nodes, only one can be in the primary state at a time, while the other is in the secondary state.
3) You cannot mount the DRBD device on a server in the secondary state.
4) The two synchronized partitions on the primary and backup servers should ideally be the same size to avoid wasting disk space, since DRBD mirroring is essentially a network RAID 1.
Refer to related documentation:
http://www.cnblogs.com/feisky/archive/2011/12/25/2310346.html
http://lucklong.blog.51cto.com/23691/591804
http://blog.sina.com.cn/s/blog_68a062130100jk1k.html
http://www.ixdba.net/a/os/linux/2010/1011/795.html