Tested On
OS: CentOS 6.3 i386
mdadm version: v3.2.3
Hardware: Virtual Machine (VirtualBox 4.1.22)
Introduction
mdadm is a linux software raid implementation. With mdadm you can build software raid from different level on your linux server. In this post I will show how to create a raid 10 array using 4 disks.
Raid 10 is stripe of mirrored disks, it uses even number of disks (4 and above) create mirror sets using disk pairs and then combine them all together using a stripe. Raid 10 have a great fault tolerance because of its mirror pairs and very good performance because of the stripe technology.
Creating partitions
- First we will create linux raid partitions on our disks
fdisk /dev/sdb
Command (m for help):
n -> enter
Command action e extended p primary partition (1-4)
p -> enter
Partition number (1-4):
1 -> enter
First cylinder (1-1044, default 1):
enter
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1044, default 1044):
+3G enter
Command (m for help):
t
Selected partition 1 Hex code (type L to list codes):
fd
Changed system type of partition 1 to fd (Linux raid autodetect) Command (m for help):
w
The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
- Continue creating partitions by repeating the above procedure on all the other disks that will participate in the raid
- Create new raid 10 using the partitions we created
yum install mdadm -y mdadm --create /dev/md0 --level raid10 --name data --raid-disks 4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 echo "MAILADDR root@localhost" >> /etc/mdadm.conf mdadm --detail --scan >> /etc/mdadm.conf
- Create new file system on the new raid device
mkfs.ext4 /dev/disk/by-id/md-name-centos-62-1.localdomain:data mkdir /data mount /dev/disk/by-id/md-name-centos-62-1.localdomain:data /data
- Change /etc/fstab if you want to automatically mount the raid device. Use the device id in fstab and not /dev/md0 because it may not be persistent across reboot
vi /etc/fstab
... /dev/disk/by-id/md-name-centos-62-1.localdomain:data /data ext4 defaults 1 1
- Reboot the system to check that raid 10 is automatically started and mounted after a reboot
reboot
Testing Our New Raid 10
Here will test two disks failures in our raid. Because raid 10 use mirror sets it will continue to function when it has faulty disks from different mirror sets.
- Check the array status
mdadm --detail /dev/disk/by-id/md-name-centos-62-1.localdomain:data
- Simulate disk sdb1 failure
mdadm --manage --set-faulty /dev/disk/by-id/md-name-centos-62-1.localdomain:data /dev/sdb1
- Check syslog for new failure messages
tail /var/log/messages Oct 3 16:43:42 centos-62-1 kernel: md/raid10:md0: Disk failure on sdb1, disabling device. Oct 3 16:43:42 centos-62-1 kernel: md/raid10:md0: Operation continuing on 3 devices.
- Check array status
mdadm --detail /dev/disk/by-id/md-name-centos-62-1.localdomain:data cat /proc/mdstat
- Simulate disk sdd1 failure
mdadm --manage --set-faulty /dev/disk/by-id/md-name-centos-62-1.localdomain:data /dev/sdd1
- Check syslog for new failure messages
tail /var/log/messages Oct 3 16:45:01 centos-62-1 kernel: md/raid10:md0: Disk failure on sdd1, disabling device. Oct 3 16:45:01 centos-62-1 kernel: md/raid10:md0: Operation continuing on 2 devices.
- Check array status
mdadm --detail /dev/disk/by-id/md-name-centos-62-1.localdomain:data cat /proc/mdstat
- Remove sdb1 from the array and re-add it
mdadm /dev/disk/by-id/md-name-centos-62-1.localdomain:data -r /dev/sdb1 mdadm /dev/disk/by-id/md-name-centos-62-1.localdomain:data -a /dev/sdb1
- Check array status
mdadm --detail /dev/disk/by-id/md-name-centos-62-1.localdomain:data cat /proc/mdstat
- Remove sdd1 from the array and re-add it
mdadm /dev/disk/by-id/md-name-centos-62-1.localdomain:data -r /dev/sdd1 mdadm /dev/disk/by-id/md-name-centos-62-1.localdomain:data -a /dev/sdd1
- Check array status
mdadm --detail /dev/disk/by-id/md-name-centos-62-1.localdomain:data cat /proc/mdstat
Please visit https://raid.wiki.kernel.org for more information about linux software raid configuration and usage.