One of my hard disks on my Fedora Core 6 server nearly failed yesterday. It sounded like it loses power every now and then. This hard disk is my primary, it has the /boot partition and an LVM partition. It holds at least 36GB of the 300GB+ LVM Volume Group. Had it died totally, then most of the OS would have been gone along with some of my data. Luckily I was still able to boot from this nearly-dead hard disk for a couple of times.

I downloaded Fedora Core 6’s Rescue CD ISO and burned it. Every time I could boot into the system without a problem, I rebooted immediately and booted from the Rescue CD. I was hoping that I could move the LVM PEs off the broken hard disk ASAP.

During my first attempt, the faulty hard disk ‘disappeared’ when I was running e2fsck. I had to shut the system down for about 15 minutes to let it cool down. This trick did work and I tried another attempt. This time e2fsck finished without a problem, and I ran pvmove to move the PEs from the faulty disk. Unfortunately my kernel is the latest version but the device-mapper and lvm2 packages are not. pvmove printed out errors (“device-mapper: reload ioctl failed: Invalid argument”) no matter how many times I tried. Initially I thought that the faulty hard disk may be too damaged, but then since I could still boot the system without a hitch so I guessed it couldn’t be that damaged.

This post has a solution, but I didn’t use it. I downloaded Fedora 8 Rescue CD ISO and used that instead of Fedora Core 6’s. This time pvmove didn’t show any error and the process completed as expected without any lost data. I was then able to vgreduce the faulty hard disk from the LVM Volume Group.

If you experience the same problem, try downloading a newer Rescue CD and give it a try. Hopefully it will address problems that are present on older Rescue CDs. Good luck! 🙂