LVM Mounting Issue at Sandhar Zimbra Archival Server while NAS Firmware Update

ISSUE Reported

LVM Failed to Mount on /opt After Hardware Vendor Updated NAS/SAN Firmware

/dev/ArchiveEMC_VG/NewArchiveEMC_lv01: read failed after 0 of 4096 at 2147483582464: Input/output error
/dev/ArchiveEMC_VG/NewArchiveEMC_lv01: read failed after 0 of 4096 at 2147483639808: Input/output error
/dev/ArchiveEMC_VG/NewArchiveEMC_lv01: read failed after 0 of 4096 at 0: Input/output error
/dev/ArchiveEMC_VG/NewArchiveEMC_lv01: read failed after 0 of 4096 at 4096: Input/output error

Also /var/log/messages showed

Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 0
Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 1
Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 2
Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 3
Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 0
Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 524287999
Jan 30 02:53:11 archive kernel: Buffer I/O error on device dm-2, logical block 524287999

Fixing SANDHAR LVM I/O Errors

There is allrady one zimbra in /opt So make sure that zimbra should off before mounting LVM .Because I have done this mistake.

Solution :

1) Check which Volume Group have the issue , run “vgscan” command .

# vgscan

Showed the Shown Errors

2) Find out the Logical Volumes attached with that Volume Group .

3) Inactivate the logical volumes as :

#lvchange -an /dev/ArchiveEMC_VG/NewArchiveEMC_lv01

4) Inactivate Volume group as :

#vgchange -an /dev/ArchiveEMC_VG

5) Again Scan Volumegroup using “vgscan” .

6) Now activate the Volume Group :

#vgchange -ay /dev/ArchiveEMC_VG

7) Run command “lvscan” , the error should be gone now .

8) Now activate the Logical Volume Name :

#lvchange -ay /dev/ArchiveEMC_VG/NewArchiveEMC_lv01

Now mount LVM in opt

#mount /dev/ArchiveEMC_VG/NewArchiveEMC_lv01 /opt