PDA

View Full Version : SLES 12 SP2 IO error at virtual disk



sharfuddin
08-Oct-2018, 13:08
Its a VM(SLES 12 SP2) running on ESX 6.5. ESX is not showing any error, while at OS we are getting following errors:

kernel: [787030.198763] end_request: I/O error, dev sda, sector 3759139416
kernel: [1530802.448197] end_request: I/O error, dev sda, sector 3754016448
kernel: [1912330.306074] end_request: I/O error, dev sda, sector 3719306720
kernel: [1954566.577314] end_request: I/O error, dev sda, sector 3722956392

The disk(sda) is a virtual disk.

Please help, what and where should I check ?

jmozdzen
08-Oct-2018, 13:35
Hi sharfuddin,

> kernel: [787030.198763] end_request: I/O error, dev sda, sector 3759139416
> Please help, what and where should I check ?

are those reported sector numbers within the virtual disk's size? If not, those errors could be the result of a on-disk corruption (invalid sector numbers/offsets stored in whatever you put on that disk).

Everything else depends on how that virtual disk is provisioned... is it from some VMFS pool? SAN? If it's some back-end storage system, does it report any errors?

Regards,
J

sharfuddin
08-Oct-2018, 19:04
Hi sharfuddin,

> kernel: [787030.198763] end_request: I/O error, dev sda, sector 3759139416
> Please help, what and where should I check ?

are those reported sector numbers within the virtual disk's size? If not, those errors could be the result of a on-disk corruption (invalid sector numbers/offsets stored in whatever you put on that disk).

Everything else depends on how that virtual disk is provisioned... is it from some VMFS pool? SAN? If it's some back-end storage system, does it report any errors?

Regards,
J

>are those reported sector numbers within the virtual disk's size?
>
How can I find this.. via "fdisk -l" ? Actually don't have access to the system right now.

>is it from some VMFS pool?
Yes.

>SAN?
No, locally attached disk on ESX host

jmozdzen
09-Oct-2018, 10:54
Hi sharfuddin,


>are those reported sector numbers within the virtual disk's size?
>
How can I find this.. via "fdisk -l" ? Actually don't have access to the system right now.

yes, can be a good information source. Have a look at the start and end sector information for your partitions.


>SAN?
No, locally attached disk on ESX host

You wrote that ESX had nothing in the logs - did you also look at ESX-system-related logs or just at those directly related to your virtual machine? If the latter, a look at the system logs might reveal information about intermittent disk errors reported by the system. Also, while I do not trust SMART further than I can throw our servers, maybe some SMART pre-failure error indicators may have increased. For below to work, you'll of course need to know which disks are actually used - or if it's hardware RAID, will have to take a totally different approach. Speaking of (hardware or software) RAID: If it's in use, do you give your arrays a check run at least once a week? The older the disks, the more important this can get to detect silently introduced consistency errors (this is, of course, completely unrelated to ESX, but a generally applicable advice).


# esxcli storage core device list
# esxcli storage core device smart get -d device

Regards,
J

KBOYLE
09-Oct-2018, 20:06
sharfuddin wrote:

>
> Its a VM(SLES 12 SP2) running on ESX 6.5. ESX is not showing any
> error, while at OS we are getting following errors:
>
> kernel: [787030.198763] end_request: I/O error, dev sda, sector
> 3759139416
> kernel: [1530802.448197] end_request: I/O error, dev sda, sector
> 3754016448
> kernel: [1912330.306074] end_request: I/O error, dev sda, sector
> 3719306720
> kernel: [1954566.577314] end_request: I/O error, dev sda, sector
> 3722956392
>
> The disk(sda) is a virtual disk.
>
> Please help, what and where should I check ?

If SLES is reporting these errors, I would do a filesystem check from
within SLES and see what it reports.

--
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below this post.
Thank you.