On 07/28/2016 08:14 AM, dafrk wrote:
> Hi experts,
> we are having two data centers. On each data center site there is a SAN
> and many different virtual machines whose file systems are running on
> several LUNs of these SANs.

.... lots of confusing stuff removed.

So.. I think you want mirrored storage off two different storage devices (and
possible each in a different SAN).

Used mdadm to create a mirrored storage block device, each LUN of the mirror is
one from each storage device. To grow this I highly recommend you consider LVM
concatentation, in other words create new mirrored sets and add the mdadm block
device into the LVM VG and extend the LV and grow the fs.

Ideally what will happen is the multipaths for the failed storage will go down
and thus that storage goes down like a disk failure.

When that disk returns, ideally you'd hope it would be like a disk replace and
the mirror would resilver the returned LUN. Now... if that doesn't happen,
then perhaps the returned LUN just stays "bad". If it somehow comes back "good"
as a part of the mirror without rebuild, something is wrong and it will be

LVM should be none the wiser.

Latencies could be a big problem if the devices are remote and different.
iSCSI sounds like a good idea (route your "disk connection" across high latency
routes, etc), but the variables are immense, and you've sort of more than
doubled the amount of variables with your scenario.

Hope you find an acceptable solution. IMHO, one day something really bad will
happen and it won't be recoverable. I recommend you do some very expensive and
high risk testing of your environment if you're able to just so your management
doesn't get any unrealistic expectations.

I've only done (I think) what you're suggesting using two storage devices in the
same rack using fibre channel. While there's still some risk, it's much lower
since it's fibre channel and very little latency issues (all local).

Other answers... IBM makes a sort of "abstractor" device that can mirror across
mixed vendor devices that sit behind it. Of course, IBM will tell you that you
need at least 3 of these so that you have one sitting in front of two... etc..
(a backup of a backup of a backup sort of thing). Nice part about these devices
is that to you it just looks like a single storage device, no mdadm involved at all.

I used to work for IBM and they are absolute evil (anti-FOSS), but if interested: