PDA

View Full Version : SLES 11 SP2 and iSCSI HBA



ashbyj
02-Mar-2012, 16:34
Is there a way to insert/load modules at the boot menu right before boot? I'm not able to get into my root partition to add it to /etc/sysconfig/kernel, so I'm really talking about the boot prompt or boot command line at the main menu before SLES boots. Here are the details of the issue I'm seeing:

I've had no problems doing both an in-place upgrade from SLES 11 SP1 to SP2 and a fresh SP2 install on VMs. However, I've had trouble upgrading and fresh-installing SP2 on a physical machine (Sun Fire x4140).

The in-place upgrade on the physical machine from SP1 to SP2 went smoothly. Upon reboot though, SLES does not see the root partition. We use the iSCSI HBA to boot the OS. The qla4xxx driver seems to load up fine and it is in INITRD_MODULES. Cat /proc/partitions in SP2 rescue mode shows NO partitions, so I can't mount up root. However, /proc/partitions in the SP1 rescue mode shows me root and I can mount it up just fine. FYI I have never had a problem with this in SP1 or pre-SP1.

A fresh install of SP2 on the physical machine had trouble as well. No hard disks were found when it went through the "probing hard disks" step. Adding withiscsi=1 to the boot line before installation took me to the special iscsi config screen right before install, where I took the defaults, and then the iSCSI lun/disk was seen. I was able to complete the install and it booted up fine.

I found two modules that need to be present in SP2 (per lsmod) that were not needed pre-SP2: iscsi_boot_sysfs and libiscsi. Next time I do an in-place install, I'll add these into /etc/sysconfig/kernel and run mkinitrd. However, I've got this one machine that I screwed up that I can't get into the root partition, and so hopefully someone can tell me how to add them at the boot menu to force them in. Thanks!

ashbyj
04-Mar-2012, 16:43
On another note, a few months ago we went through on our SLES 11 SP1 servers and changed the devices to use symbolic links in fstab and /boot/grub/menu.lst instead of the disk-by-id version that is default in SLES 11. This could be part of this issue, but I doubt it. I'll check into it. The reason we changed was due to an issue encountered when moving OS's from one filer to another, and I guess being on a new lun changed its disk id. Anyway, I don't think this is the real issue, but thought I'd mention it.

ashbyj
04-Mar-2012, 21:40
Using /dev/disk/by-id/* in fstab and menu.lst did not solve the problem.

ashbyj
06-Mar-2012, 21:46
Per Qlogic on their forums (https://forums.qlogic.com/posts/fef8cc0bcc):


Unfortunately QLogic has not released any driver for the Linux 3.x kernel, you can try to compile the existing driver Red Hat/SuSE drivers, but they may not even complie on the 3.x kernel.

QLogic Technical Support


SLES 11 SP2 is running the 3.x kernel, so I guess we're dead in the water until they release an updated driver.

ashbyj
23-Apr-2012, 13:13
Nevermind, I was able to get this going on my own. I'm surprised no one else has this issue...

jmozdzen
23-Apr-2012, 15:25
ashbyj,

would you mind sharing how you solved this, just in case someone with the same problem gets directed to your messages via a search?

Thanks & regards,
Jens

ashbyj
24-Apr-2012, 12:08
Certainly. I can't say if all of these steps are necessary in getting your machine to boot from iSCSI, but here are the steps I take.

1) Perform the in-place upgrade of SLES 11 SP1 to SP2


Yast
Software > Patch CD Update
I usually go with NFS and then point to the path of my SP2 ISO

After the upgrade you need to configure the iSCSI device at the OS level and also compile some special iSCSI-related modules into the initrd.

2) Configure iSCSI device via yast

Some of this is specific to our environment



Yast
Network Services > iSCSI Initiator

Service Start: when booting
Initiator name: iqn.2009-11.com.mydomain.myhost:qlogic.qla4060c.1
Offload card: default (software)



I'm really not sure if this below was necessary, but I was matching the settings on a machine where we were successful in installing a fresh SP2:


# via command line
cd /etc/iscsi/ifaces

# add ipv4.0 suffix to the qla4xxx* iface file
mv qla4xxx.00\:c1\:aa\:0e\:cf\:ab qla4xxx.00\:c1\:aa\:0e\:cf\:ab.ipv4.0

vi eth0-qla4xxx
# added this line which is the iSCSI initiator client IP
iface.ipaddress = 172.12.211.2



3) add modules to initrd


vi /etc/sysconfig/kernel

# change initrd_modules line to:
INITRD_MODULES="sata_nv ata_generic iscsi_boot_sysfs bnx2i"

# The two most important modules above I believe to be iscsi_boot_sysfs AND bnx2i
mkinitrd

If you encounter perl-bootloader errors when running mkinitrd (http://www.novell.com/support/viewContent.do?externalId=7009728&sliceId=1), do these steps:


cd /boot/grub/
mv device.map device.map.old
grub
# At the grub> prompt, type quit
# Rerun mkinitrd, it should no longer have the perl-bootloader errors.


4) miscellaneous steps


# We disable nscd on our servers, and SP2 seems to reenable it.
/etc/init.d/nscd stop
chkconfig nscd off

rm /etc/ssh/ldap.conf # not sure why ldap.conf gets put in /etc/ssh after SP2 upgrade... could just be me.

5) Reboot and cross your fingers

We've never had to configure iSCSI at the software level before, so I'm not sure what is different about the bootloader that doesn't play nicely with our qlogic card.

jmozdzen
24-Apr-2012, 21:36
Certainly. I can't say if all of these steps are necessary in getting your machine to boot from iSCSI, but here are the steps I take.

Thanks for your detailled response and your time to write it! From my own experience I'm sure there will be more than one "just-reader" that will be grateful to find you description and be able to fix his/her problem thanks to it.

Many regards
Jens

ashbyj
25-Jun-2012, 19:43
Novell/SuSE finally published some information about the change to the way Qlogic iscsi is handled, 3 months after SP2 was released :(. Anyway, If you came looking for help on this, my notes above may be helpful when accompanied with their doc:

How to configure and run QLogic Open-iSCSI on SLES11 SP2 (http://www.suse.com/support/kb/doc.php?id=7008181)

FYI



Prior to SLES11 SP2, discovery and login to iSCSI targets was hardware offloaded and managed by the QLogic iSCSI adapter firmware. In SLES11 SP2, the Open-iSCSI infrastructure allows system administrator to manage QLogic iSCSI target discovery and login through QLogic adapter. This provides the system administrator with a uniform and standard way to manage different vendor interfaces.

Open-iSCSI is enabled by default on SLES11 SP2 and login to boot targets is triggered by iscsiadm during installation.


My notes above show you how to offload the iscsi work onto the software (now the default in SP2), but their doc does show how to offload onto hardware (done by default pre-SP2), but I have not tried the latter.